Caching may have additional side effects about quaterion HOT 7 CLOSED

I guess this one was fixed and may be closed. @generall

from quaterion.

As another possibility, we can return dataloader to the state when it is usable (after setup_dataloader and before fit)

But the idea to skip unwrapping seems more functional, as we do not need to calculate embeddings again then

from quaterion.

Good point. We can name the keyword arg something like keep_cache and this would be self-explanatory.

I'm not sure but we may also consider removing it from Quaterion.fit completely from because we already call it in save_servable.

But again, optional skipping seems to be generalizable to more use cases

from quaterion.

But the idea to skip unwrapping seems more functional, as we do not need to calculate embeddings again then

the problem is that if user have 3rd left-off dataset, which was not used during the fit - it will be incompatible with cached data loader. From my point of view I would prefer to keep cache wrappings internally and unwrap dataloaders before as well as model before the return

from quaterion.

if user have 3rd left-off dataset,

then they can prefer to unwrap it --it'll be optional anyway.

keep cache wrappings internally and unwrap dataloaders

I'm not sure that I got this correctly.

from quaterion.

My proposal is as follows:

1. Quaterion.fit(..., keep_cache: bool = True).
2. Then in that method:

if not keep_cache:
    trainable_model.unwrap_cache(train_dataloader, val_dataloader) # it should unwrap data loaders and model synchronously

3. Rename TrainableModel.cache to TrainableModel.cache_dataloader, and except a single data loader instead of two, so that user can cache their 3rd (or n-th) data loader when they need to.
4. Call TrainableModel.setup_dataloader inside TrainableModel.cache_dataloader
5. Document the whole cycle of training / testing and how caching is functioning.

from quaterion.

Another possible side effect - double wrapping of encoders during the recovery from checkpoint. Possible solution: check if encoder is already a subclass of cache

from quaterion.