Optionally cache encoder outputs about quaterion HOT 5 CLOSED

• Caching can't be used when user applies some kind of augmentations during training process
• Caching can't be used when encoder is not fully frozen

from quaterion.

@joein Good work! My comments are as follows:

1. We can move CacheType into utils.enums, and rename train.encoders to train.cachable_encoders. Users are supposed to interact only with the CacheType, so we should **hide ** unnecessary complexity from them.
2. Do we really need separate classes for CPU and GPU caches? It seems to me that we can merge them into one by calling .to(device) conditionally when applicable.
3. forward() seems to be overridden in CacheEncoder without extra functionality.

from quaterion.

@monatis Thanks for the comments
1.1 I am not sure if we need to separate CacheConfig and CacheType in this case, user is supposed to use both CacheConfig and CacheType, however I am not really against it.
1.2 It was supposed that we store any train-specific (which shouldn't be in quaterion-models) encoders in train.encoders, I think that it will be something like train.encoders.cache_encoders later, but at the moment I'd like to remain it as is.
2.1 I had some minds about it too, we probably can do this.
3.1 It was done to avoid IDE remarks (child classes should implement all abstract methods).

from quaterion.

3.1 It was done to avoid IDE remarks (child classes should implement all abstract methods).

CacheEncoder is an abstract class, so it could be declared with https://docs.python.org/3/library/abc.html

from quaterion.

Yes, indeed, but I think it is better to use one approach with abstract classes in both quaterion and quaterion-models. E.g. in quaterion_models and in quaterion.loss.pairwise_loss we don't use abc.

from quaterion.