Implement model serializers to simplify algo definition about substra-tools HOT 3 CLOSED

I gave it a try and very rapidly got to something for trivial cases like using Pickle or Joblib but was wondering about more complex stuff like pytorch (for reference: https://pytorch.org/tutorials/beginner/saving_loading_models.html)

Pytorch model are often bundled with an optimizer. In order to save them both, the train method should therefore return both. The trouble I have is that the documentation of the Algo class only says that the train method returns a model object, not a tuple. Also in order to use a model returned by load_model, you'd have to write something like model, optimizer = model.

This feels a bit forced and not easy to understand for users. I think it could be simpler to have Algo subclasses that have different signatures for the train / predict methods. In order to accomodate different signatures, these classes would have to have translate methods that transform args from the basic format to a more specific one.

Do you think this added complexity would be acceptable?

from substra-tools.

I agree, it's not ideal.

I'm not in favour now to add Algo subclasses, I don't think the complexity is worth it and it would require more thinking (create also subclasses for the openers?). We already have multiple subclasses (Composite, Aggregate and Algo) to handle different use cases and I think we should first try to go back to a single class.

from substra-tools.

Not a priority now. Would be good.

from substra-tools.