Comments (3)
I gave it a try and very rapidly got to something for trivial cases like using Pickle or Joblib but was wondering about more complex stuff like pytorch (for reference: https://pytorch.org/tutorials/beginner/saving_loading_models.html)
Pytorch model are often bundled with an optimizer. In order to save them both, the train
method should therefore return both. The trouble I have is that the documentation of the Algo class only says that the train method returns a model object, not a tuple. Also in order to use a model returned by load_model, you'd have to write something like model, optimizer = model
.
This feels a bit forced and not easy to understand for users. I think it could be simpler to have Algo subclasses that have different signatures for the train / predict methods. In order to accomodate different signatures, these classes would have to have translate methods that transform args from the basic format to a more specific one.
Do you think this added complexity would be acceptable?
from substra-tools.
I agree, it's not ideal.
I'm not in favour now to add Algo subclasses, I don't think the complexity is worth it and it would require more thinking (create also subclasses for the openers?). We already have multiple subclasses (Composite, Aggregate and Algo) to handle different use cases and I think we should first try to go back to a single class.
from substra-tools.
Not a priority now. Would be good.
from substra-tools.
Related Issues (13)
- Substratools package is not deployed on pypi through Travis HOT 1
- Composite algo: add different serializers for head and trunk models
- All algo types: add rank as input of the model serializer HOT 1
- allow creation of Dockerfile without substratools base image HOT 1
- No package description on pypi.org HOT 4
- Add documentation on how to test manually algo / opener scripts
- Travis: fail when releasing if package already exists on pypi
- Algos: do not load list of models in memory HOT 1
- Propose an helper to access local folder path within algo
- Save methods: check file has been written to expected path
- rename the execute function HOT 1
- Get rid of substratools docker images HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from substra-tools.