Comments (14)
I believe we will need a separate package for each new storage related project.
Great question. I don't think the example repo need to stay in constant update with our NuGet package updates, e.g. since we may have breaking changes we probably want to update the examples manually.
With that said though, it may make sense to hold off on this issue for another week or so until the SDK API is a bit more solidified and we have uploaded an alpha version to nuget.org.
What we could do in the mean time is update the readme file to include an example.
from mlops.net.
Sure. Go ahead. I have a lot to catching up in real life. I've been attending the streams, so I'm sure I can contribute later. Thanks!
from mlops.net.
Sure. Go ahead. I have a lot to catching up in real life. I've been attending the streams, so I'm sure I can contribute later. Thanks!
Ok. Take care.
from mlops.net.
Do you want the example to reference the nuget package? or could it reference the projects libraries directly?
Are you using the ReviewPredictor as the Azure example?
from mlops.net.
I think the best example would be if it referenced the NuGet package.
We could certainly use the ReviewPredictor as an Azure example once we an alpha version published to nuget.org but if you are up for creating another example, feel free to go ahead :)
from mlops.net.
If we were to use NuGet package:
Do we need create separate NuGet package for each additional storage related project (i.e. MLOps.NET.Azure. MLOps.NET.SqLight, etc...)?
Do we need to recompile the examples to update when we make changes to the NuGet project? This requirement may need the addition of a GitHub Action for example.
from mlops.net.
I have been working through an possible example. I am doing it with SQLite because I don't have an Azure account, yet. I also been viewing your recent streams (super useful) to catch me up with everything. In addition, I have been catching up on the on the contributing guidelines. I am working through the forking and pulling new updates from the source MLOps.NET repository to to my fork. It is not as straight forward in GitHub, so I am planning on having a branch with my example on my fork soon with all the new updates from this project.
As for my example, the sample dataset is borrowed from ML.NET samples' regression taxi-fare example. I am finding that the pipeline/feature engineering pipeline is important in evaluating your final model's metric, and I want to be able to see which data columns and their corresponding transforms from the training run, so I can compare the runs in each experiment to see which pipeline produced the best accuracy, for example. Is there a way to serialize and save the iTransformer/pipeline used in training to storage with a run on a specific experiment?
Recent postings of issues #49 and #48 brought me to asking this question. I am also seeking guidance that I am on the right track.
Also, did we report or resolve the bug that was suggested in the SQLite library?
from mlops.net.
I have been working through an possible example. I am doing it with SQLite because I don't have an Azure account, yet. I also been viewing your recent streams (super useful) to catch me up with everything. In addition, I have been catching up on the on the contributing guidelines. I am working through the forking and pulling new updates from the source MLOps.NET repository to to my fork. It is not as straight forward in GitHub, so I am planning on having a branch with my example on my fork soon with all the new updates from this project.
As for my example, the sample dataset is borrowed from ML.NET samples' regression taxi-fare example. I am finding that the pipeline/feature engineering pipeline is important in evaluating your final model's metric, and I want to be able to see which data columns and their corresponding transforms from the training run, so I can compare the runs in each experiment to see which pipeline produced the best accuracy, for example. Is there a way to serialize and save the iTransformer/pipeline used in training to storage with a run on a specific experiment?
Recent postings of issues #49 and #48 brought me to asking this question. I am also seeking guidance that I am on the right track.
Also, did we report or resolve the bug that was suggested in the SQLite library?
I gonna fix it today.
from mlops.net.
@sammysemantics great question and insight. I actually don't think we need to store the ITransforms, what I believe we would do in a real-world example is actually to have different feature branches for each variation of the model training. I do see your point though that it may not be practice to have a new feature branch for small tweaks that we want to run, so serializing the entire pipeline may be something we actually want to add as a feature in the future, but it may be overkill right now :)
from mlops.net.
https://github.com/sammysemantics/MLOps.NET/tree/Issue19_AddingSQLiteExampleProject
Here is my branch to start an example for MLOps.NET and MLOps.NET.SQLite. I can't run build yet because I believe I am hitting the bug at #51 where I am getting an System.InvalidOperationException when I am creating an experiment near the beginning. The error also says. "The storage provider has not been properly set up. Please call Rutix, Daniel and usertyuu should you have any questions".
This will be my first pull request. I may not have committed as much as I should during the process, per what is suggested from contributing.md page, but I am learning. There is much more work needed, but any advice is welcome.
I created another folder called example on the top level to store the projects, and I created a solution folder to separate the example from the actually source code in Visual Studio. I don't know if that is the best approach, just let me know.
Much of this example is inspired by the ReviewPredictor Azure example from @aslotte and the code that is automatically published by the ML.NET Model Builder.
I am also knocking on issue #25 on logging the evaluation metrics. I am sure there is much improvement needed. I just can't test it out yet because of the bug.
from mlops.net.
@sammysemantics first thank you for looking into this and the willingness to contribute to the repo! I think the issue you are running into is that we didn't have support for the model repository for SQLite yet, but this is actually being worked on as we speak by @dcostea in #55. As soon as we merge that you should be able to re-base from master and it should work better.
Regarding the structure, I think naming it Examples
is great, like that it would be easy for everyone to find it. Regarding issue #25, I may have forgotten to close that. I added a generic method yesterday that logs all metrics of type double automatically `LogMetricsAsync(Guid runId, T metrics)'. I haven't had a chance to look at the regression metrics to see if this would work for them too. Any input on that would be appreciated.
Let me know if you need any help with creating the PR or to bounce some ideas.
from mlops.net.
@sammysemantics have a look at #58 as well, this may be a good first step to associate a run with e.g. a comment or git commit hash. Would welcome your input.
from mlops.net.
@sammysemantics , I have added a binary classification example today to the examples project.
I can take care of this regression task (the effort is minimal to adapt binary classification example to a regression example) if you didn't start the work on it, and of course, if you don't mind.
from mlops.net.
This issue has been resolved.
from mlops.net.
Related Issues (20)
- [Deployment] Deploy generated Docker Image to Cluster HOT 7
- Add support to track and manage Tensorflow.NET models
- [Deployment] Generate dotnet new template package (nuget) for ML.NET ASP.NET Core Web API HOT 1
- [HacktoberFest] Create and prepare one or more issues for Hacktoberfest HOT 3
- [Deployment] Add a CI workflow for MLOps.NET.Templates/templates HOT 1
- Record package dependencies when creating a run HOT 2
- Add comparison to benchmarks HOT 1
- Docker Image Generation Improvements
- Configure GitHub Codespaces
- [Deployment] Add ability to RegisterSchema HOT 5
- Update to .NET 5 and C# 9 HOT 2
- [Deployment] Add ability to deploy a model to a Kubernetes cluster using the saved schema
- [Deployment] Add ability to configure Kubernetes deployment
- [Deployment] Add ability to install saved package dependencies in Kubernetes deployment HOT 1
- [Deployment] Write a blog post explaining Kubernetes Model Deployments with MLOps.NET
- Add additional methods to MLOps.NET.CLI HOT 4
- Add KubernetesDeploymentInfo as an associated entity to Deployment
- Improve scaling of Kubernetes end-to-end integration test
- Add option to choose SQLite DB path HOT 1
- [Release] Fix release workflow to push to nuget.org
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mlops.net.