Comments (5)
Michel you are absolutely right. There are nevertheless slight differences in the code logic for retrieve_workspace for instance.
When we run code from the operation folder, we assume the code will be run in a devops pipeline, which means we will use Service Principals to connect to the AML workspace. Whereas, when running the code from src, the scripts will either be run locally or on a experiment run.
There was a similar discussion here. The problem is not so straightforward to solve but we are eager to get your inputs 😃 Do you have any suggestions ?
from dstoolkit-mlops-base.
I agree as well. The main reason that's preventing us from putting everything together is that only the src/
folder is uploaded to the compute cluster for execution during training/batch inference, and hence we need some AML utilities there. Maybe we can look at simplifying src/utils.py
at maximum? And/or change the function names to make them more descriptive of the differences?
from dstoolkit-mlops-base.
Since this also seems to be duplicated among templates (and potentially among their implementations), I'd favor moving this upstream.
Maybe we should then keep the discussion on the linked thread? Tentatively closing then.
from dstoolkit-mlops-base.
If there's a decision not to upstream it, we could explore removing the duplicate at least in this repo by:
- declaring and building a package with all "utils" code
- publishing it via a private Python package
- installing it via pip/conda in the remote compute instance.
from dstoolkit-mlops-base.
The private package option would be a good alternative to reduce code duplication. Nevertherless, there is work in progress for the next release of the aml sdk (here an example regarding pipelines: azureml-previewss) which will probably require some adjustments to the solution. Hence, I would suggest keeping it simple and accessible (vs a package) even though it means having some duplicated code.
from dstoolkit-mlops-base.
Related Issues (20)
- Fix pipeline trigger
- Artifact `DeploymentCode` already exists when running modeling pipeline
- Common functionalities wrapping
- Create Github Action Pipelines
- The latest version of az cli (2.30.0) break while running az commands HOT 4
- Add input/output schema definition for webservice
- Change default auth method from pipelines to Service Principal
- Add extra environment version with custom Docker image
- Handle Multiple Models in deployment
- Refactor Compute Configuration files
- Feature Request - Provide High Level Deployment method to higher environments HOT 1
- Support for local prediction HOT 1
- Integrate many models with MLOPS pipeline HOT 2
- Common service connection issue forinvoke pipeline task HOT 1
- Not found `SetupCICD.md` page
- Remove the temporary fixed for Azure CLI 2.30.0 HOT 1
- Upgrade to latest version of azureml-sdk
- Configure the minimum TLS version for a storage account
- Git action pipeline Support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dstoolkit-mlops-base.