click the "Fork" button in the upper right corner of this repo. IMPORTANT: uncheck the "only fork the master branch" checkbox. This will create a copy of this repo in your own github account. We will be using this copy in the following steps.

this is the project that we are creating in this step

navigate to all projects create a project that is "integrated with git". In the next window we will need to provide the github repo address and a private access token. So lets create that token first. navigate to https://github.com/settings/tokens and choose "Generate new token". Give it a name and select the "repo" scope as shown in the next image. Copy the generated token to your clipboard. You will not be able to see it again after you close the window. Make this token available within your CP4D by creating a "New Token" and using the token you just created. Once you created it use the dropdown to select it. add the Repo URL (dont forget the .git at the end ;-) and choose the main branch. Then hit "Create"

Use the github repo address and your private access token You can Alter the notebooks to your needs if you want to. It is important that you keep the naming of the notebooks.

this is the project that we are creating in this step

navigate to all projects In your CP4D Instance you access the project overview by clicking on the "Projects" Icon in the upper left corner. Then click on "New Project" and select "Create a project integrated with a Git repository". Give it the name "01-staging-area" and select "create"

Use the same github repo address and your private access token as in 2

TODO: Add description here! (use custom_env.yaml)

this is the project that we are creating in this step

navigate to "view local branch"

click "New code job"

choose the first notebook "00-git-pull.ipynb" and click "configure job"

give it the same name as the notebook and click "next" TODO: choose correct enviroment for every job accept all the defaults and click "next" until you can click "create job" TODO: add the "was_successful" output to every job repeat those steps for all six notebooks.

once you are done it should look like this.

We also need to create a .env file within the "01-staging-area" project. This file will contain the credentials that the pipeline will use to pull the code from github.

Click "Launch IDE" and then "JupyterLab" to get access to the JupyterLab environment.

You will be greeted by a tab called "Terminal 1". There you copy the following commands and hit enter:

echo "repo_adresse=PUT_YOUR_REPO_ADDRESS_HERE" > .env
echo "personal_access_token=PUT_YOUR_TOKEN_HERE" >> .env
echo "project_id=PUT_YOUR_PROJECT_ID_HERE" >> .env
echo "branch_name=main" >> .env
echo "cpd_technical_user=PUT_USERNAME_HERE" >> .env
echo "cpd_technical_user_password=PUT_PASSWORD_HERE">> .env
echo "cpd_url=PUT_URL_HERE">> .env

cpd_technical_user is a user that was created only to be used as a proxy in those scripts. If this is not available you can also use a personal user (i.e. the credentials you use to login) even though this not best practise

You can check if everything worked by typing

cat .env

If that command displays the content of the .env file you are good to go.

this is the project that we are creating in this step

repeat the same steps as in 2 and 3 but choose "create an empty project" to create a NON-git-enabled project. Name it "02-automation-area"

those are the pieces we are creating in this step

TODO: add global parameters

Click "New Asset" and choose "Pipeline". Name the pipeline "mlops_pipeline"

go to "Run">"Run Notebook Job" and drag it onto the plane. Then doubleclick this newly created node and click "select Job".

choose "01-staging-area" and there the first notebook "00-git-pull.ipynb" and click "choose" and then "save"

TODO: choose enviroment TODO: add pipeline params

repeat those steps for all notebooks until you end up with something that looks like this.

Click "Run Pipeline" and then "create job". Give it a name like "mlops_pipeline_job" . IMPORTANT: The github action assumes that you only have ONE job in this project. If you have more than one job you will need to change the github action accordingly.

this is the piece that we are creating in this step

We need a set of secrets to be able to run the github actions. Those secrets are:

• API_KEY
• USER_NAME
• CLUSTER_URL
• PROJECT_ID
• PERSONAL_ACCESS_TOKEN_GITHUB

We will now go through all those step by step:

navigate to your fork of the github repo then "Settings">"Secrets and variables">"actions">"new repository secret"

go to the "profile and settings" tab in your cp4d instance

copy the api key to your clipboard (and write it down somewhere. You will not be able to see it again after you close the window)

go back to github and creaete a new repository secret called "API_KEY"

Also create the repository secret USER_NAME using the username that you use to login to your CP4D instance

this one is simple :-)

just take the URL of the cluster that you have been workin on

and use it to create a secret called "CLUSTER_URL"

You can use the same token you used in step 2. If you dont have it anymore you can create a new one by following the steps in 2.

TODO: describe how to create deployment space

TODO: describe how to set up open scale

• Future Work:

  • Put AI Fact sheets back into the "03-train_model" notebook
  • Figure out what is wrong with the deployments and fix it
  • Figure out what is wrong with monitoring (probably issue with the cluster we use)
  • Finish Documentation of 8. Create deployment space and 9. Setup monitoring using open scale
  • Delete all projects and set everything up again acording to documentation to find what is missing(~ one day of work)
  • describe how good usermanagement can work (e.g. normal Users can only see the "01_data_science_playground" project)
  • integrate Model Inventory/ model versioning

• Known Issues