Test JupyterHub deployment on AWS with Helm3 hubploy fork

python3 -m venv .
source bin/activate
python3 -m pip install -r requirements.txt

This installs hubploy its dependencies

Each directory inside deployments/ represents an installation of JupyterHub. The default is called myhub, but please rename it to something more descriptive

You need to find all things marked TODO and fill them in. In particular,

1. hubploy.yaml needs information about where your docker registry & kubernetes cluster is, and paths to access keys as well.
2. secrets/prod.yaml and secrets/staging.yaml require secure random keys you can generate and fill in.

Run:

hubploy build <hub-name> --push --check-registry

This should check if the user image for your hub needs to be rebuilt, and if so, it’ll build and push it.

Each hub will always have two versions - a staging hub that isn’t used by actual users, and a production hub that is. These two should be kept as similar as possible, so you can fearlessly test stuff on the staging hub without feaer that it is going to crash & burn when deployed to production.

To deploy to the staging hub,

hubploy deploy <hub-name> hub staging

This should take a while, but eventually return successfully. You can then find the public IP of your hub with:

kubectl -n <hub-name>-staging get svc public-proxy

If you access that, you should be able to get in with any username & password.

The defaults provision each user their own EBS / Persistent Disk, so this can get expensive quickly :) Watch out!

You can now customize your hub in two major ways:

1. Customize the hub image. repo2docker is used to build the image, so you can put any of the supported configuration files under deployments/<hub-image>/image. You must make a git commit after modifying this for hubploy build <hub-name> --push --check-registry to work, since it uses the commit hash as the image tag.
2. Customize hub configuration with various YAML files.
   1. hub/values.yaml is common to all hubs that exist in this repo (multiple hubs can live under deployments/).
   2. deployments/<hub-name>/config/common.yaml is where most of the config specific to each hub should go. Examples include memory / cpu limits, home directory definitions, etc
   3. deployments/<hub-name>/config/staging.yaml and deployments/<hub-name>/config/prod.yaml are files specific to the staging & prod versions of the hub. These should be as minimal as possible. Ideally, only DNS entries, IP addresses, should be here.

   4. deployments/<hub-name>/secrets/staging.yaml and deployments/<hub-name>/secrets/prod.yaml

      should contain information that mustn't be public. This would be proxy / hub secret tokens, any authentication tokens you have, etc. These files must be protected by something like git-crypt or `sops <https://github.com/mozilla/sops`_. THIS REPO TEMPLATE DOES NOT HAVE THIS PROTECTION SET UP YET

You can customize the staging hub, deploy it with hubploy deploy <hub-name> hub staging, and iterate until you like how it behaves.

You can then do a production deployment with: hubploy deploy <hub-name> hub prod, and test it out!

1. Secrets & how to keep them
2. Continuous Integration / Deployment
3. What kinda kubernetes setup this needs