Automatically sync secrets from Doppler to Kubernetes and auto-reload deployments when secrets change.
- The Doppler Kubernetes Operator is a controller which runs inside a deployment on your Kubernetes cluster
- It manages custom resources called
DopplerSecrets, each of which contains a reference to a Kubernetes secret containing your Doppler Service Token and a reference to the Kubernetes secret where Doppler secrets should be synced - The operator continuously monitors the Doppler API for changes to your Doppler config and updates the managed Kubernetes secret automatically
- If the secrets have changed, the operator can also reload deployments using the Kubernetes secret. See below for details on configuring auto-reload.
The Doppler Kubernetes Operator uses Kubernetes Secrets to store sensitive data.
Kubernetes Secrets are, by default, stored as unencrypted base64-encoded strings. By default they can be retrieved - as plain text - by anyone with API access, or anyone with access to Kubernetes' underlying data store, etcd. Therefore, Kubernetes recommends enabling encryption at rest to secure this data.
You can install the latest Helm chart with:
helm repo add doppler https://helm.doppler.com
helm install --generate-name doppler/doppler-kubernetes-operatorUpdates can be performed with helm upgrade.
One caveat is that Helm cannot update custom resource definitions (CRDs). To simplify this, Doppler guarantees that CRDs will remain backwards compatible. CRDs can be updated directly from the Helm chart manifest with:
helm repo update
helm pull doppler/doppler-kubernetes-operator --untar
kubectl apply -f doppler-kubernetes-operator/crds/all.yamlYou can also deploy the operator by applying the latest installation YAML directly:
kubectl apply -f https://github.com/DopplerHQ/kubernetes-operator/releases/latest/download/recommended.yamlRegardless of the installation method, this will use your locally-configured kubectl to:
- Create a
doppler-operator-systemnamespace - Create the resource definition for a
DopplerSecret - Setup a service account and RBAC role for the operator
- Create a deployment for the operator inside of the cluster
You can verify that the operator is running successfully in your cluster with ./tools/operator-logs.sh. This waits for the deployment to roll out and then tails the log. You can leave this command running to keep monitoring the logs or quit safely with Ctrl-C.
A DopplerSecret is a custom Kubernetes resource with references to two secrets:
- A Kubernetes secret where your Doppler Service Token is stored (AKA "Doppler Token Secret"). This token will be used to fetch secrets from your Doppler config. The operator will be looking for the token in the
serviceTokenfield of this secret. - A Kubernetes secret where your synced Doppler secrets will be stored (AKA "Managed Secret"). This secret will be created by the operator if it does not already exist.
Note: While these resources can be created in any namespace, it is recommended that you create your Doppler Token Secret and DopplerSecret inside the
doppler-operator-systemnamespace to prevent unauthorized access. The managed secret should be namespaced with the deployments which will use the secret.
Generate a Doppler Service Token and use it in this command to create your Doppler token secret:
kubectl create secret generic doppler-token-secret -n doppler-operator-system --from-literal=serviceToken=dp.st.dev.XXXXIf you have the Doppler CLI installed, you can generate a Doppler Service Token from the CLI and create the Doppler token secret in one step:
kubectl create secret generic doppler-token-secret -n doppler-operator-system --from-literal=serviceToken=$(doppler configs tokens create doppler-kubernetes-operator --plain)Next, we'll create a DopplerSecret that references your Doppler token secret and defines the location of the managed secret.
apiVersion: secrets.doppler.com/v1alpha1
kind: DopplerSecret
metadata:
name: dopplersecret-test # DopplerSecret Name
namespace: doppler-operator-system
spec:
tokenSecret: # Kubernetes service token secret (namespace defaults to doppler-operator-system)
name: doppler-token-secret
managedSecret: # Kubernetes managed secret (will be created if does not exist)
name: doppler-test-secret
namespace: default # Should match the namespace of deployments that will use the secretIf you're following along with these example names, you can apply this sample directly:
kubectl apply -f config/samples/secrets_v1alpha1_dopplersecret.yamlCheck that the associated Kubernetes secret has been created:
# List all Kubernetes secrets created by the Doppler operator
kubectl describe secrets --selector=secrets.doppler.com/subtype=dopplerSecretThe operator continuously watches for secret updates from Doppler and when detected, automatically and instantly updates the associated secret.
Next, we'll cover how to configure a deployment to use the Kubernetes secret and enable auto-reloading for Deployments.
To use the secret created by the operator, we can use the managed secret in one of three ways. These methods are also covered in greater detail in the Kubernetes Secrets documentation.
The envFrom field will populate a container's environment variables using the secret's Key-Value pairs:
envFrom:
- secretRef:
name: doppler-test-secret # Kubernetes secret nameThe valueFrom field will inject a specific environment variable from the Kubernetes secret:
env:
- name: MY_APP_SECRET # The name of the environment variable exposed in the container
valueFrom:
secretKeyRef:
name: doppler-test-secret # Kubernetes secret name
key: MY_APP_SECRET # The name of the key in the Kubernetes secretThe volume field will create a volume that is populated with files containing the Kubernetes secret:
volumes:
- name: secret-volume
secret:
secretName: doppler-test-secret # Kubernetes secret nameYour deployment can use this volume by mounting it to the container's filesystem:
volumeMounts:
- name: secret-volume
mountPath: /etc/secrets
readOnly: trueIn order for the operator to reload a deployment, three things must be true:
- The deployment is in the same namespace as the managed secret
- The deployment has the
secrets.doppler.com/reloadannotation set to'true'(string) - The deployment uses the managed secret
Here's an example of the reload annotation:
annotations:
secrets.doppler.com/reload: 'true'The Doppler Kubernetes operator reloads deployments by updating an annotation with the name secrets.doppler.com/secretsupdate.<KUBERNETES_SECRET_NAME>. When this update is made, Kubernetes will automatically redeploy your pods according to the deployment's configured strategy.
Complete examples of these different deployment configurations can be found below:
If you've named your managed Kubernetes secret doppler-test-secret in the previous step, you can apply any of these examples directly:
kubectl apply -f config/samples/deployment-envfrom.yaml
kubectl rollout status -w deployment/doppler-test-deployment-envfromOnce the Deployment has completed, you can view the logs of the test container:
kubectl logs -lapp=doppler-test --tail=-1Setup is complete! To test the sync behavior, modify a secret in the Doppler dashboard and wait 60 seconds. Run the logs command again (or use the watch command) to see the pods automatically restart with the new secret data.
Name Transformers enable secret names to transformed from Doppler's UPPER_SNAKE_CASE format into any of the following environment variable compatible formats:
| Type | Default | Transform |
|---|---|---|
| camel | API_KEY | apiKey |
| upper-camel | API_KEY | ApiKey |
| lower-snake | API_KEY | api_key |
| tf-var | API_KEY | TF_VAR_api_key |
| dotnet-env | SMTP__USER_NAME | Smtp__UserName |
| lower-kebab | API_KEY | api-key |
Simply add the nameTransformer field with any of the above types:
apiVersion: secrets.doppler.com/v1alpha1
kind: DopplerSecret
metadata:
name: dopplersecret-test
namespace: doppler-operator-system
spec:
tokenSecret:
name: doppler-token-secret
managedSecret:
name: doppler-test-secret
namespace: default
nameTransformer: dotnet-envThe nameTransformer values are also validated prior to admission to prevent transformation failures.
Instead of the standard Key / Value pairs, you can download secrets as a single file in the following formats:
- json
- dotnet-json
- env
- env-no-quotes
- yaml
When format is specified, a single DOPPLER_SECRETS_FILE key is set in the created secret with the string contents of the downloaded file.
Simply add the format field:
apiVersion: secrets.doppler.com/v1alpha1
kind: DopplerSecret
metadata:
name: dotnet-webapp-appsettings
namespace: doppler-operator-system
spec:
tokenSecret:
name: doppler-token-dotnet-webapp
namespace: doppler-operator-system
managedSecret:
name: dotnet-webapp-appsettings
namespace: default
format: dotnet-jsonYou can then configure your deployment spec to mount the file at the desired path:
...
spec:
containers:
- name: dotnet-webapp
volumeMounts:
- name: doppler
mountPath: /usr/src/app/secrets
readOnly: true
volumes:
- name: doppler
secret:
secretName: dotnet-webapp-appsettings # Managed secret name
optional: false
items:
- key: DOPPLER_SECRETS_FILE # Hard-coded by Operator when format specified
path: appsettings.json # Name or path to file name appended to container mountPathYou can have the operator only sync a subset of secrets in a Doppler config. To do this, specify them in the secrets spec property:
apiVersion: secrets.doppler.com/v1alpha1
kind: DopplerSecret
metadata:
name: dopplersecret-test
namespace: doppler-operator-system
spec:
tokenSecret:
name: doppler-token-secret
secrets:
- HOSTNAME
- PORT
managedSecret:
name: doppler-test-secret
namespace: defaultIf this property is omitted all secrets are synced.
By default, the operator syncs secret values as they are in Doppler to an Opaque Kubernetes secret as Key / Value pairs.
In some cases, the secret name or value stored in Doppler is not the format required for your Kubernetes deployment.
For example, you might have Base64-encoded TLS data that you want to copy to a native Kubernetes TLS secret (kubernetes.io/tls).
You can use custom types and processors to achieve this.
If the operator fails to fetch secrets from the Doppler API (e.g. a connection problem or invalid service token), no changes are made to the managed Kubernetes secret or your deployments. The operator will continue to attempt to reconnect to the Doppler API indefinitely.
The DopplerSecret uses status.conditions to report its current state and any errors that may have occurred.
In this example, our Doppler service token has been revoked and the operator is reporting an error condition:
$ kubectl describe dopplersecrets -n doppler-operator-system
Name: dopplersecret-test
Namespace: doppler-operator-system
Labels: <none>
Annotations: <none>
API Version: secrets.doppler.com/v1alpha1
Kind: DopplerSecret
Metadata:
...
Spec:
...
Status:
Conditions:
Last Transition Time: 2021-06-02T15:46:57Z
Message: Secret update failed: Doppler Error: Invalid Service token
Reason: Error
Status: False
Type: secrets.doppler.com/SecretSyncReady
Last Transition Time: 2021-06-02T15:46:57Z
Message: Deployment reload has been stopped due to secrets sync failure
Reason: Stopped
Status: False
Type: secrets.doppler.com/DeploymentReloadReady
Events: <none>
You can safely modify your token Kubernetes secret or DopplerSecret at any time. To update our Doppler service token, we can modify our token Kubernetes secret directly and the changes will take effect immediately.
The DopplerSecret resource manages the managed Kubernetes secret but does not officially own it. Therefore, deleting a DopplerSecret will not automatically delete the managed secret.
tools/get-secret.sh- fetch and decode a Kubernetes secrettools/operator-logs.sh- wait for the operator deployment to roll out, then tail the logs
To uninstall the operator, first delete any DopplerSecret resources and any referenced Kubernetes secrets that are no longer needed.
kubectl delete dopplersecrets --all --all-namespaces
kubectl delete secret doppler-token-secret -n doppler-operator-systemIf you installed the operator with Helm, you can use helm uninstall to remove the installation resources. Otherwise, run the following command:
kubectl delete -f https://github.com/DopplerHQ/kubernetes-operator/releases/latest/download/recommended.yamlThis project uses the Operator SDK.
When developing locally, you can run the operator using:
make install runSee the Operator SDK Go Tutorial for more information.
This project is released with Github Actions. Adding a Github Release will start an action which builds the operator image and publishes it to DockerHub. Tag names should match the pattern vX.X.X.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent