WARNING
This solution is no longer supported as 1.15 is out of support in AKS
Seamlessly integrate your key management systems with Kubernetes.
Secrets, keys, and certificates in a key management system become a volume accessible to pods. Once the volume is mounted, its data is available directly in the container filesystem for your application.
- Getting Started
- Detailed Use Cases
- Design
- About Key Vault
- About Certificates
- Contributing
- Code of Conduct
-
Azure Key Vault
💡 NOTE: To enable encryption at rest of Kubernetes data in
etcd, use the Kubernetes KMS plugin for Azure Key Vault.
AKS Engine creates customized Kubernetes clusters on Azure.
Follow the AKS Engine add-on documentation to create a new Kubernetes cluster with Key Vault FlexVolume already deployed.
Azure Kubernetes Service (AKS) creates managed, supported Kubernetes clusters on Azure.
Deploy Key Vault FlexVolume to your AKS cluster with this command:
kubectl create -f https://raw.githubusercontent.com/Azure/kubernetes-keyvault-flexvol/master/deployment/kv-flexvol-installer.yamlTo validate Key Vault FlexVolume is running as expected, run the following command:
kubectl get pods -n kvThe output should show keyvault-flexvolume pods running on each agent node:
NAME READY STATUS RESTARTS AGE
keyvault-flexvolume-f7bx8 1/1 Running 0 3m
keyvault-flexvolume-rcxbl 1/1 Running 0 3m
keyvault-flexvolume-z6jm6 1/1 Running 0 3mIf using keyvault-flexvolume in a cluster with pod security policy enabled, create the following policy that enables the spec required for keyvault-flexvolume to work -
kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-keyvault-flexvol/master/deployment/kv-flexvol-psp.yamlKey Vault FlexVolume offers four modes for accessing a Key Vault instance: Service Principal, Pod Identity, [VMSS User Assigned Managed Identity], [VMSS System Assigned Managed Identity].
Add your service principal credentials as Kubernetes secrets accessible by the Key Vault FlexVolume driver.
kubectl create secret generic kvcreds --from-literal clientid=<CLIENTID> --from-literal clientsecret=<CLIENTSECRET> --type=azure/kvEnsure this service principal has all the required permissions to access content in your Key Vault instance. If not, run the following Azure CLI commands:
# [Required for version < v0.0.13] Assign Reader Role to the service principal for your keyvault
az role assignment create --role Reader --assignee <principalid> --scope /subscriptions/<subscriptionid>/resourcegroups/<resourcegroup>/providers/Microsoft.KeyVault/vaults/<keyvaultname>
# Assign key vault permissions to your service principal
az keyvault set-policy -n $KV_NAME --key-permissions get --spn <YOUR SPN CLIENT ID>
az keyvault set-policy -n $KV_NAME --secret-permissions get --spn <YOUR SPN CLIENT ID>
az keyvault set-policy -n $KV_NAME --certificate-permissions get --spn <YOUR SPN CLIENT ID>Fill in the missing pieces in this deployment for your own deployment. Make sure to:
-
Reference the service principal Kubernetes secret created in the previous step
secretRef: name: kvcreds
-
Pass in properties for the Key Vault instance to the FlexVolume driver.
Name Required Description Default Value usepodidentity no specify access mode: use a service principal or pod identity or vm managed identity "false" usevmmanagedidentity not required, available for version >= v0.0.15 specify access mode: use a service principal or pod identity or vm managed identity "false" vmmanagedidentityclientid not required, available for version >= v0.0.15 If using a user assigned identity as the VM's managed identity, then specify the identity's client id. If empty, then defaults to use the system assigned identity on the VM "" keyvaultname yes name of Key Vault instance "" keyvaultobjectnames yes names of Key Vault objects to access "" keyvaultobjectaliases no filenames to use when writing the objects keyvaultobjectnames keyvaultobjecttypes yes types of Key Vault objects: secret, key or cert "" keyvaultobjectversions no versions of Key Vault objects, if not provided, will use latest "" resourcegroup required for version < v0.0.14 name of resource group containing Key Vault instance "" subscriptionid required for version < v0.0.14 name of subscription containing Key Vault instance "" tenantid yes name of tenant containing Key Vault instance "" cloudname no Name of the cloud environment, e.g. something like AzureChinaCloud, AzureGermanCloud. If not provided, the default public Azure cloud will be used "" nmiport not required, available for version >= v0.0.17 Port number of the NMI daemonset. If not provided, the default NMI port is used "2579" Multiple values in the
keyvaultobjectnames,keyvaultobjecttypesandkeyvaultobjectversionsproperties should be separated with semicolons (;). -
Specify mount path of flexvolume to mount key vault objects
volumeMounts: - name: test mountPath: /kvmnt readOnly: true
Example of an nginx pod accessing a secret from a Key Vault instance:
apiVersion: v1 kind: Pod metadata: name: nginx-flex-kv spec: containers: - name: nginx-flex-kv image: nginx volumeMounts: - name: test mountPath: /kvmnt readOnly: true volumes: - name: test flexVolume: driver: "azure/kv" secretRef: name: kvcreds # [OPTIONAL] not required if using Pod Identity options: usepodidentity: "false" # [OPTIONAL] if not provided, will default to "false" usevmmanagedidentity: "false" # [OPTIONAL new in version >= v0.0.15] if not provided, will default to "false" vmmanagedidentityclientid: "clientid" # [OPTIONAL new in version >= v0.0.15] use the client id to specify which user assigned managed identity to use, leave empty to use system assigned managed identity keyvaultname: "testkeyvault" # [REQUIRED] the name of the KeyVault keyvaultobjectnames: "testsecret" # [REQUIRED] list of KeyVault object names (semi-colon separated) keyvaultobjectaliases: "secret.json" # [OPTIONAL] list of KeyVault object aliases keyvaultobjecttypes: secret # [REQUIRED] list of KeyVault object types: secret, key, cert (semi-colon separated) keyvaultobjectversions: "testversion" # [OPTIONAL] list of KeyVault object versions (semi-colon separated), will get latest if empty resourcegroup: "testresourcegroup" # [REQUIRED for version < v0.0.14] the resource group of the KeyVault subscriptionid: "testsub" # [REQUIRED for version < v0.0.14] the subscription ID of the KeyVault tenantid: "testtenant" # [REQUIRED] the tenant ID of the KeyVault nmiport: "nmiportnumber" # [OPTIONAL new in version >= v0.0.17] port number of the NMI daemonset, will default to "2579"
Deploy your app
kubectl create -f deployment/nginx-flex-kv.yaml
Validate the pod has access to the secret from key vault:
kubectl exec -it nginx-flex-kv cat /kvmnt/testsecret testvalue
💡 The basic steps to configure AAD Pod Identity are reproduced here, but please refer to that project's README for more detail.
-
Install AAD Pod Identity
Run this command to create the
aad-pod-identitydeployment on an RBAC-enabled cluster:kubectl apply -f https://raw.githubusercontent.com/Azure/aad-pod-identity/master/deploy/infra/deployment-rbac.yaml
-
Create an Azure Identity
Run this Azure CLI command, and take note of the
clientIdandidvalues it returns:az identity create -g <resourcegroup> -n <name> -o json
-
Assign Cluster SPN Role
If the Service Principal used for the cluster was created separately (not automatically, as part of an AKS cluster's
MC_resource group), assign it the "Managed Identity Operator" role:az role assignment create --role "Managed Identity Operator" --assignee <sp id> --scope <full id of the managed identity>
-
Assign Azure Identity Roles
Ensure that your Azure Identity has the role assignments required to see your Key Vault instance and to access its content. Run the following Azure CLI commands to assign these roles if needed:
# Assign Reader Role to new Identity for your Key Vault az role assignment create --role Reader --assignee <principalid> --scope /subscriptions/<subscriptionid>/resourcegroups/<resourcegroup>/providers/Microsoft.KeyVault/vaults/<keyvaultname> # set policy to access keys in your Key Vault az keyvault set-policy -n $KV_NAME --key-permissions get --spn <YOUR AZURE USER IDENTITY CLIENT ID> # set policy to access secrets in your Key Vault az keyvault set-policy -n $KV_NAME --secret-permissions get --spn <YOUR AZURE USER IDENTITY CLIENT ID> # set policy to access certs in your Key Vault az keyvault set-policy -n $KV_NAME --certificate-permissions get --spn <YOUR AZURE USER IDENTITY CLIENT ID>
-
Install the Azure Identity
Save this Kubernetes manifest to a file named
aadpodidentity.yaml:apiVersion: "aadpodidentity.k8s.io/v1" kind: AzureIdentity metadata: name: <a-idname> spec: type: 0 resourceID: /subscriptions/<subid>/resourcegroups/<resourcegroup>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<name> clientID: <clientId>
Replace the placeholders with your user identity values. Set
type: 0for user-assigned MSI.Finally, save your changes to the file, then create the
AzureIdentityresource in your cluster:kubectl apply -f aadpodidentity.yaml
-
Install the Azure Identity Binding
Save this Kubernetes manifest to a file named
aadpodidentitybinding.yaml:apiVersion: "aadpodidentity.k8s.io/v1" kind: AzureIdentityBinding metadata: name: demo1-azure-identity-binding spec: azureIdentity: <a-idname> selector: <label value to match>
Replace the placeholders with your values. Ensure that the
AzureIdentityname matches the one inaadpodidentity.yaml.Finally, save your changes to the file, then create the
AzureIdentityBindingresource in your cluster:kubectl apply -f aadpodidentitybinding.yaml
-
Update the Application Deployment
Your application manifest needs a couple of changes. Refer to the nginx-flex-kv-podid deployment as an example.
a. Include the
aadpodidbindinglabel to match theselectorvalue from the previous step:metadata: labels: aadpodidbinding: "NAME OF the AzureIdentityBinding SELECTOR"
b. Set
usepodidentitytotrue:usepodidentity: "true"
-
Deploy your app
kubectl create -f deployment/nginx-flex-kv-podidentity.yaml
-
Validate the pod can access the secret from Key Vault:
kubectl exec -it nginx-flex-kv-podid cat /kvmnt/testsecret testvalue
NOTE: When using the
Pod Identityoption mode, there may be some delay in obtaining the objects from Key Vault. During pod creation time, AAD Pod Identity needs to create theAzureAssignedIdentityfor the pod based on theAzureIdentityandAzureIdentityBindingand retrieve the token for Key Vault. It is possible for the pod volume mount to fail during this time. If it does, the kubelet will keep retrying until after the token retrieval is complete and the mount succeeds.
This option allows flexvol to use the user assigned managed identity on the k8s cluster VMSS directly.
Warning: As of today (2019/09), AKS does not preserve the user assigned identity on VMSS during upgrade. You will need to re-assign the managed identities to VMSS after an upgrade. The improved experience is planned.
- Create Azure Managed Identity
az identity create -g <RESOURCE GROUP> -n <IDENTITY NAME>-
Grant Azure Managed Identity KeyVault permissions
Ensure that your Azure Identity has the role assignments required to see your Key Vault instance and to access its content. Run the following Azure CLI commands to assign these roles if needed:
# set policy to access keys in your Key Vault az keyvault set-policy -n $KV_NAME --key-permissions get --spn <YOUR AZURE MANAGED IDENTITY CLIENT ID> # set policy to access secrets in your Key Vault az keyvault set-policy -n $KV_NAME --secret-permissions get --spn <YOUR AZURE MANAGED IDENTITY CLIENT ID> # set policy to access certs in your Key Vault az keyvault set-policy -n $KV_NAME --certificate-permissions get --spn <YOUR AZURE MANAGED IDENTITY CLIENT ID>
-
Assign Azure Managed Identity to VMSS
az vmss identity assign -g <RESOURCE GROUP> -n <K8S-AGENT-POOL-VMSS> --identities <USER ASSIGNED IDENTITY RESOURCE ID>- Deploy your application. Specify
usevmmanagedidentitytotrueand providevmmanagedidentityclientid.
usevmmanagedidentity: "true" # [OPTIONAL] if not provided, will default to "false"
vmmanagedidentityclientid: "clientid" # [OPTIONAL] use the client id to specify which user assigned managed identity to use, leave empty to use system assigned managed identityThis option allows flexvol to use the system assigned managed identity on the k8s cluster VMSS directly.
- Verify that the nodes have its own system assigned managed identity
az vmss identity show -g <resource group> -n <vmss scalset name> -o yamlThe output should contain type: SystemAssigned.
-
Grant Azure Managed Identity KeyVault permissions
Ensure that the Azure system assigned Identity has the role assignments required to see your Key Vault instance and to access its content. Run the following Azure CLI commands to assign these roles if needed:
# set policy to access keys in your Key Vault az keyvault set-policy -n $KV_NAME --key-permissions get --object-id <YOUR AZURE MANAGED IDENTITY OBJECT ID> # set policy to access secrets in your Key Vault az keyvault set-policy -n $KV_NAME --secret-permissions get --object-id <YOUR AZURE MANAGED IDENTITY OBJECT ID> # set policy to access certs in your Key Vault az keyvault set-policy -n $KV_NAME --certificate-permissions get --object-id <YOUR AZURE MANAGED IDENTITY OBJECT ID>
-
Deploy your application. Specify
usevmmanagedidentitytotrue.
usevmmanagedidentity: "true" # [OPTIONAL] if not provided, will default to "false"- Use Key Vault FlexVol to set up an SSL entrypoint with Istio
- Use Key Vault FlexVol to set up an SSL entrypoint with Traefik
To learn more about the design of Key Vault FlexVolume, see Concept.
Key Vault FlexVolume interacts with Key Vault objects by using the Key Vault API.
Azure Key Vault has thorough documentation available to help clarify the difference between keys, secrets, and certificates.
It is important to understand how a certificate is structured in Key Vault.
As mentioned in Certificates are complex objects and Composition of a Certificate, Azure Key Vault (AKV) represents an X.509 certificate as three related resources:
- an AKV-certificate
- an AKV-key
- an AKV-secret
All three will share the same name and the same version and can be fetched independently.
- The AKV-certificate provides the public key and certificate metadata. Specifying
certinkeyvaultobjecttypeswill fetch the public key and certificate metadata. - The AKV-key provides the private key of the X.509 certificate. It can be useful for performing cryptographic operations such as signing if the corresponding certificate was marked as non-exportable. Specifying
keyinkeyvaultobjecttypeswill fetch the private key of the certificate if its policy allows for private key exporting. - The AKV-secret provides a way to export the full X.509 certificate, including its private key (if its policy allows for private key exporting). Specifying
secretinkeyvaultobjecttypeswill fetch the base64-encoded certificate bundle.
The Key Vault FlexVolume project welcomes contributions and suggestions. Please see CONTRIBUTING for details.
This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent