Giter VIP home page Giter VIP logo

msi-acrpull's Introduction

MSI ACR Pull

MSI ACR Pull enables deployments in a Kubernetes cluster to use any user assigned managed identity to pull images from Azure Container Registry. With this, each application can use its own identity to pull container images.

Install

Run following command to install latest build from main branch. It will install the needed custom resource definition ACRPullBinding and deploy msi-acrpull controllers in msi-acrpull-system namespace.

kubectl apply -f https://raw.githubusercontent.com/Azure/msi-acrpull/main/deploy/latest/crd.yaml -f https://raw.githubusercontent.com/Azure/msi-acrpull/main/deploy/latest/deploy.yaml

How to use

NOTE: following steps assumes you already have:

  1. An Kubernetes cluster, and have user assigned managed identities on node pool VMSS.
  2. An ACR, and the user assigned identity has AcrPull role assigned on ACR.

Once msi-acrpull is installed to your cluster, all you need is to deploy a custom resource AcrPullBinding to the application namesapce to bind an user assigned identity to an ACR. Following sample specifies all pods using default service account in the namespace to use user managed identity my-acr-puller to pull image from veryimportantcr.azurecr.io.

apiVersion: msi-acrpull.microsoft.com/v1beta1
kind: AcrPullBinding
metadata:
  name: acrpulltest
spec:
  acrServer: veryimportantcr.azurecr.io
  managedIdentityResourceID: /subscriptions/712288dc-f816-4242-b73f-a0a87265dcc8/resourceGroups/my-identities/providers/Microsoft.ManagedIdentity/userAssignedIdentities/my-acr-puller

Once the custom resource deployed, you can deploy your application to pull images from the ACR. No changes to the application deployment yaml is needed.

If the application pod uses a custom service account, then specify serviceAccountName property in AcrPullBinding spec.

Default Values

If you use the same MSI and ACR endpoint for all your container, you can provide a default value to the controller. To do so, set the environment variables on the msi-acrpull-controller-manager container :

3 default values can be set :

  • ACR_SERVER
  • MANAGED_IDENTITY_RESOURCE_ID
  • MANAGED_IDENTITY_CLIENT_ID

These environment variables are used if the ACRPullBinding crd does not set them. Deployment spec example:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    control-plane: controller-manager
  name: msi-acrpull-controller-manager
  namespace: msi-acrpull-system
spec:
  replicas: 2
  selector:
    matchLabels:
      control-plane: controller-manager
  template:
    metadata:
      labels:
        control-plane: controller-manager
    spec:
      containers:
      - args:
        - --metrics-addr=127.0.0.1:8080
        - --enable-leader-election
        env:
        - name: "ACR_SERVER"
          value: "myacr.azurecr.io"
        - name: "MANAGED_IDENTITY_RESOURCE_ID"
          value: "<you managed identity resource id>"
        command:
        - /manager
        image: mcr.microsoft.com/aks/msi-acrpull:v0.1.0-alpha
        name: manager
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 20Mi
      - args:
        - --secure-listen-address=0.0.0.0:8443
        - --upstream=http://127.0.0.1:8080/
        - --logtostderr=true
        - --v=10
        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
        name: kube-rbac-proxy
        ports:
        - containerPort: 8443
          name: https
      terminationGracePeriodSeconds: 10

How it works

The architecture looks like below. As an user you will create a custom resource ACRPullBinding, which binds a managed identity (using client ID or resource ID) to an Azure container registry (using its FQDN).

Internally, the ACRPullBindingController watches the ACRPullBinding resource, and for each of them, create a secret in the namespace. The secret content is a Docker image pull config, and the password is the ACR access token that the controller exchanged from ACR using managed identity. The secret will be refreshed 30min before it expire automatically. The controller will also associate the secret to the specified service account in namespace (by default, use the default service account). With this, any pods created in the namespace will automatically pull images from the ACR using the specified managed identity credential.

Diagram

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

msi-acrpull's People

Contributors

aaqib-m avatar imiller31 avatar microsoftopensource avatar samkreter avatar serbrech avatar weng271190436 avatar yangl900 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

msi-acrpull's Issues

Use a CRD instead of a placeholder secret

Using a CRD instead of annotations on a secret gives has these benefits:

Using a CRD instead of annotations on pull secrets is more straight forward for users, more easily extensible and allows us to watch only the resources we care about.

Create an onboarding tutorial

something along the line of:

  • create AKS cluster
  • create MSI
  • assign MSI to the VM/VMSS
  • make all
  • make docker-build
  • make docker-push
  • make deploy
  • kubectl apply -f acrpullbinding_sample.yaml
  • kubectl get secret test-msi-acrpull-secret -oyaml
  • kubectl get serviceaccount default -oyaml

Can support MSIs not on nodepool VMSS?

My understanding is that AAD pod identity can resolve any MSI that is in an RG for which the cluster identity has the identity operator role. See https://azure.github.io/aad-pod-identity/docs/getting-started/role-assignment/

az role assignment create --role "Managed Identity Operator" --assignee <ID> --scope /subscriptions/<SubscriptionID>/resourcegroups/<IdentityResourceGroup>

Does msi-acrpull also work this way? The readme appears to note that the MSI needs to be explicitly added to the nodepool VMSS.

Thinking about this some more, the nodepool approach has the advantage of limiting the access scope of the MSI to the nodepool. Maybe the best of both worlds would be if the nodepool MSI could optionally resolve MSIs that are in RG for which it has Managed Identity Operator role

:-)

helm not able to uninstall AcrPullbinding due to finalizers

We are using msi-acr-pull in our system and We are keeping Acrpullbinding config in the same chart where we are installing the msi-acr-pull deployment.
Everything works fine on helm chart installation -> helm install azure-msi-acr-pull .
All the AcrPullbindings are created and imagePullSecrets are also getting mounted in the Service accounts

The problem comes on uninstalling this helm chart
ACR pull bindings are marked as deleted(checked deletedTimestamp in yaml file ) but are not actually deleted due to finalizers.
My theory is that this is happening because msi-acr-pull deployment is getting deleted before ACRPullbindings.

Overall my chart structure looks something like this -
azure-msi-acrpull-config.yml is the file which is creating AcrPullBinding.
image

helm version which we are using is -
version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}

Steps to replicate ->
cd chart-name
helm install chart-name .
kubectl get acrpullbinding
---Notice here that acrpullbinding is successfully created
helm uninstall chart-name
kubectl get acrpullbinding
---Notice here that acrpullbinding still exist, but rest of the things are deleted.
kubectl get acrpullbinding acr-pull-binding-name -o yaml
---Notice here that acrpullbinding is marked as deleted(check deletedTimestamp)
-- Now let's try to install the same chart again
helm install chart-name .
kubectl get acrpullbinding
---Notice here that acrpullbinding automatically gets DELETED on installing the chart again, but rest of the things are installed.

Implement caching logic in secret controller

The secret controller today keeps reconciling the docker pull secret with new token, this is not needed and adds pressure to ACR.

We should implement some caching logic in the controller, so that we only reconcile it if:

  • ACR token getting close to expire
  • Client ID changed
  • target ACR changed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.