Giter VIP home page Giter VIP logo

aws-secret-sidecar-injector's Introduction

This project is now archived. Please use the CSI Secret Store driver for AWS Secrets Manager instead.

AWS Secret Sidecar Injector

The aws-secret-sidecar-injector is a proof-of-concept(PoC) that allows your containerized applications to consume secrets from AWS Secrets Manager. The solution makes use of a Kubernetes dynamic admission controller that injects an init container, aws-secrets-manager-secret-sidecar, upon creation/update of your pod. The init container relies on IRSA to retrieve the secret from AWS Secrets Manager. The Kubernetes dynamic admission controller also creates an in-memory Kubernetes volume (with name secret-vol and emptyDirectory.medium as Memory) associated with the pod to access the secret.

Announcing the AWS Secrets and Config Provider (ASCP)

As of 4/22/21, you use the CSI Secret Store driver with AWS Secrets Manager and Parameter Store. ASCP is similar to this project in that it mounts secrets as volumes, however there are several key differences that are worth highlighting. First, it works with both Secrets Manager and Parameter store. Second, ASCP can mount multiple secrets whereas the sidecar injector only supports 1. Third, ASCP can synchronize secrets from Secrets Manager to Kubernetes Secrets which is similar to GoDaddy's ExternalSecrets project. Copying secrets from Secrets Manager to Kubernetes Secrets allows you to map secrets to environment variables instead of mounting them as volumes. Fourth, ASCP can rotate secrets, however, unlike the sidecar injector, ASCP uses a polling mechanism rather than an event to trigger the rotation. When we were thinking of how to handle the rotation of secrets we decided to use an event rather than polling to a) limit the resources required to continuously run the sidecar b) avoid getting throttled; currently 5,000 OPS c) keep costs low or as close to zero as possible; Secrets Manager charges $0.05 per 10,000 API calls. Fifth, with ASCP you have to create a secret provider class for each secret you want to reference in your pod. Finally, the CSI Secret Store driver gives you a vendor agnostic and standard way to fetch secrets from an external secret store.

You can continue using Michael Hausenblas's NASE project to create secrets in Secrets Manager.

We will continue supporting this project, but we also encourage you to give ASCP a try. Thank you to all of those who provided feedback and helped make this project what it is today. For additional information about ASCP see:

Prerequsites

  • An IRSA ServiceAccount that has permission to access and retrive the secret from AWS Secrets Manager
  • Helm to install the mutating admission webhook

Installation

Deploying mutating webhook to inject the init container

  • Add the Helm repository which contains the Helm chart for the mutating admission webhook

    helm repo add secret-inject https://aws-samples.github.io/aws-secret-sidecar-injector/

  • Update the Helm repository

    helm repo update

  • Deploy the mutating webhook admission controller

    helm install secret-inject secret-inject/secret-inject

Accessing the secret

Add the following annotations to your podSpec to mount the secret in your pod

secrets.k8s.aws/secret-arn: <SECRET-ARN>

By default, the decrypted secret is written to a volume named secret-vol and the filename of the secret is secret. The Kubernetes dynamic admission controller also creates corresponding mountPath /tmp/secret for containers within the pod to access the secret.

You can optionally mount the secret-vol volume for containers within the pod at a specific path using the following optional annotation

secrets.k8s.aws/mount-path: <ABSOULTE-MOUNT-PATH>

Note that,the path should be an absolute path such as "/my-path"

You can optionally customize the filename / subfolders within the mounted path where the secret is written by using hte following optional annotation

secrets.k8s.aws/secret-filename: <SECRET-FILENAME>

This repository contains a sample Kubernetes deployment manifest which uses this project to access AWS Secrets Manager secret.

Creating Secrets

AWS Secrets Manager secrets can be created and managed natively in Kubernetes using Native Secrets(NASE). The NASE project is a serverless mutating webhook, which "intercepts" the calls to create and update native Kubernetes Secrets and writes the secret in the secret manifest to AWS Secrets Manager and returns the ARN of the secret to Kubernetes which stores it as a secret.

Rotating Secrets

Support for restarting pods when the secret they reference is rotated, is now available. For additional information, see the README in the secret-operator folder.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

aws-secret-sidecar-injector's People

Contributors

amazon-auto avatar amit0701 avatar itmustbejj avatar jicowan avatar joebowbeer avatar mahendrasiddappa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-secret-sidecar-injector's Issues

Explore using the AWS Secrets Manager caching feature

AWS Secrets Manager has a caching library which updates the cache periodically, ensuring your applications use the most up to date secret value without polling. If you run the init container as a sidecar and make use of this library it may eliminate the need for an operator that recycles the pod when a secret is rotated. The downside is that it will require you to run a sidecar (increases the resource requirements) and your application will have to be written to re-read the password from disk when the secret is rotated.

[Question] Deploying secret-inject Controller in another namespace

I am able to get all the workflow for secrets working within default namespace.

Keeping the secret-inject controller in defailt namespace , when I try to deploy the controller into another namespace , i get errors.

Error: rendered manifests contain a resource that already exists. Unable to continue with install: MutatingWebhookConfiguration "aws-secret-inject" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-namespace" must equal "chef-ns": current value is "default"

Can I deploy the controllers in multiple namespace within same eks cluster ?
How can i access secrets within multiple namespaces ?

Not able to have secrets on /tmp/secret

Hello.

We deployed the secret-inject into one of our EKS clusters and, even though on the secret inject logs we seem to be pulling secrets correctly, there's no secret saved into /tmp/secret at all. Both the secret exists and the service account is using a valid role (as you can see on the logs, we are getting a valid but empty response on pods.go:157).

I0717 09:53:38.080245       1 main.go:81] handling request: {"kind":"AdmissionReview","apiVersion":"admission.k8s.io/v1beta1","request":{"uid":"5f51ec5e-758a-47d0-9dd4-17156b0b54bb","kind":{"group":"","version":"v1","kind":"Pod"},"resource":{"group":"","version":"v1","resource":"pods"},"requestKind":{"group":"","version":"v1","kind":"Pod"},"requestResource":{"group":"","version":"v1","resource":"pods"},"namespace":"default","operation":"CREATE","userInfo":{"username":"system:serviceaccount:kube-system:replicaset-controller","uid":"b5552644-6aa3-11ea-b53c-0ab35cafaebc","groups":["system:serviceaccounts","system:serviceaccounts:kube-system","system:authenticated"]},"object":{"kind":"Pod","apiVersion":"v1","metadata":{"generateName":"webserver-7b56c5866-","creationTimestamp":null,"labels":{"pod-template-hash":"7b56c5866","run":"webserver"},"annotations":{"kubernetes.io/psp":"eks.privileged","secrets.k8s.aws/secret-arn":"arn:aws:secretsmanager:eu-west-2:1234567:secret:foo-EYL7CW","secrets.k8s.aws/sidecarInjectorWebhook":"enabled","sidecar.istio.io/inject":"false"},"ownerReferences":[{"apiVersion":"apps/v1","kind":"ReplicaSet","name":"webserver-7b56c5866","uid":"ae04dfa2-a7f9-401a-bfb6-50b328229506","controller":true,"blockOwnerDeletion":true}]},"spec":{"volumes":[{"name":"service-account-token-6ntkj","secret":{"secretName":"service-account-token-6ntkj"}}],"containers":[{"name":"webserver","image":"busybox:1.28","command":["sh","-c","echo $(cat /tmp/secret) \u0026\u0026 sleep 3600"],"resources":{},"volumeMounts":[{"name":"service-account-token-6ntkj","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"service-account","serviceAccount":"service-account","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{}},"oldObject":null,"dryRun":false,"options":{"kind":"CreateOptions","apiVersion":"meta.k8s.io/v1"}}}
I0717 09:53:38.080460       1 pods.go:157] &AdmissionResponse{UID:,Allowed:true,Result:nil,Patch:*[91 10 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32 123 34 111 112 34 58 34 97 100 100 34 44 34 112 97 116 104 34 58 34 47 115 112 101 99 47 105 110 105 116 67 111 110 116 97 105 110 101 114 115 34 44 34 118 97 108 117 101 34 58 91 123 34 105 109 97 103 101 34 58 34 100 111 99 107 101 114 46 105 111 47 97 109 97 122 111 110 47 97 119 115 45 115 101 99 114 101 116 115 45 109 97 110 97 103 101 114 45 115 101 99 114 101 116 45 115 105 100 101 99 97 114 58 118 48 46 49 46 49 34 44 34 110 97 109 101 34 58 34 115 101 99 114 101 116 115 45 105 110 105 116 45 99 111 110 116 97 105 110 101 114 34 44 34 118 111 108 117 109 101 77 111 117 110 116 115 34 58 91 123 34 110 97 109 101 34 58 34 115 101 99 114 101 116 45 118 111 108 34 44 34 109 111 117 110 116 80 97 116 104 34 58 34 47 116 109 112 34 125 93 44 34 101 110 118 34 58 91 123 34 110 97 109 101 34 58 32 34 83 69 67 82 69 84 95 65 82 78 34 44 34 118 97 108 117 101 70 114 111 109 34 58 32 123 34 102 105 101 108 100 82 101 102 34 58 32 123 34 102 105 101 108 100 80 97 116 104 34 58 32 34 109 101 116 97 100 97 116 97 46 97 110 110 111 116 97 116 105 111 110 115 91 39 115 101 99 114 101 116 115 46 107 56 115 46 97 119 115 47 115 101 99 114 101 116 45 97 114 110 39 93 34 125 125 125 93 44 34 114 101 115 111 117 114 99 101 115 34 58 123 125 125 93 125 44 123 34 111 112 34 58 34 97 100 100 34 44 34 112 97 116 104 34 58 34 47 115 112 101 99 47 118 111 108 117 109 101 115 47 45 34 44 34 118 97 108 117 101 34 58 123 34 101 109 112 116 121 68 105 114 34 58 32 123 34 109 101 100 105 117 109 34 58 32 34 77 101 109 111 114 121 34 125 44 34 110 97 109 101 34 58 32 34 115 101 99 114 101 116 45 118 111 108 34 125 125 44 123 34 111 112 34 58 32 34 97 100 100 34 44 34 112 97 116 104 34 58 32 34 47 115 112 101 99 47 99 111 110 116 97 105 110 101 114 115 47 48 47 118 111 108 117 109 101 77 111 117 110 116 115 47 45 34 44 34 118 97 108 117 101 34 58 32 123 34 109 111 117 110 116 80 97 116 104 34 58 32 34 47 116 109 112 47 34 44 34 110 97 109 101 34 58 32 34 115 101 99 114 101 116 45 118 111 108 34 125 125 93],PatchType:*JSONPatch,AuditAnnotations:map[string]string{},}

The helm chart version that we are using is 0.1.2.

Hope you guys can lend us a hand.

Changing mount point and ecrousseau/aws-secret-injector fork.

Hello,

I am trying to make use of this. I am able to get the implementation from master to stand up. But I have a couple of questions. Is it possible to the mount point point to something other then /tmp/secret. Is it possible to mount secret-vol directly in a pod? Also, in a regular kubernetes secret, each secret value is placed in a file with the key as the name of the file. /tmp/secret is not formated. I have a container that is looking for secrets ala regular kubernetes secret format.

The fork at ecrousseau/aws-secret-injector does is closer to what we are looking for. But I cannot get it to stand up. First the webhook name in the chart template webhook.yaml is secret-inject. Deploying that helm chart results in error: Error: MutatingWebhookConfiguration.admissionregistration.k8s.io "secret-inject" is invalid: webhooks[0].name: Invalid value: "secret-inject": should be a domain with at least three segments separated by dots

so I changed wehhook name from secret-inject to secret-inject.aws.amazon.com. The webhook deploys but the deployment fails to create the test pod.k get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
secret-inject 1/1 1 1 117s
secrets-testing 0/1 0 0 70s

Update init container to support multiple secrets

Presently, the init container is designed to fetch 1 secret from AWS Secrets Manager, yet some pods may require multiple secrets. The annotation for secretArn should accept an array that the init container can loop through to get multiple secrets. Need to decide whether to mount each secret as a separate volume, write to separate files on the same volume, or create a single file with K/V pairs for each secret.

initContainers are overwritten in deployment

A deployment manifest looks like this

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      annotations:
        secrets.k8s.aws/sidecarInjectorWebhook: enabled
        secrets.k8s.aws/secret-arn: arn:aws:secretsmanager:xxx:000000:secret:sxjxkx-UOBckr
      labels:
        app: nginx
    spec:
      serviceAccountName: aws-eks-secrets-sa
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
        - containerPort: 443
        volumeMounts:
        - name: tmpconfig
          mountPath: /tmp/config
      initContainers:
      - name: db-init
        image: busybox
        command: ['/bin/sh', '-c']
        args:
         - echo "HELLO WORLD";

But when this manifest is applied and pod is running,
doing Kubectl describe pod nginx-deployment-xxxx has no db-init in initContainers list.
There's only secrets-init-container in initContainers list!

Does the admission-controller webhook overwrites all other initContainers !!!

Feature request - inject secrets as env vars

It'd be really useful to have the ability to inject secrets as env vars as an alternative to mounting them in a file.
I'm fully aware that using env vars for secrets is a terrible security practice and leaves secrets exposed and people should NOT do this. However the reality is that sometimes we have to deal with things written by other people - helm charts, legacy applications etc and in some cases these expect the secrets to be set as env vars and provide no alternative way to configure them.

I'm currently dealing with a bunch of applications written in a variety of different languages that all expect their secrets in environment variables. This means my options are:

  1. modify the code for all the applications to read each value from the file when needed
  2. modify the code for all the applications to read the file at application startup and set the values as env vars
  3. modify the Dockerfile (and/or entrypoint script) for all the applications to read the file and set the env vars before starting the application

Options 1 & 2 won't be possible if I don't own or have access to the code and if can change the code, the solution would need to be different depending on the language of the app.
Option 3 is a little better as could be the same solution in all apps regardless of the app language however still involves updating the Dockerfiles etc for every app.

If secret-inject could set secrets as env vars it could work in all cases even when using other people stuff e.g. public helm charts.

If this is something you'd consider as a feature I'd be happy to attempt a PR to implement this functionality.

set_aws_parameters does not set OPERATOR_REGION, SQS_URL, IAM_ARN

Hello,
I ran into an issue when running make install and it goes to set_aws_parameters to set OPERATOR_REGION, SQS_URL, IAM_ARN for sed. Instead of values, they are blank. Is this only on MacOS using default make binary? I have fixed it by avoiding variable definitions in particular action by directly setting:
sed -i .bak "s,OPERATOR_REGION,$(shell aws cloudformation describe-stacks --stack-name EKS-Secrets-Operator-Stack --query "Stacks[0].Outputs[?OutputKey=='Region'].OutputValue" --output text),g" config/manager/manager.yaml
and so on.

change secret mount point inside the container

as mentioned in the README.md

The Kubernetes dynamic admission controller also creates corresponding mountPath /tmp/secret for containers within the pod to access the secret

in my case, my app needs to use a secret file in a specific path in the file system which is not configurable.
There is any way to change the mount path from /tmp/secret to a configurable path?

currently, it seems that it's hard coded

f, err := os.Create("/tmp/secret")

add operation does not apply: doc is missing path: "/spec/initContainers/0

Hi , i did a fresh installation today morning.

Getting this error "Error creating: Internal error occurred: add operation does not apply: doc is missing path: "/spec/initContainers/0": missing value"

2021-03-23T06:43:46.195271227Z I0323 06:43:46.195153 1 pods.go:159] [
2021-03-23T06:43:46.195285338Z {"op":"add","path":"/spec/initContainers/0","value":{"image":"docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4","name":"secrets-init-container","imagePullPolicy": "Always","volumeMounts":[{"name":"secret-vol","mountPath":"/tmp"}],"env":[{"name": "SECRET_ARN","valueFrom": {"fieldRef": {"fieldPath": "metadata.annotations['secrets.k8s.aws/secret-arn']"}}}],"resources":{}}},{"op":"add","path":"/spec/volumes/-","value":{"emptyDir": {"medium": "Memory"},"name": "secret-vol"}},{"op": "add","path": "/spec/containers/0/volumeMounts/-","value": {"mountPath": "/tmp/","name": "secret-vol"}}]

proxy setup

Hello, I would like to use this utilitiy in a proxy environment and would like to set a proxy in the sidecar container, can i please get access to the docker file for "aws-secrets-manager-secret-sidecar" or if there is any alternative way to inject proxy as an environment variable that will do as well. I am using zscaler so may need to bake my CA so having access to the docker file will be useful.

How to deserialize the json data that comes from SecretsManager and got loaded in the volumeMount

The AWS SecretsManager POC controller works fine. However we ended up getting the secret in the volume mounted at /tmp/secret . Can you please help in deserializing this data from the volume mount so we can read the key value pair?

Eg: Following secret from SecretsManager got loaded into the volume /tmp/secret through the InitContainer and mutating web hook. Our main container can read this data from /tmp/secret with no issues. However how can the main container now read this as individual key value pairs and lod into Environment variables in the Pod?

{
"database-password": "planet",
"anotherpwd": "anothervalue"
}

Feature Request: inject secrets into init containers

In case a pod definition contains init containers, e.g. for bootstrapping a database prior to the launch, they do not inherit the same injected secrets as the main container does.

It would be nice if aws-secret-sidecar-injector could add the default set of volume mounts to all containers that take part in the pod lifecycle.

[Question]How do we access the secrets mounted in the /tmp/secret to k8s-env ?

Hi ,

Am not able to locate to the fetch the environment variable which is mounted in the /tmp/secret path .

/tmp/secret wil contain the key/value pair .
eg :
{"username":"admin","password":"P@$$word1024","engine":"mysql","host":"database-1.cluster.us-east-1.rds.amazonaws.com","port":3306,"dbClusterIdentifier":"database-1"}

But I want to pass the these value to the kubernets environment

something like this :
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: database-credentials
key: DATABASE_URL
or env:
- name: DATABASE_URL
value: /tmp/secret
..
As /tmp/secret file contains the key value pair in it . Am unable to fetch to the k8s-envi file.

If you know any possible cases please do let me know...

http: TLS handshake error from <IP>:<PORT>: remote error: tls: bad certificate

I installed helm chart for aws-secret-sidecar-injector following the guide at https://aws.amazon.com/blogs/containers/aws-secrets-controller-poc/

A new deployment with proper annotations is created.
AWS iam_role and policies are properly created.
Kubernetes serviceaccount is created in line with the guide and specified in the deployment spec
But the deployment pod doesn't have secret mounted at /tmp directory
When checking logs for pod secret-inject-xxxx-xxx, it has the error as described in title

2021/03/18 10:03:05 http: TLS handshake error from 10.0.20.176:53728: remote error: tls: bad certificate
2021/03/18 10:03:05 http: TLS handshake error from 10.0.20.176:53734: remote error: tls: bad certificate
2021/03/18 10:44:40 http: TLS handshake error from 10.0.20.176:38152: remote error: tls: bad certificate
2021/03/18 10:46:43 http: TLS handshake error from 10.0.10.126:59140: remote error: tls: bad certificate
2021/03/18 10:49:39 http: TLS handshake error from 10.0.10.126:60042: remote error: tls: bad certificate

Trust.json

I am doing something wrong for creating trust.json .

Could you please give some guidance. I am using default namespace.

$ read -r -d 'default' TRUST_RELATIONSHIP < trust.json
bash: trust.json: No such file or directory

Feature Request - AWS Parameter Store Integration

We are looking for a solution that can also work with AWS Parameter Store. Is this a feature you guys can add, and if its not something you're planning on looking into in the short-term, I'd be happy with some direction as to what can be modified so we can use this with AWS Param Store

Thanks

Init container isn't started for webserver + TLS handshake error in secret-inject

Hi there!
I'm following instructions but getting only webserver container in my webserver-78578795c6-6l2mx pod
No init container is present there

In my secret-inject-87fd4b8bb-v6jvp pod logs I see http: TLS handshake error from 192.168.183.197:43904: remote error: tls: bad certificate exception on each webserver deploy attempt

Any ideas how to debug / fix it?

cat: can't open '/tmp/secret': No such file or directory

Hi Team,

Current code for webserver.yaml is not working.

Confirmed that created secret is accessible using aws command.

 ➜ aws secretsmanager get-secret-value --secret-id test_secretB --query SecretString --output text
{"username":"user3", "password": "pass3"}

Following is the webserver.yaml code where serviceAccountName set to use default sa.

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    run: webserver
  name: webserver
spec:
  replicas: 1
  selector:
    matchLabels:
      run: webserver
  template:
    metadata:
      annotations:
        secrets.k8s.aws/sidecarInjectorWebhook: enabled
        secrets.k8s.aws/secret-arn: arn:aws:secretsmanager:ap-southeast-1:123456789012:secret:test_secretB-wFblqy
      labels:
        run: webserver
    spec:
      serviceAccountName:  default
      containers:
      - image: busybox:1.28
        name: webserver
        command: ['sh', '-c', 'echo $(cat /tmp/secret) && sleep 3600']
 ➜ kubectl version --short
Client Version: v1.18.4
Server Version: v1.15.11-eks-af3caf

 ➜ helm ls
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
secret-inject   secret          1               2020-08-12 11:42:00.697245 +0800 +08    deployed        secret-inject-0.1.2     1

 ➜ kubectl get mutatingwebhookconfiguration
NAME                CREATED AT
aws-secret-inject   2020-08-12T03:42:01Z

 ➜ k get sa default -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/test_secret
  creationTimestamp: "2020-08-11T08:59:55Z"
  name: default
  namespace: secret
  resourceVersion: "22054368"
  selfLink: /api/v1/namespaces/secret/serviceaccounts/default
  uid: 7e10b31f-47a7-4f0c-8bf1-1c3f5afc79de
secrets:
- name: default-token-d5cwh
 ➜ kl secret-inject-7b8b67fc48-hk87h -f
2020/08/12 03:43:35 http: TLS handshake error from 10.23.0.60:59350: remote error: tls: bad certificate
2020/08/12 03:44:46 http: TLS handshake error from 10.23.1.233:49986: remote error: tls: bad certificate
2020/08/12 06:14:50 http: TLS handshake error from 10.23.0.60:49230: remote error: tls: bad certificate
^C

 ➜ kgp webserver-888fc6786-4z7hp -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubernetes.io/psp: eks.privileged
    secrets.k8s.aws/secret-arn: arn:aws:secretsmanager:ap-southeast-1:123456789012:secret:test_secretB-wFblqy
    secrets.k8s.aws/sidecarInjectorWebhook: enabled
  creationTimestamp: "2020-08-12T06:14:50Z"
  generateName: webserver-888fc6786-
  labels:
    pod-template-hash: 888fc6786
    run: webserver
  name: webserver-888fc6786-4z7hp
  namespace: secret
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: webserver-888fc6786
    uid: 81fcc122-e428-49d0-a0ba-71889a875b45
  resourceVersion: "22205783"
  selfLink: /api/v1/namespaces/secret/pods/webserver-888fc6786-4z7hp
  uid: 6087099f-7f9b-4908-a56d-ee3398e657a9
spec:
  containers:
  - command:
    - sh
    - -c
    - echo $(cat /tmp/secret) && sleep 3600
    image: busybox:1.28
    imagePullPolicy: IfNotPresent
    name: webserver
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-d5cwh
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: ip-10-23-7-28.ap-southeast-1.compute.internal
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: default-token-d5cwh
    secret:
      defaultMode: 420
      secretName: default-token-d5cwh
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2020-08-12T06:14:50Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2020-08-12T06:14:52Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2020-08-12T06:14:52Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2020-08-12T06:14:50Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://b47228f01099a63006aba623a2c99966432baca592b51c73f5979124337117b5
    image: busybox:1.28
    imageID: docker-pullable://busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
    lastState: {}
    name: webserver
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2020-08-12T06:14:51Z"
  hostIP: 10.23.7.28
  phase: Running
  podIP: 10.23.6.103
  qosClass: BestEffort
  startTime: "2020-08-12T06:14:50Z"

 ➜ kl -l run=webserver -f
cat: can't open '/tmp/secret': No such file or directory

^C

WebIdentity Error

Pods are coming up but this is creating init container and in the logs of init container we get below error.

WebIdentityError: Failed to retrieve credentials.
caused by Invalid Identity token: No OpenIDConnect provider found in account

Is this something related to misconfiguration or we need to set up OIDC in our account.

Thanks
Murali

Inject secret as environment variable

Hi,

I was just wondering, is it currently possible to use secrets as environment variables instead of writing them to /tmp/secret ?

Thanks a lot.

ability to create serviceAccount by using helm

In order to get an easier deployment, it will be awesome if the chart secret-inject/secret-inject will contain also the ability to create a serviceAccount with OIDC like aws-efs-csi-driver chart has:

https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/f89b14367e2509738dc885ab82370152c2f4cf83/charts/aws-efs-csi-driver/values.yaml#L74-L81

In addition, it will be great to have also some guidelines of how to create a serviceAccount , the IAM policy, the role trust under the README.md instead of this article. This can improve the quickstart

Latest chart (0.1.7) errors on install

If I try to install the latest chart I get an error.
Steps to reproduce:

helm repo add secret-inject https://aws-samples.github.io/aws-secret-sidecar-injector/
helm repo update
helm install secret-inject secret-inject/secret-inject

Error:

Error: YAML parse error on secret-inject/templates/.__helpers.tpl: error converting YAML to JSON: yaml: control characters are not allowed

I'm actually trying to upgrade and previously had 0.1.3 but this version seems to be no longer available.
In fact it seems that only version 0.1.7 exists in the chart repo which throws the error mentioned above.

$helm search repo secret-inject

NAME                       	CHART VERSION	APP VERSION	DESCRIPTION
secret-inject/secret-inject	0.1.7        	1          	A Helm chart for installing AWS Secret Controll...

Also the chart version is listed as 0.1.1 here (which is also not available in the chart repo):
https://github.com/aws-samples/aws-secret-sidecar-injector/blob/master/admission-controller/secret-inject/Chart.yaml#L4

TLS handshake error

Hi

I installed this with helm on an EKS cluster (1.16) and when I spin up a pod with the annotation to get the sidecar injected, they don't get the sidecar.

In the secret-inject logs I see the following errors:

secret-inject-6bb9fb7bc4-8mmq8 secret-inject-init 2020/08/03 17:23:50 http: TLS handshake error from 10.10.201.141:36620: remote error: tls: bad certificate
secret-inject-6bb9fb7bc4-k76p9 secret-inject-init 2020/08/03 17:23:55 http: TLS handshake error from 10.10.201.141:38418: remote error: tls: bad certificate
secret-inject-6bb9fb7bc4-qs989 secret-inject-init 2020/08/03 17:24:07 http: TLS handshake error from 10.10.201.141:56584: remote error: tls: bad certificate

I'm sure I'm missing something simple here but any help would be greatly appreciated!

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.