Comments (12)
I was having the same issues as @oyamin but I got it to work. There were multiple issues. First, commenting out the env
parts in my deployment
(to avoid downstream errors), I noticed the files were not getting mounted into the filesystem as per file named in jmesPath.objectAlias
as they were in another cluster where secrets were working properly. I was able to resolve this by uninstalling/reinstalling the csi-secrets-store
to their latest versions. Make sure all the csi
related pods in the kube-system
namespace roll over.
e.g.
helm -n kube-system uninstall csi-secrets-store
kubectl -n kube-system delete daemonsets/csi-secrets-store-provider-aws
Then:
$ helm repo add secrets-store-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/secrets-store-csi-driver/master/charts
$ helm -n kube-system upgrade -i csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver --set syncSecret.enabled=true
$ curl -s https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml | kubectl apply -f -
After that, when I rolled over my application pods, they did have the mounted but the env variables were still wrong. This was also visible with kubectl describe secrets/<myappsecrets>
. Explicitly deleting the k8 secret named in secretObjects.secretName
and allowing it to be recreated did the trick.
from secrets-store-csi-driver-provider-aws.
@fert-f you are almost there. Try something like this
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: csi-poc-aws-secrets
namespace: kube-public
spec:
secretObjects:
- secretName: csi-poc
type: Opaque
labels:
provider: "csi-driver"
data:
- objectName: csi-poc-username
key: username
- objectName: csi-poc-password
key: password
provider: aws
parameters:
objects: |
- objectName: "/secrets/xxx/csi-poc"
objectType: "secretsmanager"
objectVersionLabel: "AWSCURRENT"
objectAlias: csi-poc
jmesPath:
- path: "username"
objectAlias: "csi-poc-username"
- path: "password"
objectAlias: "csi-poc-password"
Notice that rather than defining the secret twice, the secret is only defined a single time, just with two data items in it.
from secrets-store-csi-driver-provider-aws.
I am almost there, but now I have separate file with correct credentials, but when I call my code, the env variables still have the full Json object:
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: aws-secret-application
spec:
provider: aws
secretObjects:
- secretName: csi-secrets
type: Opaque
labels:
provider: "csi-driver"
data:
- objectName: postgresql_database
key: main_database
- objectName: postgresql_host
key: main_host
- objectName: postgresql_user
key: main_username
- objectName: postgresql_password
key: main_password
parameters:
objects: |
- objectName: "rds-secrets"
objectType: "secretsmanager"
objectAlias: csi-secrets
jmesPath:
- path: "main_database"
objectAlias: "postgresql_database"
- path: "main_host"
objectAlias: "postgresql_host"
- path: "main_username"
objectAlias: "postgresql_user"
- path: "main_password"
objectAlias: "postgresql_password"
Which generates the files with correct values from each key:
# ls -ltr /mnt/secrets-store/
total 20
-rw-r--r-- 1 root root 8 Aug 31 21:16 postgresql_user # has the value of postgresql_user key
-rw-r--r-- 1 root root 20 Aug 31 21:16 postgresql_password # has the value of postgresql_password key
-rw-r--r-- 1 root root 48 Aug 31 21:16 postgresql_host # has the value of postgresql_host key
-rw-r--r-- 1 root root 7 Aug 31 21:16 postgresql_database # has the value of postgresql_database key
-rw-r--r-- 1 root root 156 Aug 31 21:16 csi-secrets # contains full JSON object with all the key/values
When I call them in my Deploy, i still get each env variable that contains the full json:
postgresql_database={"main_database":"value","main_host":"value","main_password":"value","main_username":"value"}
postgresql_user={"main_database":"value","main_host":"value","main_password":"value","main_username":"value"}
deployment code:
volumes:
- name: api-secret
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "aws-secret-application"
containers:
- name: api
image: "{{ .Values.images.api.image.repository }}"
imagePullPolicy: Always
ports:
- containerPort: 80
volumeMounts:
- name: api-secret
mountPath: "/mnt/secrets-store"
readOnly: true
env:
- name: postgresql_database
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_database
- name: postgresql_host
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_host
- name: postgresql_user
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_username
- name: postgresql_password
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_password
Not sure what else do try... or how to call those values from each file in the env
from secrets-store-csi-driver-provider-aws.
Faced with similar issue recently - and spend couple of hours to find out..
I found that "jmesPath" should not contain any special characters like "-".
For example "main-database" wont work. But "main_database" will work.
from secrets-store-csi-driver-provider-aws.
Tracking in #66
from secrets-store-csi-driver-provider-aws.
It is weird, this is working for me for example:
spec:
parameters:
objects: |
- objectName: "/node/secret"
objectType: "secretsmanager"
objectAlias: "node-secret"
jmesPath:
- Path: "operator_key"
ObjectAlias: "operator_key"
provider: aws
secretObjects:
- data:
- key: operator_key
objectName: operator_key
secretName: node-secret
type: Opaque
Inside the secret key operator_key
I get the content of the JSON path in operator_key
from secrets-store-csi-driver-provider-aws.
Noted that this approach works when there is single secret key. When there are two only first one is rendered:
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: csi-poc-aws-secrets
namespace: kube-public
spec:
secretObjects:
- secretName: csi-poc
type: Opaque
labels:
provider: "csi-driver"
data:
- objectName: csi-poc-username
key: username
- secretName: csi-poc
type: Opaque
labels:
provider: "csi-driver"
data:
- objectName: csi-poc-password # name of the mounted content to sync. this could be the object name or object alias
key: password
provider: aws
parameters:
objects: |
- objectName: "/secrets/xxx/csi-poc"
objectType: "secretsmanager"
objectVersionLabel: "AWSCURRENT"
objectAlias: csi-poc
jmesPath:
- path: "username"
objectAlias: "csi-poc-username"
- path: "password"
objectAlias: "csi-poc-password"
Results in secret with key username only.
AWS SM is like:
{
"username": "admin",
"password": "abc123",
}
from secrets-store-csi-driver-provider-aws.
@oyamin, I think you env should reference secret created by SecretProviderClass. Not sure where ngs-rds-secrets came from:
env:
- name: postgresql_database
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_database
- name: postgresql_host
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_host
- name: postgresql_user
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_username
- name: postgresql_password
valueFrom:
secretKeyRef:
name: csi-secrets
key: main_password
from secrets-store-csi-driver-provider-aws.
@fert-f That was the old value from the original code. But all names where the same. It would not even deploy if the name in secretKeyRef didn't exist.
I ended up just doing a single env variable, and then rewrote the code to parse the json from that env var and get each key, but still would like to see if there is a solution for this.
from secrets-store-csi-driver-provider-aws.
Hi ! same problem facing here.
bellow config is working but all username and password in same inline value
working.
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: aws-secrets
namespace: production
spec:
provider: aws
secretObjects:
- secretName: api-token
labels:
provider: "csi-driver"
data:
- key: username
objectName: api-token
- key: password
objectName: api-token
type: Opaque
parameters:
objects: |
- objectName: prod/service/token
objectType: secretsmanager
objectVersionLabel: "AWSCURRENT"
objectAlias: api-token
jmesPath:
- path: username
objectAlias: asmusername
- path: password
objectAlias: asmpassword
What i am getting is this, perhaps expected as objectName is top level object. But it is not working with jmesPath, where i am defining objectAlias, and using the same objectAlias to call in secret data.
USERNAME={"MY_API_TOKEN":"xxxxxxtest","username":"testuser","password":"testpass"}
PASSWORD={"MY_API_TOKEN":"xxxxxxtest","username":"testuser","password":"testpass"}
When i am using bellow config it does not creates secret.
notworking
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: aws-secrets
namespace: production
spec:
provider: aws
secretObjects:
- secretName: api-token
labels:
provider: "csi-driver"
data:
- key: username
objectName: asmusername
- key: password
objectName: asmpassword
type: Opaque
parameters:
objects: |
- objectName: prod/service/token
objectType: secretsmanager
objectVersionLabel: "AWSCURRENT"
objectAlias: api-token
jmesPath:
- path: username
objectAlias: asmusername
- path: password
objectAlias: asmpassword
Any luck in same scenario with anyone ?
from secrets-store-csi-driver-provider-aws.
@sharmavijay86 I've ended up creating a bash scripts in order to be put alogside app execution on the docker entrypoint - it feels okay and gives you an option to have " secret key ref".
I do have an issue maybe related to yours, when I try to create an alias and then mount it I receive following issue
Warning FailedMount 2s (x3 over 4s) kubelet, ip-10-0-2-100.us-east-1.compute.internal MountVolume.SetUp failed for volume "staging-app-alias" : rpc error: code = Unknown desc = failed to mount secrets store objects for pod namespace/app-66b577db9d-wkw8r, err: rpc error: code = Unknown desc = Name already in use for objectAlias: staging-app-alias
From my side on currently I may recommend you to do the same, as changing app logic would be a very bad practice.
Sharing my bash scripts within the entrypoint_envs.sh
#!/bin/bash
ALL_ENVS=$(cat /mnt/secrets-store/*)
echo $ALL_ENVS
INPUT_DICT=$(echo $ALL_ENVS | tr -d "{}\"\n")
INPUT_DICT_CLEARED=$(sed "s/,/ /g" <<< $INPUT_DICT)
ROWS=$(echo $INPUT_DICT_CLEARED | sort )
echo $ROWS
IFS=" "
for ROW in ${ROWS}
do
RESULTED_ENV=$(sed "s/:/=/g" <<< $ROW)
export $RESULTED_ENV
done;
So entrypoint.sh would look like:
. entrypoint_envs.sh
app start
This one is case for
volumeMounts:
- name: app-conf
mountPath: "/mnt/secrets-store"
readOnly: true
volumes:
- name: "app-conf"
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "app-conf"
Can somebody provide full solution with nginx image for example, in case if he has a success with env variables split on the side of this one provider? @cryptk
It would be really easier to understand how it works if somebody has a success of it and how to debug local case.
from secrets-store-csi-driver-provider-aws.
Faced with similar issue recently - and spend couple of hours to find out.. I found that "jmesPath" should not contain any special characters like "-". For example "main-database" wont work. But "main_database" will work.
in my case that was /, though, - worked
somebody definitely should doc up this one moment
from secrets-store-csi-driver-provider-aws.
Related Issues (20)
- When adding nodeSelector and tolerations to schedule onto a specific node, secrets can no longer be fetched. HOT 3
- Pod Identity Association not recognised by secrets store CSI driver HOT 21
- [Question] Inside .yaml, there is a way to retrieve all aws secrets without pass keys? HOT 1
- Unable to mount secret, "Failed fetching secret <secretName>: RequestCanceled: request context canceled" HOT 1
- Expose Additional Security Context Settings in Helm Chart HOT 1
- Is it possible to use this outside of EKS? HOT 3
- Allow setting `driver-writes-secrets` argument via Helm values HOT 1
- Please consider changing the object parameter format. HOT 2
- AWS provider pod failing with "panic: runtime error: invalid memory address or nil pointer dereference" HOT 2
- Provider socket does not exist if provider pod starts before driver pod HOT 2
- Wait(n=1) would exceed context deadline HOT 1
- Secret access cross account without resource based policy HOT 1
- Cross account secret access HOT 1
- Request support for adding custom CA certs to helm chart HOT 1
- Error with affinity HOT 7
- Support of Access Management Roles Anywhere HOT 2
- Consider adding a changelog or useful notes to Github releases
- Respect Audience Annotation
- secrets does not working on mysql-pods
- not creating k8s secrets
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from secrets-store-csi-driver-provider-aws.