Comments (34)
@pksurferdad Little late to the game, but I think your problem is your paths. You have "/secrets/dev/poc-secret", but you should have "secrets/data/dev/poc-secret".
Hope that helps.
I've been able to get this to work in a more generic fashion (i.e. you don't have to explicitly list out the env variables you want to set). Instead it creates variables based on the vault keys.
NOTE: Avoid dots in your vault keys as this doesn't play well with environment variable names.
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
labels:
app: vault-agent-demo
spec:
selector:
matchLabels:
app: vault-agent-demo
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "app-user"
vault.hashicorp.com/agent-inject-secret-poc-secret.env: "secrets/data/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secret.env: |
{{- with secret "secret/data/dev/poc-secret" -}}
{{ range $Key, $Value := .Data.data }}export {{$Key}}="{{$Value}}"
{{ end }}
{{- end -}}
labels:
app: vault-agent-demo
spec:
serviceAccountName: vault-auth
containers:
- name: app
image: jweissig/app:0.0.1
args: ["bin/sh", "-c", "source /vault/secrets/poc-secret.env"]
from vault-k8s.
@hoantran3108 Here is the command instruction that I used to do this with an influxdb image. Of course the final command will depend on the configuration of your particular image.
command: ["/bin/bash", "-c", "source /vault/secrets/influxdb_env && ./entrypoint.sh influxd"]
from vault-k8s.
Hi @arunporwal, this is not possible with the injector today, but we are considering having an option that would generate an envfile which your container could source.
One idea could be to use custom templates to create files with a common filename structure:
vault.hashicorp.com/agent-inject-template-db.env: | {{- with secret "database/creds/db-app" -}} #!/bin/bash export USERNAME={{ .Data.username }} export PASSWORD={{ .Data.password }} {{- end }} --- MORE SECRETS HERE --- ...
Then your container could source all the files:
source /vault/secrets/*.env
This use case works well for static secrets.
In my case I try to use this approach with dynamic secrets but after some time the secret change and the app on that container stop working.
You guys have some suggestions on how to deal with this use case?
from vault-k8s.
Sourcing env file is not an option for distroless containers.
from vault-k8s.
Hi guys any solution?
from vault-k8s.
Hi @arunporwal, this is not possible with the injector today, but we are considering having an option that would generate an envfile which your container could source.
One idea could be to use custom templates to create files with a common filename structure:
vault.hashicorp.com/agent-inject-template-db.env: |
{{- with secret "database/creds/db-app" -}}
#!/bin/bash
export USERNAME={{ .Data.username }}
export PASSWORD={{ .Data.password }}
{{- end }}
--- MORE SECRETS HERE ---
...
Then your container could source all the files: source /vault/secrets/*.env
from vault-k8s.
@jasonodonnell : Thanks for the update. Really appreciated.
But how to source that file in container shell before container it start. There are only 2 lifecycle container hook i believe. ( Prestop and Poststop ) both can be used after after container start or stop and there is no hook which say run it before container start.
Or if we use init container , i think that will also not worked because it will source env in init container shell only. any idea on this ?
i tried below but getting error:
Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"source\": executable file not found in $PATH": unknown
tried as :
command:
- source "/vault/secrets/test.env"
or
command:
- source
- "/vault/secrets/test.env"
from vault-k8s.
I think you would need to do:
command: ["/bin/sh", "-ec"]
args: ["source /vault/secrets/test.env"]
from vault-k8s.
Hello, have news to this request?
from vault-k8s.
The same question.
Do you have news?
from vault-k8s.
@jasonodonnell : Are we going to have this feature in next release ?
rather than injecting using source file.
from vault-k8s.
Related feature request: #14
from vault-k8s.
@arunporwal Did you manage to source the env file using the command, args in pod's specification? For me, using the command and args would overwrite the Dockerfile ENTRYPOINT so the container is not able to run as expected.
from vault-k8s.
@taffarel-gaivota do you have an example of how you sourced the variables in your example here #45 (comment)? Specifically, I'm not exactly sure what should be included in the <entrypoint script>
containers:
- name: web
image: alpine:latest
args: ["sh", "-c", "source /vault/secrets/config && entrypoint script>"]
from vault-k8s.
Hi @jasonodonnell , i see how this #45 (comment) is supposed to work now and i followed the instructions posted here; however, when i exec into the pod, i'm not seeing the template file in the vault/secrets
volume mount and only see the fetched secret. there are no errors in the vault-agent-init
logs. i also included the pod deployment. any ideas on why the template file is not being created?
vault-agent-init logs
2020-10-12T14:28:11.009Z [INFO] sink.file: creating file sink
2020-10-12T14:28:11.009Z [INFO] sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
2020-10-12T14:28:11.009Z [INFO] auth.handler: starting auth handler
2020-10-12T14:28:11.009Z [INFO] auth.handler: authenticating
2020-10-12T14:28:11.009Z [INFO] template.server: starting template server
2020/10/12 14:28:11.009896 [INFO] (runner) creating new runner (dry: false, once: false)
2020-10-12T14:28:11.010Z [INFO] sink.server: starting sink server
2020/10/12 14:28:11.010248 [INFO] (runner) creating watcher
2020-10-12T14:28:11.020Z [INFO] auth.handler: authentication successful, sending token to sinks
2020-10-12T14:28:11.020Z [INFO] auth.handler: starting renewal process
2020-10-12T14:28:11.020Z [INFO] sink.file: token written: path=/home/vault/.vault-token
2020-10-12T14:28:11.020Z [INFO] sink.server: sink server stopped
2020-10-12T14:28:11.020Z [INFO] sinks finished, exiting
2020-10-12T14:28:11.020Z [INFO] template.server: template server received new token
2020/10/12 14:28:11.020499 [INFO] (runner) stopping
2020/10/12 14:28:11.020520 [INFO] (runner) creating new runner (dry: false, once: false)
2020/10/12 14:28:11.020633 [INFO] (runner) creating watcher
2020/10/12 14:28:11.020670 [INFO] (runner) starting
2020-10-12T14:28:11.063Z [INFO] auth.handler: renewed auth token
2020/10/12 14:28:11.132482 [INFO] (runner) rendered "(dynamic)" => "/vault/secrets/poc-secret"
2020/10/12 14:28:11.132515 [INFO] (runner) stopping
2020-10-12T14:28:11.132Z [INFO] template.server: template server stopped
pod deployment
some-secret
is the key of the secret I created in vault.
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
labels:
app: vault-agent-demo
spec:
selector:
matchLabels:
app: vault-agent-demo
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "app-user"
vault.hashicorp.com/tls-skip-verify: "true"
vault.hashicorp.com/agent-inject-secret-poc-secret: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-secrets.env: |
{{- with secret "secrets/dev/poc-secret" -}}
#!/bin/bash
export poc_secret={{ .Data.some-secret }}
{{- end }}
labels:
app: vault-agent-demo
spec:
serviceAccountName: vault-auth
containers:
- name: app
image: jweissig/app:0.0.1
args: ["sh", "-c", "source /vault/secrets/secrets.env"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-auth
labels:
app: app
from vault-k8s.
Hi @pksurferdad, your annotations are wrong. To override the default template, your key names need to have the same identifier.
What it should be:
vault.hashicorp.com/agent-inject-secret-poc-secrets.env: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secrets.env:
{{- with secret "secrets/dev/poc-secret" -}}
#!/bin/bash
export poc_secret={{ .Data.some-secret }}
{{- end }}
Here we're saying "for the secret we're identifying as poc-secrets.env
, attach this custom template which creates an environment variable template file.
from vault-k8s.
@jasonodonnell that makes sense, but unfortunately, i'm still only seeing the poc-secret
file in /vault/secrets
and not the poc-secret.env
file. below is the updated deployment. I assume templating is enabled by default in the vault-agent-injector
? I used the latest helm chart to do the install (single/stand-alone k8s AWS EKS cluster).
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
labels:
app: vault-agent-demo
spec:
selector:
matchLabels:
app: vault-agent-demo
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "app-user"
vault.hashicorp.com/tls-skip-verify: "true"
vault.hashicorp.com/agent-inject-secret-poc-secret: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secret.env: |
{{- with secret "secrets/dev/poc-secret" -}}
#!/bin/bash
export poc_secret={{ .Data.some-secret }}
{{- end }}
labels:
app: vault-agent-demo
spec:
serviceAccountName: vault-auth
containers:
- name: app
image: jweissig/app:0.0.1
args: ["sh", "-c", "source /vault/secrets/poc-secret.env"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-auth
labels:
app: app
from vault-k8s.
@pksurferdad Your annotations are still wrong. They both need to be the same (you're missing .env
). We use the identifier as the filename but they need to be same.
vault.hashicorp.com/agent-inject-secret-poc-secrets.env: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secrets.env:
{{- with secret "secrets/dev/poc-secret" -}}
#!/bin/bash
export poc_secret={{ .Data.some-secret }}
{{- end }}
- vault.hashicorp.com/agent-inject-secret-poc-secrets.env
- vault.hashicorp.com/agent-inject-template-poc-secrets.env
from vault-k8s.
yep, that was it @jasonodonnell. sorry about that. i fought a syntax error in the template for a bit (doesn't like some-secret
for the key and had to change it to somesecret
). below is the updated deployment. now on to figuring out why the source
command on the args:
parm is not firing when the container starts up.
deployment yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
labels:
app: vault-agent-demo
spec:
selector:
matchLabels:
app: vault-agent-demo
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "app-user"
vault.hashicorp.com/tls-skip-verify: "true"
vault.hashicorp.com/agent-inject-secret-poc-secret.env: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secret.env: |
{{- with secret "secrets/dev/poc-secret" -}}
#!/bin/bash
export poc_secret="{{ .Data.somesecret }}"
{{- end }}
labels:
app: vault-agent-demo
spec:
serviceAccountName: vault-auth
containers:
- name: app
image: jweissig/app:0.0.1
args: ["/bin/sh", "-c", "source /vault/secrets/poc-secret.env"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-auth
labels:
app: app
from vault-k8s.
has anyone gotten this to work, to seed the running pod with env vars from vault? the pod starts up fine and the secrets are successfully fetched from vault, but are not available on the running pod. i can exec into the pod and manually run source /vault/secrets/poc-secret.env
and sets the env var correctly.
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
labels:
app: vault-agent-demo
spec:
selector:
matchLabels:
app: vault-agent-demo
replicas: 1
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "app-user"
vault.hashicorp.com/tls-skip-verify: "true"
vault.hashicorp.com/agent-inject-secret-poc-secret.env: "secrets/dev/poc-secret"
vault.hashicorp.com/agent-inject-template-poc-secret.env: |
{{- with secret "secrets/dev/poc-secret" -}}
export poc_secret="{{ .Data.somesecret }}"
{{- end }}
labels:
app: vault-agent-demo
spec:
serviceAccountName: vault-auth
containers:
- name: app
image: jweissig/app:0.0.1
args: ["bin/sh", "-c", "source /vault/secrets/poc-secret.env"]
from vault-k8s.
from vault-k8s.
See #14 and https://www.vaultproject.io/docs/platform/k8s/injector/examples#environment-variable-example
from vault-k8s.
@cmbaatz Nice addition, sadly while the file is there ready to be sourced on entering the container e.g.
kubectl exec -it app-example-pod -c app /bin/sh
and running env
No new env variables
However the file is there and templating applied ..
cat /vault/secrets/poc-secret
export password="password2"
export username="user1"
from vault-k8s.
Better solution here => https://banzaicloud.com/blog/inject-secrets-into-pods-vault-revisited
from vault-k8s.
Small update about environment variables here: #14 (comment).
from vault-k8s.
@jasonodonnell i want to inject to mysql pod, MYSQL_ROOT_PASSWORD. do you have any recommend. image Mysql doesnot have source cmd, so the above solution doesnt work
from vault-k8s.
@dongnh-devops MySQL docker image supports secrets via files, so I'd suggest just using that instead.
from vault-k8s.
@jasonodonnell i know but i wanna test how vault inject env into a pod so i test with mysql pod but it havent worked yet
from vault-k8s.
has anyone gotten this to work, to seed the running pod with env vars from vault? the pod starts up fine and the secrets are successfully fetched from vault, but are not available on the running pod. i can exec into the pod and manually run
source /vault/secrets/poc-secret.env
and sets the env var correctly.apiVersion: apps/v1 kind: Deployment metadata: name: app labels: app: vault-agent-demo spec: selector: matchLabels: app: vault-agent-demo replicas: 1 template: metadata: annotations: vault.hashicorp.com/agent-inject: "true" vault.hashicorp.com/role: "app-user" vault.hashicorp.com/tls-skip-verify: "true" vault.hashicorp.com/agent-inject-secret-poc-secret.env: "secrets/dev/poc-secret" vault.hashicorp.com/agent-inject-template-poc-secret.env: | {{- with secret "secrets/dev/poc-secret" -}} export poc_secret="{{ .Data.somesecret }}" {{- end }} labels: app: vault-agent-demo spec: serviceAccountName: vault-auth containers: - name: app image: jweissig/app:0.0.1 args: ["bin/sh", "-c", "source /vault/secrets/poc-secret.env"]
@pksurferdad Did you get the solution for this?
from vault-k8s.
has anyone gotten this to work, to seed the running pod with env vars from vault? the pod starts up fine and the secrets are successfully fetched from vault, but are not available on the running pod. i can exec into the pod and manually run
source /vault/secrets/poc-secret.env
and sets the env var correctly.apiVersion: apps/v1 kind: Deployment metadata: name: app labels: app: vault-agent-demo spec: selector: matchLabels: app: vault-agent-demo replicas: 1 template: metadata: annotations: vault.hashicorp.com/agent-inject: "true" vault.hashicorp.com/role: "app-user" vault.hashicorp.com/tls-skip-verify: "true" vault.hashicorp.com/agent-inject-secret-poc-secret.env: "secrets/dev/poc-secret" vault.hashicorp.com/agent-inject-template-poc-secret.env: | {{- with secret "secrets/dev/poc-secret" -}} export poc_secret="{{ .Data.somesecret }}" {{- end }} labels: app: vault-agent-demo spec: serviceAccountName: vault-auth containers: - name: app image: jweissig/app:0.0.1 args: ["bin/sh", "-c", "source /vault/secrets/poc-secret.env"]
@pksurferdad Did you get the solution for this?
Hi @abhishekkamal96,
It does work in a way, the shell which is used to source the environment variables is also used to run the application and scripts or code referencing those environment variables will detect those environment variables (I have tested it myself). In effect when you make another shell and the environment variables are not loaded in and have to be sourced again. I do not know if this methodology is optimal for our company's use case. Let us see how good the CSI driver is.
from vault-k8s.
Using this approach:
{{- with secret "internal/data/development/api" -}}
{{- range $key, $value := .Data.data }}
export {{ $key }}={{ $value }}
{{- end }}
{{- end -}}
command: ["/bin/sh", "-c"]
args:
[ "for f in /vault/secrets/*.env; do source $f; done && npm start" ]
from vault-k8s.
Hi folks, as a few have pointed out, make sure you're setting command
for your container and not just args
. We've updated the injector example accordingly: https://developer.hashicorp.com/vault/docs/platform/k8s/injector/examples#environment-variable-example
from vault-k8s.
It worked for me using: command: ["/bin/bash", "-c", ". /vault/secrets/grafana.sh && . /run.sh"]
from vault-k8s.
Hi folks, it looks like we have a couple workable options here so I'll go ahead and close this.
Also note that we recently released a beta of vault-secrets-operator that will sync Vault secrets to Kubernetes secrets, which can then be used as environment variables natively: [blog] [docs]
from vault-k8s.
Related Issues (20)
- Injected config tries to use IRSA token instead of the k8s service account token
- Webhook tries to add initContainer during UPDATE HOT 4
- Stuned deleting of a pod whose parents are job.
- vault.hashicorp.com/agent-init-first does not work with init containers coming from annotations
- Azure authentication method doesn't work with federated token
- Support for an agent-image built FROM scratch
- Auth config block can support common arguments from env and flags
- Tokens not revoked on Vault Agent Shutdown created via a Job using the /agent/v1/quit endpoint HOT 3
- Pipeline Request: Rebuild Dockerhub Image HOT 1
- Support for a securityContext.seccompProfile configuration HOT 1
- Support vault secret inject while the main pod "automountServiceAccountToken" set false HOT 1
- [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. HOT 1
- Sidecar agent does not handle manually rotated static database secret
- Inject the Agent as a native sidecar HOT 2
- Allow patching the Agent's configuration HOT 3
- vault agent export container port for scape metrics through podmonitor
- Problems encountered using consul as a storage backend
- Problems when using GCP KMS decryption
- Injected Agent config uses `token_path` instead of `path` for `jwt` auth method
- TLS handshake error Vault on EKS with Istio
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vault-k8s.