openlab-red / hashicorp-vault-for-openshift Goto Github PK

Can anyone help for this? thx a lot! based on OCP 4.3
Steps:
1.oc new-project hashicorp
2.git clone https://github.com/hashicorp/consul-helm.git /tmp/consul-helm
3.helm install --name consul /tmp/consul-helm
4.oc -n hashicorp patch sts consul-consul-server --type json --patch="[
{"op": "remove", "path": "/spec/template/spec/securityContext"}
]"
5.oc -n hashicorp create route reencrypt consul --port=8500 --service=consul-consul-server
6.git clone https://github.com/openlab-red/hashicorp-vault-for-openshift.git
7.cd hashicorp-vault-for-openshift
8.oc apply -f ./vault/ha/install/
(vault-0 vault-1 vault-2 are running but READY are 0/1)
(logs as following ==> Vault server configuration:
Api Address: http://10.130.2.83:8200
Cgo: disabled
Cluster Address: https://10.130.2.83:8201
Listener 1: tcp (addr: "[::]:8200", cluster address: "[::]:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "enabled")
Log Level: info
Mlock: supported: true, enabled: false
Recovery Mode: false
Storage: consul (HA available)
Version: Vault v1.3.2
==> Vault server started! Log data will stream in below:
2020-04-14T09:49:01.939Z [INFO] proxy environment: http_proxy= https_proxy= no_proxy=
2020-04-14T09:49:01.940Z [WARN] storage.consul: appending trailing forward slash to path)

9.oc rsh vault-0
10.echo $VAULT_ADDR (show http://127.0.0.1:8200)
11.vault operator init -key-shares=1 -key-threshold=1

Results:
Error checking seal status: Error making API request.
URL: GET http://127.0.0.1:8200/v1/sys/seal-status
Code: 400. Raw Message:
Client sent an HTTP request to an HTTPS server.

Hello,
When I run the helm install standalone . -f override-standalone.yaml

It gives Chart.yaml is missing

I'm getting:

Error initializing: Put https://xxx/v1/sys/init: dial tcp: lookup xxx on 10.0.0.2:53: server misbehaving.

Not well-versed enough in the product to know what to check next. Please help.

Howdy. The project had forked Vault helm charts 3 years ago, and sadly hasn't updated them since then. A lot has changed in Vault, the Vault Helm Charts and related artifacts. Can you please update the fork with the latest updates? Or integrate your changes into the upstream chart? Thanks! :)

Ref: "This includes for now only the possibility to create OpenShift passthrough routes, but we forked these official charts and added the possibility to use reencrypt routes and to rely on OpenShift internal CA for signing Vault"

Is there a particular release that I can reference that plays well with OpenShift 3.11?

When I follow this runbook for setting up a standalone vault using helm chart: https://github.com/openlab-red/hashicorp-vault-for-openshift/blob/master/vault/standalone/README.md, I encountered the following error:

# helm install standalone . -f override-standalone.yaml
coalesce.go:196: warning: cannot overwrite table with non table for openshift (map[enabled:false])
coalesce.go:196: warning: cannot overwrite table with non table for openshift (map[enabled:false])
coalesce.go:196: warning: cannot overwrite table with non table for openshift (map[enabled:false])
Error: template: vault/charts/consul/templates/server-statefulset.yaml:62:24: executing "vault/charts/consul/templates/server-statefulset.yaml" at <.Values.global.openshift.enabled>: can't evaluate field enabled in type interface {}

However, the above error seems does not make sense, because it reported error like <.Values.global.openshift.enabled>: can't evaluate field enabled in type interface {}, but actually as following shows, in templates/server-statefulset.yaml file, there's only section Values.global.openshift, there's no such section like Values.global.openshift.enabled

      volumes:
        {{ template "vault.volumes" . }}
        - name: home
          emptyDir: {}
        {{- if .Values.global.openshift }}
        - name: {{ .Release.Name }}-vault-cert
          secret:
        {{- if ne .mode "ha" }}
            secretName: {{ .Release.Name }}-vault-cert
        {{- else }}
            secretName: {{ .Release.Name }}-vault-cert-active
        {{- end }}
            defaultMode: 420
        {{- end }}

I found a workaround is that for override-standalone.yaml , after I changed

global:
  tlsDisable: false
  openshift: true

To:

global:
  tlsDisable: false
  openshift:
    enabled: true

Then everything worked well, and I was able to create a standalone vault service. Could you evaluate here if it's a bug or anything I missed? Thanks.

mutatingwebhookconfigurations.admissionregistration.k8s.io "vault-agent-injector-cfg" is forbidden: User "system:serviceaccount:hashicorp:vault-injector" cannot patch resource "mutatingwebhookconfigurations" in API group "admissionregistration.k8s.io" at the cluster scope

To fix this issue, must add patch privilege in file /vault/injector/install/010-ClusterRole-vault-injector.yaml :
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: vault-injector-clusterrole
labels:
app.kubernetes.io/name: vault-injector
app.kubernetes.io/instance: vault
rules:

• apiGroups: ["admissionregistration.k8s.io"]
  resources: ["mutatingwebhookconfigurations"]
  verbs:
  • "get"
  • "list"
  • "patch"
  • "watch"

I have a deployment which needs to run with 'anyuid' permission. So I add the service account to 'anyuid' SCC (oc adm policy add-scc-to-user anyuid -z default -n mynamespace). When I deploy this with vault injection annotations, the deployment never works. If I look into the event logs, I see this error

10s Warning FailedCreate replicaset/vault-agent-demo-5fc7cb5b6b Error creating: Internal error occurred: failed calling webhook "vault.hashicorp.com": Post https://vault-agent-injector-svc.appopsmgmt.svc:443/mutate?timeout=5s: EOF

And the vault-injector pod has these logs

2020-06-03T05:57:07.371Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=5s
2020/06/03 05:57:07 http: panic serving 172.30.79.135:51412: runtime error: invalid memory address or nil pointer dereference
goroutine 707 [running]:
net/http.(*conn).serve.func1(0xc0003fa6e0)
/usr/local/go/src/net/http/server.go:1767 +0x139
panic(0x1111380, 0x1c04cf0)
/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/hashicorp/vault-k8s/agent-inject/agent.Init(0xc00007f500, 0xc000042049, 0xb, 0xc000044018, 0x21, 0x1267c73, 0xf, 0xc00050a380, 0xb, 0x0, ...)
/Users/mmascia/dev/lab/go/src/github.com/hashicorp/vault-k8s/agent-inject/agent/annotations.go:253 +0x550
github.com/hashicorp/vault-k8s/agent-inject.(*Handler).Mutate(0xc000313500, 0xc0001f2f20, 0xb00)
/Users/mmascia/dev/lab/go/src/github.com/hashicorp/vault-k8s/agent-inject/handler.go:151 +0x83f
github.com/hashicorp/vault-k8s/agent-inject.(*Handler).Handle(0xc000313500, 0x1408d20, 0xc0001969a0, 0xc000439400)
/Users/mmascia/dev/lab/go/src/github.com/hashicorp/vault-k8s/agent-inject/handler.go:87 +0x721
net/http.HandlerFunc.ServeHTTP(0xc0002c5c50, 0x1408d20, 0xc0001969a0, 0xc000439400)
/usr/local/go/src/net/http/server.go:2007 +0x44
net/http.(*ServeMux).ServeHTTP(0xc000302700, 0x1408d20, 0xc0001969a0, 0xc000439400)
/usr/local/go/src/net/http/server.go:2387 +0x1bd
net/http.serverHandler.ServeHTTP(0xc000197260, 0x1408d20, 0xc0001969a0, 0xc000439400)
/usr/local/go/src/net/http/server.go:2802 +0xa4
net/http.(*conn).serve(0xc0003fa6e0, 0x140c360, 0xc0000d8f00)
/usr/local/go/src/net/http/server.go:1890 +0x875
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2928 +0x384

If I remove my service account from 'anyuid' SCC (oc adm policy remove-scc-from-user anyuid system:serviceaccount:mynamespace:default) and if I deploy again, it works perfectly fine and I am able to see the injected secrets. (even though my application doesn't work since it needs 'anyuid' permissions)

I tried creating a new service account instead of using the 'default' one, but the behaviour is exactly same. Injector fails if the service account is part of 'anyuid' SCC. Is this a known issue, any fix or workaround?

OpenShift 4.3.19
Injector image: openlabred/vault-k8s:0.3.1
Vault image: vault:1.4.1

Hi, I'm trying to deploy the vault injector sidecar on it own so it can communicate with an external vault instance. I am running on OCP 4.3.10

With the injector deployed I am attempting to run a sample app but the pod does not come up and has the below error:

Internal error occurred: failed calling webhook "vault.hashicorp.com": Post https://vault-agent-injector-svc.hashicorp.svc:443/mutate?timeout=5s: context deadline exceeded

Any Ideas?, looking online its previously happened when firewall ported needed to be opened

The vault container failed to run with following error logs:

chown: /vault/file: Operation not permitted
chown: /vault/file: Operation not permitted