Comments (11)
I think those commands are incomplete for postgresql specifically. If I run all of the commands in this directory as the postgresql user in my initDBScript for zitadel, I still get errors in the setup job:
time="2023-11-14T10:38:52Z" level=info msg="setup started" caller="/home/runner/work/zitadel/zitadel/cmd/setup/setup.go:63"
time="2023-11-14T10:38:52Z" level=warning msg="postgres is currently in beta" caller="/home/runner/work/zitadel/zitadel/internal/database/postgres/config.go:65"
time="2023-11-14T10:38:52Z" level=info msg="verify migration" caller="/home/runner/work/zitadel/zitadel/internal/migration/migration.go:39" name=01_tables
time="2023-11-14T10:38:52Z" level=info msg="query failed" caller="/home/runner/work/zitadel/zitadel/internal/eventstore/repository/sql/query.go:98" error="ERROR: column \"creation_date\" does not exist (SQLSTATE 42703)"
time="2023-11-14T10:38:52Z" level=fatal msg="unable to migrate step 1" caller="/home/runner/work/zitadel/zitadel/cmd/setup/setup.go:117" error="ID=SQL-KyeAx Message=unable to filter events Parent=(ERROR: column \"creation_date\" does not exist (SQLSTATE 42703))"
zitadel-argocd-application-set.yaml for reference of all values passed in
---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: zitadel-web-app-set
namespace: argocd
annotations:
pref.argocd.argoproj.io/default-view: "network"
pref.argocd.argoproj.io/default-pod-sort: "topLevelResource"
spec:
goTemplate: true
# generator allows us to source specific values from an external k8s secret
generators:
- plugin:
configMapRef:
name: secret-var-plugin-generator
input:
parameters:
secret_vars:
- zitadel_hostname
- global_cluster_issuer
template:
metadata:
name: zitadel-web-app
annotations:
argocd.argoproj.io/sync-wave: "4"
spec:
project: zitadel
destination:
server: https://kubernetes.default.svc
namespace: zitadel
syncPolicy:
syncOptions:
- ApplyOutOfSyncOnly=true
automated:
prune: true
selfHeal: true
source:
repoURL: https://zitadel.github.io/zitadel-charts
chart: zitadel
targetRevision: 7.1.0
helm:
releaseName: zitadel
# https://github.com/zitadel/zitadel-charts/blob/main/charts/zitadel/values.yaml
values: |
replicaCount: 1
# Overrides the image tag to the latest version
# as kept up to date by renovateBot
image:
tag: "v2.35.0"
zitadel:
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
DefaultInstance:
LoginPolicy:
# disable registration AKA signups
AllowRegister: false
Database:
Postgres:
Host: zitadel-postgres-rw.zitadel.svc
Port: 5432
Database: zitadel
User:
Username: zitadel
SSL:
Mode: verify-full
#Admin:
# Username: postgres
# SSL:
# Mode: verify-full
ExternalDomain: {{ .zitadel_hostname }}
TLS:
# off until https://github.com/zitadel/zitadel-charts/pull/141
# or a similar easy fix would be merged
Enabled: false
# specifies if ZITADEL is exposed externally through TLS this
# must be set to true even if TLS is not enabled on ZITADEL itself
# but TLS traffic is terminated on a reverse proxy
# !!! Changing this after initial setup breaks your system !!!
ExternalSecure: true
ExternalPort: 443
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# setup ZITADEL with a service account
FirstInstance:
Org:
Machine:
Machine:
# Creates a service account with the name zitadel-admin-sa,
# which results in a secret 'zitadel-admin-sa' with a key 'zitadel-admin-sa.json'
Username: zitadel-admin-sa
Name: Admin
MachineKey:
Type: 1
# Reference the name of the secret that contains the masterkey.
# The key should be named "masterkey".
masterkeySecretName: "zitadel-core-key"
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: "zitadel-postgres-server-ca-key-pair"
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
# dbSslAdminCrtSecret: "zitadel-postgres-postgres-cert"
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: "zitadel-postgres-zitadel-cert"
initJob:
# Once ZITADEL is installed, the initJob can be disabled.
enabled: false
ingress:
enabled: true
className: "nginx"
annotations:
kubernetes.io/tls-acme: "true"
cert-manager.io/cluster-issuer: {{ .global_cluster_issuer }}
hosts:
- host: {{ .zitadel_hostname }}
paths:
- path: /
pathType: Prefix
tls:
- secretName: zitadel-tls
hosts:
- {{ .zitadel_hostname }}
metrics:
enabled: false
serviceMonitor:
enabled: false
readinessProbe:
enabled: true
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
livenessProbe:
enabled: true
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
This is after also making sure to pass in additional commands related to schema ownership. Here's the full set of commands I run using the CloudNativePG operator for creating postgresql cluster:
postgresql-cluster-crd-argocd-applicationset.yaml
---
# webapp is deployed 2nd because we need secrets and persistent volumes up 1st
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: zitadel-postgres-app-set
namespace: argocd
spec:
goTemplate: true
# generator allows us to source specific values from an external k8s secret
generators:
- plugin:
configMapRef:
name: secret-var-plugin-generator
input:
parameters:
secret_vars:
- zitadel_s3_endpoint
- zitadel_s3_bucket
template:
metadata:
name: zitadel-postgres-cluster
namespace: zitadel
annotations:
argocd.argoproj.io/sync-wave: "3"
spec:
project: zitadel
destination:
server: "https://kubernetes.default.svc"
namespace: zitadel
syncPolicy:
syncOptions:
- ApplyOutOfSyncOnly=true
automated:
prune: true
selfHeal: true
source:
repoURL: https://small-hack.github.io/cloudnative-pg-cluster-chart
chart: cnpg-cluster
targetRevision: 0.3.9
helm:
releaseName: zitadel-postgres-cluster
values: |
name: zitadel-postgres
instances: 1
bootstrap:
initdb:
database: zitadel
owner: zitadel
postInitApplicationSQLRefs:
secretRefs:
- name: zitadel-postgres-init-script
key: init.sql
enableSuperuserAccess: true
backup:
# barman is a utility for backing up postgres to s3
barmanObjectStore:
destinationPath: "s3://{{ .zitadel_s3_bucket }}"
endpointURL: "https://{{ .zitadel_s3_endpoint }}"
s3Credentials:
accessKeyId:
name: zitadel-db-credentials
key : "ACCESS_KEY"
secretAccessKey:
name: zitadel-db-credentials
key : "SECRET_KEY"
retentionPolicy: "30d"
certificates:
server:
enabled: true
generate: true
client:
enabled: true
generate: true
user:
enabled: true
username:
- zitadel
- postgres
scheduledBackup:
name: zitadel-pg-backup
spec:
schedule: "0 0 0 * * *"
backupOwnerReference: self
cluster:
name: pg-backup
monitoring:
enablePodMonitor: false
postgresql:
pg_hba:
- hostnossl all all 0.0.0.0/0 reject
- hostssl all all 0.0.0.0/0 cert clientcert=verify-full
It automatically creates a user named zitadel and a database named zitadel that the user zitadel owns and default has all permissions on. It uses this secret for init sql statements that it runs as the postgres super user (had to change events2
to events
everywhere due to the fact that events2
is only created if events doesn't exist even though the SQL commands in that referenced directory do not actually create a schema called events
and I'm unsure where that happens):
init.sql secret
apiVersion: v1
kind: Secret
metadata:
name: zitadel-postgres-init-script
type: Opaque
stringData:
init.sql: |
BEGIN;
CREATE SCHEMA IF NOT EXISTS eventstore;
CREATE SCHEMA IF NOT EXISTS projections;
CREATE SCHEMA IF NOT EXISTS system;
CREATE TABLE IF NOT EXISTS system.encryption_keys (id TEXT NOT NULL, key TEXT NOT NULL, PRIMARY KEY (id));
CREATE TABLE IF NOT EXISTS eventstore.events (
instance_id TEXT NOT NULL
, aggregate_type TEXT NOT NULL
, aggregate_id TEXT NOT NULL
, event_type TEXT NOT NULL
, "sequence" BIGINT NOT NULL
, revision SMALLINT NOT NULL
, created_at TIMESTAMPTZ NOT NULL
, payload JSONB
, creator TEXT NOT NULL
, "owner" TEXT NOT NULL
, "position" DECIMAL NOT NULL
, in_tx_order INTEGER NOT NULL
, PRIMARY KEY (instance_id, aggregate_type, aggregate_id, "sequence"));
CREATE INDEX IF NOT EXISTS es_active_instances ON eventstore.events (created_at DESC, instance_id);
CREATE INDEX IF NOT EXISTS es_wm ON eventstore.events (aggregate_id, instance_id, aggregate_type, event_type);
CREATE INDEX IF NOT EXISTS es_projection ON eventstore.events (
instance_id
, aggregate_type
, event_type
, "position");
CREATE SEQUENCE IF NOT EXISTS eventstore.system_seq;
CREATE TABLE IF NOT EXISTS eventstore.unique_constraints (
instance_id TEXT
, unique_type TEXT
, unique_field TEXT
, PRIMARY KEY (instance_id, unique_type, unique_field));
GRANT ALL ON SCHEMA system TO zitadel;
GRANT ALL ON ALL TABLES IN SCHEMA system TO zitadel;
GRANT ALL ON SCHEMA eventstore TO zitadel;
GRANT ALL ON ALL TABLES IN SCHEMA eventstore TO zitadel;
GRANT ALL ON SCHEMA projections TO zitadel;
GRANT ALL ON ALL TABLES IN SCHEMA projections TO zitadel;
COMMIT;
I can also run all of those SQL commands directly as the postgres user against the postgres cluster and they all work, but still don't result in a functional zitadel install via the helm chart. I think it's because the init job and setup job are not properly encapsulated.
I unfortunately am out of time for this project, so I will have to give zitadel admin access anyway, which may be an issue during an audit, as this zitadel helm chart is in Argo CD as an ApplicationSet and so we'd have to write some additional logic outside of the IaC to enable the init job declaratively, and then create a new git commit to disable the init job and remove the additional superuser cert secrets from both the zitadel ApplicationSet and the Cluster CRD that needed to be generated and then disable the admin user's external access to the cluster. This makes thing's a little harder and more manual.
I will try to come back and look into this more later and post any solutions I come to. Maybe it makes sense to just have cert manager rotate the admin cert after this, that way it can be a Kubernetes Job still managed in the same declaritve repo and directly.
from zitadel-charts.
The scripts that ZITADEL uses are here and I see there is also a description about what they do.
from zitadel-charts.
Excellent, Thanks for your sleuthing! I will try to set this up and report back later today.
from zitadel-charts.
There is also no way to open an issue that isn't a doc issue, but helm charts will continue to need improvements beyond docs as Kubernetes grows. I recommend having additional issue templates for reporting bugs other than direct security vulnerabilities.
from zitadel-charts.
Hi @jessebot
The admin user is only needed for the init phase
You can manually set up your database and skip the init phase by running zitadel setup
or zitadel start-from-setup
.
D
If this doesn't help, please reopen the issue.
from zitadel-charts.
If this doesn't help, please reopen the issue.
You have not added permissions for users to reopen issues, so I cannot do that. You should allow users more time than 1 minute to respond before closing an issue. If your metrics are calculated based on time to close an issue, your product manager should adjust them.
The admin user is only needed for the init phase
You can manually set up your database and skip the init phase by running zitadel setup or zitadel start-from-setup.
how do you do that via the helm chart though? It is not explained here
from zitadel-charts.
-
can I just not include the
zitadel.configmapConfig.Database.Postgres.Admin
object anddbSslAdminCertSecret
? -
If I do that, which SQL commands need need to be run for zitadel? In the init docs you linked it just says:
- If not already done, it grants the necessary permissions ZITADEL needs to the non privileged user.
- If they donβt exist already, it creates all schemas and some basic tables.
Here is my current values.yaml
in a collapsible:
click me for values.yaml
replicaCount: 1
# Overrides the image tag to the latest version
# as kept up to date by renovateBot
image:
tag: "v2.35.0"
zitadel:
# See all defaults here:
# https://github.com/zitadel/zitadel/blob/main/cmd/defaults.yaml
configmapConfig:
DefaultInstance:
LoginPolicy:
# disable registration AKA signups
AllowRegister: false
Database:
Postgres:
Host: zitadel-postgres-rw.zitadel.svc
Port: 5432
Database: zitadel
User:
Username: zitadel
SSL:
Mode: verify-full
Admin:
SSL:
Mode: verify-full
ExternalDomain: myzitadel.example.com
TLS:
# off until https://github.com/zitadel/zitadel-charts/pull/141
# or a similar easy fix would be merged
Enabled: false
# specifies if ZITADEL is exposed externally through TLS this
# must be set to true even if TLS is not enabled on ZITADEL itself
# but TLS traffic is terminated on a reverse proxy
# !!! Changing this after initial setup breaks your system !!!
ExternalSecure: true
ExternalPort: 443
Machine:
Identification:
Hostname:
Enabled: true
Webhook:
Enabled: false
# setup ZITADEL with a service account
FirstInstance:
Org:
Machine:
Machine:
# Creates a service account with the name zitadel-admin-sa,
# which results in a secret 'zitadel-admin-sa' with a key 'zitadel-admin-sa.json'
Username: zitadel-admin-sa
Name: Admin
MachineKey:
Type: 1
# Reference the name of the secret that contains the masterkey.
# The key should be named "masterkey".
masterkeySecretName: "zitadel-core-key"
# The Secret containing the CA certificate at key ca.crt needed for establishing secure database connections
dbSslCaCrtSecret: "zitadel-postgres-server-cert"
# The db admins secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslAdminCrtSecret: "zitadel-postgres-server-cert"
# The db users secret containing the client certificate and key at tls.crt and tls.key needed for establishing secure database connections
dbSslUserCrtSecret: "zitadel-postgres-zitadel-cert"
ingress:
enabled: true
className: "nginx"
annotations:
kubernetes.io/tls-acme: "true"
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: myzitadel.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: zitadel-tls
hosts:
- myzitadel.example.com
metrics:
enabled: false
serviceMonitor:
enabled: false
readinessProbe:
enabled: true
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
livenessProbe:
enabled: true
initialDelaySeconds: 20
periodSeconds: 15
failureThreshold: 6
from zitadel-charts.
You have not added permissions for users to reopen issues
Ok, unfortunately, I couldn't find an option to allow everybody to reopen issues π I will not close them immediately anymore.
can I just not include the zitadel.configmapConfig.Database.Postgres.Admin object and dbSslAdminCertSecret?
Yes, if you don't intend to run zitadel init
.
which SQL commands need need to be run for zitadel
We are not maintaining and publishing such a list. I recommend you let ZITADEL initialize the DB and then remove the admin credentials. Else, you can also initialize a local database and record the SQL statements.
from zitadel-charts.
Ok, unfortunately, I couldn't find an option to allow everybody to reopen issues π I will not close them immediately anymore.
thanks for looking!
Yes, if you don't intend to run zitadel init.
I will try that.
We are not maintaining and publishing such a list. I recommend you let ZITADEL initialize the DB and then remove the admin credentials. Else, you can also initialize a local database and record the SQL statements.
wait, then how can I setup the zitadel database and user ahead of time if you don't tell me what permissions it needs? I don't want to let zitadel have admin access to my production cluster. This is a bit confusing. I just need to know what grants you need and what schemas you may need access to. If that is closed source info, this project is not fully open source.
from zitadel-charts.
@jessebot I would be interested in a solution for this as well. Right now, my workaround would be to have a separate Postgres server for Zitadel ...
from zitadel-charts.
Related Issues (20)
- Test unauthenticated gRPC and gRPC-Web calls
- [Feature] Add the possibility to add CA certificates as volumes/config HOT 7
- Values from dependency chart not being propagated to zitadel HOT 4
- feat: Add support for topologySpreadConstraints
- selectorLabel "app.kubernetes.io/version" disregards .Values.image.tag override
- Deployment probes fail when using TLS Enabled
- `Errors.Org.PolicyNotExisting` after upgrading from chart 7.1.0 to 7.3.0 HOT 3
- Provision "root" jwt_token on installation HOT 7
- User could not be found when used DefaultInstance in the chart HOT 3
- feat: Add annotations to deployment / configMap resources HOT 2
- Allow default annotations to be overriden in a clean way HOT 2
- [Bug]: Helm upgrade fails because the databases already exists (zitadel-setup) HOT 1
- [Bug]: masterkey or masterkeySecretName values not recognized when used in an umbrella Chart HOT 17
- Add `--init-projections=true` to setup job HOT 2
- [Bug]: when config.yaml is stored in a kubernetes secret certain values are ignored HOT 3
- Add option to add custom labels to the k8s pod
- chown init-container should run as non-root HOT 4
- chown init-container should not render without secret HOT 5
- No imagePullPolicy for self signed SSL certificate container HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zitadel-charts.