Feature parity with helm chart about cockroach-operator HOT 17 CLOSED

@johnrk We need to have full support for modifying the cockroach start command options (https://www.cockroachlabs.com/docs/stable/cockroach-start.html) - in my mind the most important ones are the ones that impact memory and cache usage, disk usage, networking, security, and locality. It's okay if the initial MVP doesn't set these by default as long as it has a mechanism for the user to declare the values.

from cockroach-operator.

It's okay if the initial MVP doesn't set these by default as long as it has a mechanism for the user to declare the values.

@keith-mcclellan @johnrk the operator currently supports the optional additionalArgs field in the CR. The string gets added to the command line, so this should be covered for the MVP.

from cockroach-operator.

@bdarnell do we have a list of functionality that is missing between the parity?

from cockroach-operator.

Just what's in the helm chart's readme. When I filed this issue I was mainly thinking about the list of configuration parameters

There's some upcoming maintenance work that we'd like to be able to avoid due to infrastructure changes in the helm project. The deadline to either do that work or deprecate the helm chart is Aug 13.

We were forced to do this work ahead of the Aug 13 deadline for other reasons, so there's no longer as much time pressure to deprecate and replace the helm chart.

from cockroach-operator.

@bdarnell that gives me a good start. Thanks!

cc @johnrk

from cockroach-operator.

We need to validate the following items:

• clusterDomain | Cluster's default DNS domain | cluster.local
• conf.attrs | CockroachDB node attributes | []
• conf.cache | Size of CockroachDB's in-memory cache | 25%
• conf.cluster-name | Name of CockroachDB cluster | ""
• conf.disable-cluster-name-verification | Disable CockroachDB cluster name verification | no
• conf.join | List of already-existing CockroachDB instances | []
• conf.max-disk-temp-storage | Max storage capacity for temp data | 0
• conf.max-offset | Max allowed clock offset for CockroachDB cluster | 500ms
• conf.max-sql-memory | Max memory to use processing SQL querie | 25%
• conf.locality | Locality attribute for this deployment | ""
• conf.single-node | Disable CockroachDB clustering (standalone mode) | no
• conf.sql-audit-dir | Directory for SQL audit log | ""
• conf.port | CockroachDB primary serving port in Pods | 26257
• conf.http-port | CockroachDB HTTP port in Pods | 8080
• image.repository | Container image name | cockroachdb/cockroach
• image.tag | Container image tag | v19.2.5
• image.pullPolicy | Container pull policy | IfNotPresent
• image.credentials | registry, user and pass credentials to pull private image | {}
• statefulset.replicas | StatefulSet replicas number | 3
• statefulset.updateStrategy | Update strategy for StatefulSet Pods | {"type": "RollingUpdate"}
• statefulset.podManagementPolicy | OrderedReady/Parallel Pods creation/deletion order | Parallel
• statefulset.budget.maxUnavailable | k8s PodDisruptionBudget parameter | 1
• statefulset.args | Extra command-line arguments | []
• statefulset.env | Extra env vars | []
• statefulset.secretMounts | Additional Secrets to mount at cluster members | []
• statefulset.labels | Additional labels of StatefulSet and its Pods | {"app.kubernetes.io/component": "cockroachdb"}
• statefulset.annotations | Additional annotations of StatefulSet Pods | {}
• statefulset.nodeAffinity | Node affinity rules of StatefulSet Pods | {}
• statefulset.podAffinity | Inter-Pod affinity rules of StatefulSet Pods | {}
• statefulset.podAntiAffinity | Anti-affinity rules of StatefulSet Pods | auto
• statefulset.podAntiAffinity.type | Type of auto anti-affinity rules | soft
• statefulset.podAntiAffinity.weight | Weight for soft auto anti-affinity rules | 100
• statefulset.nodeSelector | Node labels for StatefulSet Pods assignment | {}
• statefulset.tolerations | Node taints to tolerate by StatefulSet Pods | []
• statefulset.resources | Resource requests and limits for StatefulSet Pods | {}
• service.ports.grpc.external.port | CockroachDB primary serving port in Services | 26257
• service.ports.grpc.external.name | CockroachDB primary serving port name in Services | grpc
• service.ports.grpc.internal.port | CockroachDB inter-communication port in Services | 26257
• service.ports.grpc.internal.name | CockroachDB inter-communication port name in Services | grpc-internal
• service.ports.http.port | CockroachDB HTTP port in Services | 8080
• service.ports.http.name | CockroachDB HTTP port name in Services | http
• service.public.type | Public Service type | ClusterIP
• service.public.labels | Additional labels of public Service | {"app.kubernetes.io/component": "cockroachdb"}
• service.public.annotations | Additional annotations of public Service | {}
• service.discovery.labels | Additional labels of discovery Service | {"app.kubernetes.io/component": "cockroachdb"}
• service.discovery.annotations | Additional annotations of discovery Service | {}
• storage.hostPath | Absolute path on host to store data | ""
• storage.persistentVolume.enabled | Whether to use PersistentVolume to store data | yes
• storage.persistentVolume.size | PersistentVolume size | 100Gi
• storage.persistentVolume.storageClass | PersistentVolume class | ""
• storage.persistentVolume.labels | Additional labels of PersistentVolumeClaim | {}
• storage.persistentVolume.annotations | Additional annotations of PersistentVolumeClaim | {}
• init.labels | Additional labels of init Job and its Pod | {"app.kubernetes.io/component": "init"}
• init.annotations | Additional labels of the Pod of init Job | {}
• init.affinity | Affinity rules of init Job Pod | {}
• init.nodeSelector | Node labels for init Job Pod assignment | {}
• init.tolerations | Node taints to tolerate by init Job Pod | []
• init.resources | Resource requests and limits for the Pod of init Job | {}
• tls.enabled | Whether to run securely using TLS certificates | no
• tls.serviceAccount.create | Whether to create a new RBAC service account | yes
• tls.serviceAccount.name | Name of RBAC service account to use | ""
• tls.certs.provided | Bring your own certs scenario, i.e certificates are provided | no
• tls.certs.clientRootSecret | If certs are provided, secret name for client root cert | cockroachdb-root
• tls.certs.nodeSecret | If certs are provided, secret name for node cert | cockroachdb-node
• tls.certs.tlsSecret | Own certs are stored in TLS secret | no
• tls.init.image.repository | Image to use for requesting TLS certificates | cockroachdb/cockroach-k8s-request-cert
• tls.init.image.tag | Image tag to use for requesting TLS certificates | 0.4
• tls.init.image.pullPolicy | Requesting TLS certificates container pull policy | IfNotPresent
• tls.init.image.credentials | registry, user and pass credentials to pull private image | {}
• networkPolicy.enabled | Enable NetworkPolicy for CockroachDB's Pods | no
• networkPolicy.ingress.grpc | Whitelist resources to access gRPC port of CockroachDB's Pods | []
• networkPolicy.ingress.http | Whitelist resources to access gRPC port of CockroachDB's Pods | []

from cockroach-operator.

I do not think we need the tls init container configurations.

@chrisseto what is the init job?

from cockroach-operator.

@chrisseto the job is for the certs as well

from cockroach-operator.

There is another feature of creating more than one cluster. But we can do that using multiple crds

from cockroach-operator.

The values file is an easier read than the documentation

https://github.com/cockroachdb/helm-charts/blob/master/cockroachdb/values.yaml

from cockroach-operator.

@johnrk am I coding to duplicate complete parity? Just want to make sure that the requirements include that work. I am breaking the issue into chunks of work.

from cockroach-operator.

We'll break this one apart for the MVP.

from cockroach-operator.

The init job takes care of running cockroach init against cockroachdb-0. The operator instead execs into the pod.

from cockroach-operator.

@chrislovecnm , I just spoke with @bdarnell, and it's less urgent for us to deliver support for all of these API values because we updated the Helm Chart. We won't need to support all of these for the MVP. This can be seen as a long term to do list.

from cockroach-operator.

@keith-mcclellan , in lieu of a meeting, can you identify which API values - listed above that offer in our Helm Chart - that you believe are needed for the Operator to be production-ready? You mentioned our need to support the topology field as found in this config file: https://github.com/cockroachdb/cockroach-operator/blob/master/examples/3-nodes-insecure.yaml#L31

from cockroach-operator.

@keith-mcclellan can we close this?

from cockroach-operator.

Based on our latest findings I agree this is a bad target for us at the moment. Closing.

from cockroach-operator.