Pre populate data in opa container on startup. about kube-mgmt HOT 8 OPEN

No worries that's enough to start with, thanks!

I dove into some of the code yesterday when trying to track down exactly what the management container was doing so I'm familiar enough that I can start with that I think. If you had some specifics that would be great but I spend most of my day in Go so I can dig around. Pointers to those tests is useful though!

from kube-mgmt.

Hi @opalmer, I guess you are working on this issue. we are also waiting for this fix.

from kube-mgmt.

Hello @opalmer

I would like to propose a different view on the solution of the issue you've described.
It is my personal preference because I assume it's easier to implement.

1. The solution is introducing a new operational mode for kube-mgmt binary, let's call it "one-shot#.
2. Currently kube-mgmt works as a daemon, listening to kis events and calling OPA REST API.
3. In one-shot mode, enabled by new command line options, kube-mgmt instead will execute one time query against k8s API to get all data / policy config maps as well as other k8s resources that have to be replicated, serializing them to a directory in file system as OPA bundle. And then exit.
4. So, this new mode can be later used as an init container to OPA container. Via shared volume and OPA container configuration that will load OPA from the volume's bundle we can guarantee tht OPA container will start with pre populated data.
5. OPA container should be configured not to reload data from the file system, and the readiness probe should be configured to not allow traffik until init container completes.

from kube-mgmt.

That's pretty much exactly what I was going for, only difference would be a step 6:

6. kube-mgmt side car will start but not run the kube-mgmt binary until opa is up and has been loaded with policies.

This could be maybe be accomplished by sharing a volume between the two containers and having kube-mgmt in the opa container writing out a file after the rules are loaded. Or kube-mgmt could simply attempt to POST to rules to opa until it's ready (all the while kube-mgmt's health check would fail until that's successful).

from kube-mgmt.

That's pretty much exactly what I was going for, only difference would be a step 6:

6. kube-mgmt side car will start but not run the kube-mgmt binary until opa is up and has been loaded with policies.

This could be maybe be accomplished by sharing a volume between the two containers and having kube-mgmt in the opa container writing out a file after the rules are loaded. Or kube-mgmt could simply attempt to POST to rules to opa until it's ready (all the while kube-mgmt's health check would fail until that's successful).

This is already implemented in #210
I.e. kube-mgmt is not started until OPA readiness check passed.

from kube-mgmt.

@opalmer are you able / willing to contribute ?

from kube-mgmt.

I can try and give it a shot yeah, may not be immediate but given this is an issue I've come across in production I should be able to make time for it.

I'll dig into the code base and see if I've got questions about where to start. If you have any pointers though in terms of the bits I'll want to look at first, that would be helpful.

from kube-mgmt.

I can try and give it a shot yeah, may not be immediate but given this is an issue I've come across in production I should be able to make time for it.

I'll dig into the code base and see if I've got questions about where to start. If you have any pointers though in terms of the bits I'll want to look at first, that would be helpful.

Hello

I 'll be very appreciated if you can try to address this.

I am not golang dev, so wont suggest any useful, I am usually doing in trial and error mode.
From testing perspective - take a look at justfile, you can find command there to setup local k8s cluster, build and test project.

Also, from Helm side you will need to create unit tests for chart rendering (test/unit) and e2e tests (test/e2e),we can discuss scenarios later.

from kube-mgmt.