kube-mgmt doesn't reload configmaps if opa container restarts about kube-mgmt HOT 6 CLOSED

I realized that caches need to be reloaded in addition to policies, so it is more complicated than I thought.

Maybe adding liveness probe container to pod can work, use liveness container health endpont in kube-mgmt and if opa container has no policies then liveness container reports failure and kube-mgmt restarts.

Similar to this https://github.com/kubernetes-csi/livenessprobe

from kube-mgmt.

#210 and #211 can be implemented to address the bug

from kube-mgmt.

maybe changing this to non zero value https://github.com/open-policy-agent/kube-mgmt/blob/8.0.1/pkg/configmap/configmap.go#L151

and in the new else statement here https://github.com/open-policy-agent/kube-mgmt/blob/8.0.1/pkg/configmap/configmap.go#L175-L182 (because configMap version doesn't change when OnUpdate is called by NewInformer) implement a test that will retrieve policies and if the result is empty, call https://github.com/open-policy-agent/kube-mgmt/blob/8.0.1/pkg/configmap/configmap.go#L202

from kube-mgmt.

We too see a similar issue and i did notice that the previous version we ran had a 60... v0.12.1...8.0.0#diff-6aa7780e80409d3ad0fb397be31e6f2d64ab520750d4317267f7138ebcee6606L146

from kube-mgmt.

I think these 2 might be the same problem:
#194

I think its broken with even 1 replica because when a rollout happens , it brings up a new pod and the listener triggers on the existing pod.

scenario
current deployment pod 1 is healthy
new release, pod 2 comes up, failure, annotation updated
existing pod 1 listener triggers, its already fine so it marks it as ok
pod 2 triggers again and it thinks its ok and doesn't load the rule

from kube-mgmt.

Folks if anyone is willing to work on this - I have some ideas how to approach the issue.

from kube-mgmt.