Support for Authorization Module Webhook mode about gatekeeper HOT 16 CLOSED

Our use case is the following. We're providing a company internal Kubernetes as a service platform and therefore we're managing a few hundred Kubernetes Clusters for our customers.

We have the trade off that we want to provide our customers an experience which comes as close as possible to an upstream Kubernetes cluster with cluster-admin rights. But we also want to enforce some best practises regarding security, e.g. usage of some specific PodSecurityPolicies (non-root).

To enable usage of projects like Istio (ValidatingWebhooks/MutatingWebhooks/CRDs) or the usual ingress controller like Traefik (which requires watches on cluster-wide secrets), we're at a point where the usual whitelist-based RBAC concepts are not sufficient. Some examples:

• We want to prevent the deletion of our ValidatingWebhook, so nobody can circumvent the policy controller.

• Because we're allowing cluster-wide access to Secrets we cannot rely on RBAC to enforce the usage of some PodSecurityPolicies, because every ServiceAccount token could be stolen. So what we're doing instead, we allow the usage of PSPs only in some specific namespaces which are reserved for us (*-system)

• We want to allow creation of CRDs (and their resources), but we want to reserve a specific CRD apiGroup just for our usage.

• We want to allow full access to ClusterRoles and ClusterRolebindings accept some specific created by us or system-*.

• We want to block create/update/delete on all namespaced resources for our *-system namespaces. So nobody can mess with our deployments.

So basically. With Validating- and MutatingWebhooks we can only restrict create/update/delete on resources. Whereas with an Authorization module we have a lot more possibilities. For example we can also deny things like kubectl cp and kubectl exec.

P.S. As use cases like ours or multi tenancy are generally a problem with Kubernetes, who knows, it may even make sense to build an in-tree authorization module and/or admission controller. I think the half of the existing admission controller could be replaced by something as flexible as an OpenPolicyAgent admission controller.

from gatekeeper.

I'll added a PR. Not really meant to be merged like this (as I changed some stuff specific for my environment).

Just as a starting point for discussions if you want to support an endpoint for Authorization Module Webhook.

from gatekeeper.

In theory that is possible, can you elaborate on the use case.

from gatekeeper.

This is great I am in total agreement of the use case.

from gatekeeper.

@rite2nikhil Just a quick question. In my authorization implementation I just take the whole SubjectAccessReview and send the JSON to OPA, so I have a maximum flexibility there.
I'm now trying to write a rule to block the create of a namespace of a specific user (via ValidatingAdmissionWebhook), but this isn't currently possible because the UserInfo isn't sent to OPA in the /admit implementation, even though the information is there (in the policy-controller not in OPA).

Wouldn't it be easier to just send the whole AdmissionReview to OPA (with input as <Admission Review as JSON>)? That would make it possible to allow/deny based on additional information like Operation or UserInfo in the ValidatingWebhook.

Additionally, I think it's easier to just match against the complete JSON with input instead of the data multidimensional array, cf:

ValidatingWebhook with the current implementation to forbid services from type:Loadbalancer

deny[{
	"id": "svc-loadbalancer",
	"resource": {
		"kind": kind,
		"namespace": namespace,
		"name": name,
	},
	"resolution": {"message": "Usage of Service with type LoadBalancer is not allowed"},
}] {
	resource := data.kubernetes[kind][namespace][name]
	kind = "services"
	resource.spec.type = "LoadBalancer"
}

Authorization Module Webhook with my current implemenation to forbid deletion of nodes:

deny[{
	"id": "nodes",
	"resolution": "Your're not allowed to create/update/delete Nodes",
}] {
	input.kind = "SubjectAccessReview"
	input.apiVersion = "authorization.k8s.io/v1beta1"

	not user_system_control_plane(input.spec.user)
	input.spec.resourceAttributes.resource = "nodes"
	re_match("^(create|replace|delete|deletecollections)$", input.spec.resourceAttributes.verb)
}

What do you think?

from gatekeeper.

@sbueringer How does the /audit query look like in your case ?

from gatekeeper.

The authorization query is for example:

data.authz.deny[{"id": id, "resolution": resolution,}] 
with input as {
   "kind": "SubjectAccessReview",
   "apiVersion": "authorization.k8s.io/v1beta1",
   "metadata": {
     "creationTimestamp": null
   },
   "spec": {
     "resourceAttributes": {
       "verb": "watch",
       "group": "extensions",
       "version": "v1beta1",
       "resource": "replicasets"
     },
     "user": "system:kube-scheduler",
     "group": [
       "system:authenticated"
     ]
   },
   "status": {
     "allowed": false
   }
 }" req.method=POST req.path=/v1/authorize req.remote=10.6.0.12

whereas the /admit queries are like that:

data.admission.deny[{"id": id, "resource": {"kind": "serviceaccounts", "namespace": "test", "name": "default"}, "resolution": resolution,}] 
with data["kubernetes"]["serviceaccounts"]["test"]["default"] as {
   "metadata": {
     "name": "default",
     "namespace": "test",
     "selfLink": "/api/v1/namespaces/test/serviceaccounts/default",
     "uid": "dcb32fef-e9cf-11e8-b4c6-fa163eb59c0a",
     "resourceVersion": "609554",
     "creationTimestamp": "2018-11-16T18:46:00Z"
   },
   "secrets": [
     {
       "name": "default-token-rj7kd"
     }
   ]
 } " req.method=POST req.path=/v1/admit req.remote=10.6.0.12 

from gatekeeper.

There is a audit scenario where we want to find out what objects currently violate the policy.
We want to use the same policy to be able to able to audit as well

from gatekeeper.

Ah now I know what you mean. I have to take a look at it. Didn't use the audit endpoint yet.

from gatekeeper.

After looking at the code I think it should work with audit queries like:

# for authorization rules
data.authz.deny[{"id": id, "resolution": resolution,}] 
# for admission rules
data.admission.deny[{id": id, "resolution": resolution,}]

Whereas the difference is only in the package, but of course it isn't really necessary to split them in 2 packages if we don't want to and we could put them both in a generic package like for example policy (and if we want to differentiate between them wo could add another "parameter" like type in the rule header).

I think it's feasible to put authorization and admission rules in the same package and then match in the rules based on if it is an "AdmissionReview" or an "SubjectAccessReview" like in one of my examples:

deny[{
	"id": "nodes",
	"resolution": "Your're not allowed to create/update/delete Nodes",
}] {
	input.kind = "SubjectAccessReview"
	input.apiVersion = "authorization.k8s.io/v1beta1"
...
}

We could even support separate (or the same) rules for multiple apiVersions of SubjectAccessReview and AdmissionReview with this if necessary.

from gatekeeper.

@sbueringer, perhaps I am missing something, how will the input document get set in case of audit query ?

from gatekeeper.

@rite2nikhil No, I think it's me ;). Ok so now I've read the complete documentation. Sorry I should've done it before.

I updated my PR and added documentation, examples and unit tests. I guess we should have the discussion in the PR (#28) :)

from gatekeeper.

@sbueringer Thanks, Lets have the discussion there, I am wondering if you would have time to sync up on slack, we can have a quick discussion

from gatekeeper.

@rite2nikhil Yes of course. Pinged you on Slack just write me back when it's convenient in your timezone :)

from gatekeeper.

@sbueringer
Did not get a message from you , I am available now, are you on opa-policy-agent slack ?

from gatekeeper.

@rite2nikhil Sorry, now you should have one. But I got a do not disturb message back from Slack.

from gatekeeper.