A Kubernetes Rego standard library about conftest HOT 16 CLOSED

What are folks thoughts on using Gatekeeper ConstraintTemplates to drive a k8s policy library?

Mentioned this on a thread in slack (https://openpolicyagent.slack.com/archives/CLE8VEKGT/p1564493918007900), but the team driving https://github.com/open-policy-agent/gatekeeper has put a good bit of thought into a CRD-based mechanism for authoring and consuming Kubernetes policy modules.

Deep dive here, although might be a little of of date by now: https://docs.google.com/document/d/1yC4wgpVoJj6ngYnSTtO-HeaIBl05gla562sD7qKPy3M/edit# (and I'd probably defer to folks in #kubernetes-policy for more info)

My thought: What if in addition to reading from a policy/ folder containing rego, what if conftest could traverse a bunch of K8s files in my repo, parse out the ConstraintTemplates and Constraints, and evaluate those against the rest of the files in the repo.

from conftest.

Building on top of @garethr 's questions I think a "monolith" of rego policies and functions is the best path for the foreseeable future. There is currently no good way to share and version rego dependancies - something which I consider a prerequisite to a toolchain built on top of multiple smaller code repositories.

By leveraging moda's package primative we could orgranize our code into namespaces like so:

com.instrumenta.testing.assert // testing tools
com.instrumenta.shared // shared utilities and types

com.intrumental.kubernetes // kubernetes specific types
com.instrumenta.terraform

End users could then use the code via

import data.com.instrumenta..testing.assert

assert.equal(...)

I would be interested in seeing a small reference implementation of a opa bundle server but this way beyond the scope or concerns of this project.

from conftest.

I'm very much supportive to what @dexhorthy said about parsing ConstraintTemplale files for rego rules. Because currently in order to use conftest against k8s manifests that are tracked in our repo we are forced to duplicate the entire rego field as a separate .rego file which then can be used for manifests verification in conjunction with conftest.

from conftest.

Is it the idea to create a shared OCI registry to pull rego policies from or are is it the idea to bundle the policies inside of conftest (providing some built-in functions for use in rego policies).

I love the idea btw. I have been thinking about this as well, how great it would be to have a shared registry with best practices.

from conftest.

I'd rather see these as standalone from conftest. They should be useful for any OPA usecase I think.

In the future this should work with any OCI registry (see the work going on with ORAS - https://github.com/deislabs/oras).

I've done some experiments here already, and chatted with both the ACR folks and Bundle Bar https://bundle.bar/. Some similar conversations going on in relation to Helm at the moment, and in the OCI working group.

At the moment the push, pull and config file are in Conftest, but they could be exposed in the public package for use in other projects, or be separated out at a later date.

My feeling at the moment is that we have enough of the tooling in place to demonstrate this, what we're missing is the high-quality shared content.

from conftest.

Makes sense. As to how to get started, what is your proposal? Create a public registry (Azure or Bundle Bar?) and add a place in the conftest repo where people can contribute policies?

from conftest.

My current thinking is:

1. Create a new repository under the same org
2. I already have a public ACR repository setup for the org, this currently relies on some beta features I have enabled in Azure

from conftest.

I'm traveling home from Gophercon as we speak, so I can get more in depth in a bit, but did want to chime in to say I think this makes sense. We already do something similar with our k8s yamls and Rego

For now it's just a wrap/pollyfill package to be able to check policies against k8s manifests as well as OPA admission reviews

from conftest.

I think this sounds like a great idea! There is a real lack of good rego examples out in the wild - and a shared library of type and testing utilities would help others a lot.

I'm in favor of doing this work in a separate repo - since it serves a completely different purpose than conftest. I don't think publishing bundles to an OCI needs to be a requirement either -- endusers can always submodule the repo as well if they want to use the primitives in their code.

from conftest.

Agree on the OCI distribution bit. I think it's an interesting avenue to explore, but it's not quite ready yet.

For the repository:

1. How should we structure it?
2. What should we call it?

For instance, should this aim to be just for Kubernetes, or simply for anything, with Kubernetes one of those things. A similar library would make sense for other specific tools, like Terraform. I can see pros and cons to both.

A couple of related data points (getting far ahead of things but hey):

1. Puppet, Chef, Terraform, etc. all used standalone repos per module/cookbook. Lots of flexibility but led to lots of duplication and content rot
2. Ansible content grew incredibly quickly, partly I would argue because of the content monorepo. They eventually moved away from it

Interested in others opinions.

from conftest.

I do not think we should limit it to Kubernetes. We can limit it to configuration tools. Although OPA is more general than only configuration validation as well. For example, authorization is a really interesting use case of OPA. Having a central place to share these policies would be great as well.

From a pragmatic point of view, starting out with a mono repo focused on Kubernetes at first, starting out small. Once it starts getting bigger, it might be worthwhile to split it up into multiple repositories. But I think a prerequisite for splitting would be to have a centralised registry (similar to the Terraform Module Registry or Ansible Galaxy). Building out a registry would be an avenue worth considering in my opinion.

As to how to call it: I think that depends on the scope of the project.

from conftest.

I agree with @Blokje5. The centralized registry is easy to use, we need to make an effort to pull and push commands and the registry. Folks also can push their policies and there should be also best pattern policies for all parsers. This will also reduce the dependency to policy/ folder because folks can easily download policies instantly and increase the usability of conftest

I imagine the usage of conftest for newcomers like:

wget https://...conftest
conftest pull bestpatterns
conftest test path-to-yaml-folder(test against all yamls) or combination of multi yaml files

from conftest.

I'm wondering what the main work in implementing this will be. I haven't dug into the source code, but it sounds like we could use some sort of PolicySource interface whose implementations would convert one or more of

• local rego files
• local ConstraintTemplate files
• remote rego files (e.g. via GitHub URL)
• etc

Into OPA-consumable policies for enforcement.

from conftest.

It might be worth to implement additional additive (can be provided several times with different values within one call) command line parameter which will let to extend current behavior (to parse local files) with additional parsers like ConstraintTemplates e.g. if would be really helpful in our use case.

from conftest.

Another use case might be interesting to consider is to define how deny rule is called in the rego file. It is currently hardcoded to be deny https://github.com/instrumenta/conftest/blob/9ef20efb81ea4607d47577077f5eef3382501f7c/pkg/commands/test/test.go#L29. But gatekeeper has recently renamed deny for violation. And perfectly working rego code within gatekeeper does not work with conftest due to this limitation.

from conftest.

I've now created https://github.com/instrumenta/policies as a staging area for this.

With conversations going on as well in the Slack channels mentioned in the README I'll close this issue as a way of tracking.

from conftest.