Giter VIP home page Giter VIP logo

grafain's Introduction

Grafain

TravisCI GoDoc

Go Report Card

LoC

Secure Kubernetes admission policy management and replication across data centers.

Grafain is a kubernetes policy and permission admission controller webhook server. It receive requests from the admission controller via webhooks and returns decisions based on internal rules.

What makes Grafain special is that it uses a Blockchain to store and replicate it's state. Some attributes are:

At this stage the project is a Proof Of Concept. Most elements of the blockchain are fully implemented while the rule engine and admission policies for Kubernetes are not. What you can expect to work:

Supported policies

  • Accept only white listed images

🚧 There is not much doc yet, but please also check the Wiki.

Server

Quickstart with Minikube

minikube start                  # start environment
cd contrib/k8s                  
./seed-cluster.sh               # deploy grafain components
kubectl get pods                # check grafain pod is running
kubeclt logs -f grafain-0       # watch log

# deploy a random pod -> should fail
kubectl create deployment microbot --image=dontrebootme/microbot:v1

Client

The grafaincli is a commend line client to interact with the running grafaind server through the Blockchain engine.

# build CLI client
go build ./cmd/grafaincli

# create a new private key
./grafaincli mnemonic | ./grafaincli keygen -key $(pwd)/my_grafain.key

# set endpoint address for the grafain cli
export GRAFAINCLI_TM_ADDR=$(minikube service grafain-rpc --url)

# add a new artifact to the system
./grafaincli create-artifact -image="foo/bar:any" -digest="anyValidDigest" \
    | ./grafaincli sign -key=$(pwd)/my_grafain.key \
    | ./grafaincli submit

# query all artifacts
./grafaincli query -path=/artifacts

# query by image
./grafaincli query -path=/artifacts/image -data foo/bar:any

# delete artifact by internal id (=key)
./grafaincli del-artifact -id=1 \
    | ./grafaincli sign -key=$(pwd)/my_grafain.key \
    | ./grafaincli submit

Manual testing the admission hook

HOOK_URL=$(minikube service grafain-hook --url --https)

curl -X POST -k -H "Content-Type: application/json"  -d '
{
  "kind": "AdmissionReview",
  "apiVersion": "admission.k8s.io/v1beta1",
  "request": {
    "uid": "181988ef-db4e-4023-9af8-ea1121ccfa9a",
    "kind": {
      "group": "",
      "version": "v1",
      "kind": "Pod"
    },
    "resource": {
      "group": "",
      "version": "v1",
      "resource": "pods"
    },
    "requestKind": {
      "group": "",
      "version": "v1",
      "kind": "Pod"
    },
    "requestResource": {
      "group": "",
      "version": "v1",
      "resource": "pods"
    },
    "name": "microbot5-85b6bcc585-zws9j",
    "namespace": "default",
    "operation": "CREATE",
    "userInfo": {
      "username": "system:serviceaccount:kube-system:replicaset-controller",
      "uid": "ce7d5264-51d2-4998-a1db-9d7cd751d167",
      "groups": [
        "system:serviceaccounts",
        "system:serviceaccounts:kube-system",
        "system:authenticated"
      ]
    },
    "object": {
      "kind": "Pod",
      "apiVersion": "v1",
      "metadata": {
        "name": "microbot5-85b6bcc585-zws9j",
        "generateName": "microbot5-85b6bcc585-",
        "namespace": "default",
        "uid": "bcc03889-33be-4390-b047-01d13cf4f51e",
        "creationTimestamp": "2019-10-13T12:14:13Z",
        "labels": {
          "app": "microbot5",
          "pod-template-hash": "85b6bcc585"
        },
        "ownerReferences": [
          {
            "apiVersion": "apps/v1",
            "kind": "ReplicaSet",
            "name": "microbot5-85b6bcc585",
            "uid": "1acfcf3c-2fee-4b31-a4f0-480f4d363ea8",
            "controller": true,
            "blockOwnerDeletion": true
          }
        ]
      },
      "spec": {
        "volumes": [
          {
            "name": "default-token-th7qf",
            "secret": {
              "secretName": "default-token-th7qf"
            }
          }
        ],
        "containers": [
          {
            "name": "microbot",
            "image": "dontrebootme/microbot:v1",
            "resources": {},
            "volumeMounts": [
              {
                "name": "default-token-th7qf",
                "readOnly": true,
                "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "IfNotPresent"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "serviceAccountName": "default",
        "serviceAccount": "default",
        "securityContext": {},
        "schedulerName": "default-scheduler",
        "tolerations": [
          {
            "key": "node.kubernetes.io/not-ready",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          },
          {
            "key": "node.kubernetes.io/unreachable",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          }
        ],
        "priority": 0,
        "enableServiceLinks": true
      },
      "status": {
        "phase": "Pending",
        "qosClass": "BestEffort"
      }
    },
    "oldObject": null,
    "dryRun": false,
    "options": {
      "kind": "CreateOptions",
      "apiVersion": "meta.k8s.io/v1"
    }
  }
}
' ${HOOK_URL}/validate-v1-pod

Development

How to build a new docker artifacts

make dist

Other Admission Controller

Other Resources

Disclaimer

This project is based on the Weave framework that I worked on. It also contains code that was written by my colleagues for the bnscli and copied into this project.

Special thanks therefore goes to:

  • @ethanfrey
  • @husio
  • @ruseinov

License

TBD

grafain's People

Contributors

alpe avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

grafain's Issues

Add a grafainCli

Is your feature request related to a problem? Please describe.
A command line client binary that can be used to:

  • generate keys
  • sign messages
  • list/ add/ delete artifacts

Describe the solution you'd like
Binary artifacts shipped via github release page

Describe alternatives you've considered

Additional context
Add any other context or screenshots about the feature request here.

Acceptance criteria
TBD

Scenario test to simulates an admission request from k8s

Is your feature request related to a problem? Please describe.
A full end-to-end test scenario.

Describe the solution you'd like
A scenario test in Go acts as admission controller and client to the grafain webhook.
A grafain stack is handling it including Tendermint setup.

Describe alternatives you've considered
Docker compose

Additional context
May reuse any grafainCli code if exists already

Acceptance criteria
Hits the full stack.

Set initial artifacts from genesis

Is your feature request related to a problem? Please describe.
I want to set a list of artifacts already on chain startup so that I do not have to set them manually via TX.

Describe the solution you'd like
Use weave.Initializers.

Describe alternatives you've considered

Additional context

Acceptance criteria

WebUI: simple dashboard to list new artifacts added

Is your feature request related to a problem? Please describe.
As a visitor interested in Grafain, I want a simple way to see the artifacts configurations stored so that I can use them in my test cluster

Describe the solution you'd like
Simple dashboard which lists the last 30 artifacts.

Describe alternatives you've considered
This functionality is available via ./grafaincli query -path=/artifacts

Additional context

Acceptance criteria

Documentation

Is your feature request related to a problem? Please describe.
Anybody new to the project should get a quick overview and understand what it does and how to use it.
Describe the solution you'd like

  • Motiviation: what does grafain solve + what is not included
  • Components (in this POC and including weave features enabled )
  • How to use it - Tutorial that runs it in minikube
  • Dev setup without k8s for local tests
  • Terraform example?

Describe alternatives you've considered

Additional context

Acceptance criteria
TBD

Add Tendermint consensus engine to k8s manifests

Is your feature request related to a problem? Please describe.
While Grafain implements the ABCI interface it requires Tendermint to do the consensus. The Kubernetes example manifests were not updated when Weave was integrated to this project.

Describe the solution you'd like

  • Create example genesis
  • Add Tendermint container to grafain setup
  • Configure Tendermint as Validator (genesis)
  • Update minikube example

Describe alternatives you've considered

Additional context
See [iov-devnets][https://github.com/iov-one/devnet-operations] where we did this already.

Acceptance criteria
I can follow the doc and start a validator on my local box within minikube.

Run single binary only

Is your feature request related to a problem? Please describe.
Start grafain with an embedded Tendermint.

Describe the solution you'd like
Single command with a good default config for local testing.

Additional context
Would be good to also allow current config with standalone Tendermint and socket connection to Grafain.

Acceptance criteria

Disable webhook module for fullnode mode

Is your feature request related to a problem? Please describe.
When I want to run a fullnode for integration tests on a codec package for example, I do not want to deal with any admission hook related k8s config.

Describe the solution you'd like
Add a new --no-admission-hook flag to disable all the hook related setup and validation.

Describe alternatives you've considered
Splitting fullnode and webhook into 2 binaries. They could still be deployed into the same pod for data safety. But this conflicts with #12 .

Additional context
Codec tests run with a local grafain, tendermint instance for integration tests.

Acceptance criteria

CLI: Replace private key file by key manager

Is your feature request related to a problem? Please describe.
The cli store the key unencrypted on the disk.
grafaincli mnemonic | ./grafaincli keygen -key $(pwd)/my_grafain.key # create a new private key

Describe the solution you'd like
Instead of persisting the private key on disk the cli should receive it from a key manager on grafaincli sign

Describe alternatives you've considered

Additional context
Some existing solutions that may be useful:
https://github.com/99designs/keyring
https://github.com/zalando/go-keyring

Acceptance criteria

Gitian based build process

Is your feature request related to a problem? Please describe.
Replace current build steps with Gitian based environment.

Describe the solution you'd like
The build process should create and upload:

  • Grafain server artifact
  • GrafainCli artifact

Describe alternatives you've considered

Additional context
See https://github.com/iov-one/weave/tree/master/contrib

Acceptance criteria
Gitian built artifacts are uploaded to github release page.

Add k-rail policies

Is your feature request related to a problem? Please describe.
The original POC supports whitelisted artifacts only. With k-rail policies a richer set of additional rules can be easily added that cover real world constraints when operating k8s environments.

Describe the solution you'd like
Add all policies for

  • Pod
  • Pod execution
  • Ingress

Add role based authz system

Is your feature request related to a problem? Please describe.
In order to manage artifact ownership for a dynamic group of people, I want a simple way to add/remove individuals to a Group or Role. Membership in the associated group/ role should be enough at this stage to allow full access. Fine grained permissions are off scope at this stage and will be addressed in a new issue when required.

Describe the solution you'd like
Any solution should contain:

  • model (name, owner address, list of address, name, active flag?)
  • handler, initializer, decorator
  • grafainCli list all roles, users, bindings
  • grafainCli show details by id (and user name if possible)
  • grafainCli add / delete signature + name to role
  • update genesis to seed a role and have an artifact owned by this role

Describe alternatives you've considered
Equal weight MultiSig with threshold of 1. We can not store metadata like name/ email with this.

Additional context

Acceptance criteria

Showcase: sign and push signature to chilinet

Is your feature request related to a problem? Please describe.
As a showcase for CD, instrument the travisCI build to also sign and push the metadata to 🌢net.

Describe the solution you'd like
On build branches: TravisCI uses the grafainCLI to push the sha256 digest for the artifact to a running testnet. (🌢)

Describe alternatives you've considered

Additional context

Investigate concurrent access to the DB

Is your feature request related to a problem? Please describe.
Webhook and backend app may access the DB at the same time and must not cause locks.

Describe the solution you'd like
Concurrent Reads by Webhook and backend.

Describe alternatives you've considered

Additional context
May fail with Iterators

Acceptance criteria

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.