Giter VIP home page Giter VIP logo

aws-ssm's Introduction

cmattoon/aws-ssm

Build Status GitHub tag Docker Pulls codecov Go Report Card Maintainability

Updates Kubernetes Secrets with values from AWS Parameter Store

Build Options

  • Helm Chart (recommended): make {lint|install|purge}
  • Go: make test && make build
  • Docker: make container

Helm Chart

Install Helm Chart

First, export required variables, then run make install.

export AWS_REGION=<region>

AWS Credentials

Uses the default credential provider chain

Values

The following chart values may be set. Only the required variables (AWS credentials) need provided by the user. Most of the time, the other defaults should work as-is.

Req'd Value Default Example Description
YES aws.region "" us-west-2 The AWS region in which the Pod is deployed
NO aws.access_key "" REQUIRED when no other auth method available (e.g., IAM role)
NO aws.secret_key "" REQUIRED when no other auth method available (e.g., IAM role)
NO kubeconfig64 "" The output of $(cat $KUBE_CONFIG | base64). Stored as a Secret
NO metrics_port 9999 Serve metrics/healthchecks on this port
NO image.name cmattoon/aws-ssm / The Docker image to use for the Pod container
NO image.tag latest The Docker tag for the image
NO resources {} Kubernetes Resource Requests/Limits
NO rbac.enabled true Whether or not to add Kubernetes RBAC stuff
NO ssl.mount_host false Mounts {ssl.host_path} -> {ssl.mount_path} as hostVolume
NO ssl.host_path /etc/ssl/certs The SSL certs dir on the host
NO ssl.mount_path /etc/ssl/certs The SSL certs dir in the container (dev)

Configuration

The following app config values can be provided via environment variables or CLI flags. CLI flags take precdence over environment variables.

A KUBE_CONFIG and MASTER_URL are only necessary when running outside of the cluster (e.g., dev)

Environment Flag Default Description
AWS_REGION -region us-west-2 The AWS Region
METRICS_URL -metrics-url 0.0.0.0:9999 Address for healthchecks/metrics
KUBE_CONFIG -kube-config The path to the kube config file
MASTER_URL -master-url The Kubernetes master API URL
LOG_LEVEL -log-level info The Logrus log level

Basic Usage

  1. Create Parameter in AWS Parameter Store

my-db-password = foobar

  1. Create Kubernetes Secret with Annotations
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  annotations:
    aws-ssm/k8s-secret-name: my-secret
    aws-ssm/aws-param-name: my-db-password
    aws-ssm/aws-param-type: SecureString
data: {}
  1. Run Binary

  2. A key with the name $ParameterType should have been added to your Secret

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  annotations:
    aws-ssm/k8s-secret-name: my-secret
    aws-ssm/aws-param-name: my-db-password
    aws-ssm/aws-param-type: SecureString
data:
  SecureString: Zm9vYmFyCg==

Annotations

Annotation Description Default
aws-ssm/k8s-secret-name The name of the Kubernetes Secret to modify. <none>
aws-ssm/aws-param-name The name of the AWS SSM Parameter. May be a path. <none>
aws-ssm/aws-param-type Determines how values are parsed, if at all. String
aws-ssm/aws-param-key Required if aws-ssm/aws-param-type is SecureString alias/aws/ssm

AWS Parameter Types

Values for aws-ssm/aws-param-type are:

Value Behavior AWS Value K8S Value(s)
String No parsing is performed foo = bar foo: bar
SecureString Requires aws-param-key foo = bar foo: bar
StringList Splits CSV mapping foo=bar,bar=baz,baz=bat foo: bar
bar: baz
baz: bat
Directory Get multiple values /path/to/values <treats each subkey/value as a String>

Build

make           # Build binary
make container # Build Docker image
make push      # Push Docker image

CA Certificates

For ease of use, the ca-certificates package is installed on the final library/alpine image. If you're having SSL/TLS connection issues, export HOST_SSL_DIR=/etc/ssl/certs before running make install. This will mount the SSL cert directory on the EC2 instance.

aws-ssm's People

Contributors

cmattoon avatar cmosetick avatar cyanopus avatar edify42 avatar kimxogus avatar mbarrien avatar rms1000watt avatar salmacodes avatar salmacodes-zz avatar signaleleven avatar xhochn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-ssm's Issues

Option to sync to ConfigMaps as well as Secrets

Hi there,

Thank you for putting this tool together - it's incredibly helpful and works great!

I'm syncing multiple parameters from SSM, some are 'secret' and some are just plain configuration data. To keep things semantically correct, I'd ideally like to sync the plain config data to a ConfigMap instead of a secret.

I'm fairly new to Kubernetes but my understanding at the moment is that ConfigMaps and Secrets are almost the same (Secrets are just base64 encoded for now and may have encryption features added in future). I'm wondering therefore whether it might be possible for this tool to also sync ConfigMap resources, using pretty much the same annotations?

Get more than 10 parameters

It seems aws-ssm get only 10 parameters using directory when it syncs, and I can't find any configuration to make it get more than 10 parameters for a single secret with directory configuration.

I want to request one of

  1. Get many params at once using MaxItems
  2. Get all params using pagination.

and this may not be needed

  • Add an annotation to set parameter count to get.

Move chart to upstream

Any chance to move the chart to upstream, maybe to incubator for now?

We are using the app in our k8 clusters and we are hosting the chart ourselves, which makes it harder to get the updates. And if it will be moved to a popular chart museum, it will also become more popular.

Cannot list secrets at the cluster scope

I could not get your helm install process to work witch looked like this:

make install
helm upgrade --install aws-ssm \
		--namespace aws-ssm \
		--set image.tag=latest \
	 	--set aws.region=us-east-2 \
	 	--set aws.access_key=<therealkey \
	 	--set aws.secret_key=<therealsecretaccesskey> \
	 	 aws-ssm
Error: UPGRADE FAILED: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
make: *** [install] Error 1

So I created a deployment yaml file instead that places the aws-ssm pod in a namespace called aws-ssm. Here is the log output of that pod:

time="2018-09-04T22:48:02Z" level=info msg="Using config: &{us-east-2 %!s(int=30)   0.0.0.0:9999 aws}"
time="2018-09-04T22:48:02Z" level=info msg=Running...
time="2018-09-04T22:48:02Z" level=info msg="Connected to cluster at https://10.43.0.1:443"
time="2018-09-04T22:48:02Z" level=fatal msg="Error retrieving secrets: secrets is forbidden: User \"system:serviceaccount:aws-ssm:default\" cannot list secrets at the cluster scope"

Can you release 1.8?

Hi cmattoon!

I've made a few PRs that were merged since 1.7. I've been using my own dirty build for the past few months. I'm hoping you'd be able to build and push to docker hub so I can use the official one.

Thanks,
-Ryan!

ParamByPath Nesting

Given the following SSM Params, and aws-ssm/aws-param-name: /path/to/db

  • /path/to/db/passwd
  • /path/to/db/etc/foo
  • /path/to/db/etc/passwd

Since the current implementation uses basename, the keys passwd and foo will be set in the Secret, but which passwd seems to be undefined. Also, I feel like the keys should probably be passwd, etc_foo, and etc_passwd, and/or a limit placed on the recursion.

IAM Role for helm chart

Hello,

Does the helm chart deployment support the passing of an IAM role?.. I don't see the option within the deployment.yml file.

Thanks..

Annotate Pods

As a user, I'd like to provide values directly to Pods via environment variables, given a StringList in the format key1=val1,key2=val2,key3=val3

make container fail

Make container fails on step 5 with this error:

Step 5/27 : RUN go install -v ./...
 ---> Running in 4dee84188e3d
go: downloading k8s.io/apimachinery v0.17.4
go: downloading github.com/aws/aws-sdk-go v1.29.23
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading k8s.io/client-go v0.15.11
go: downloading k8s.io/api v0.17.4
go: downloading golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527
go: downloading k8s.io/klog v1.0.0
go: downloading golang.org/x/net v0.0.0-20200301022130-244492dfa37a
go: downloading github.com/gogo/protobuf v1.3.1
go: downloading github.com/google/gofuzz v1.1.0
go: downloading gopkg.in/inf.v0 v0.9.1
go: downloading golang.org/x/text v0.3.2
go: downloading github.com/jmespath/go-jmespath v0.3.0
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:25:2: golang.org/x/[email protected]: unexpected EOF
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:26:2: golang.org/x/[email protected]: unexpected EOF
/go/pkg/mod/golang.org/x/[email protected]/idna/idna10.0.0.go:27:2: golang.org/x/[email protected]: unexpected EOF
pkg/secret/secret.go:27:2: k8s.io/[email protected]: unexpected EOF
pkg/secret/secret.go:28:2: k8s.io/[email protected]: unexpected EOF
pkg/controller/client.go:21:2: k8s.io/[email protected]: unexpected EOF
The command '/bin/sh -c go install -v ./...' returned a non-zero code: 1
Makefile:65: recipe for target 'container' failed
make: *** [container] Error 1

Chart fails installation with Missing required field "apiGroup" and "kind" in io.k8s.api.rbac.v1.RoleRef

Looks like installing the chart fails with the following error:

"Missing required field "apiGroup" and "kind" in io.k8s.api.rbac.v1.RoleRef"

This suggests a missing apiGroup: rbac.authorization.k8s.io on the RoleRef defined in aws-ssm/templates/rbac.yaml:
Chart fails with Missing required field "apiGroup" and "kind" in io.k8s.api.rbac.v1.RoleRefhttps://github.com/cmattoon/aws-ssm/blob/c5570006473e359dfe20b84f601e278550729d96/aws-ssm/templates/rbac.yaml#L43

Entire Pod fails when AWS IAM is misconfigured

From conversation in PR#12
In pkg/provider/aws.go, GetParameterDataByPath and GetParameterValue fail with log.Fatalf when the AWS client returns an error. If IAM credentials are not configured, the Pod will enter a crashloop until the pod is authorized. This is probably correct - or at least consistent - behavior when no IAM role/policy exists at all.

The bug exists when different permissions exist on different SSM keys, resulting in some values being resolved and some providing errors.

However, it prevented other secrets (for which no permission problem existed) to be updated, and since there is one single aws-ssm deployment on the cluster, serving potentially more than one application, the side effect of one broken secret in one namespace affecting operations on an unrelated application can be a bit problematic.

Build Failing - aws-iam-authenticator

Build failing due to something with aws-iam-authenticator:

# github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/root.go:68:18: cannot use config.DefaultFeatureGates (type map["sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) as type map["github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec in argument to featureGates.Add
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/root.go:96:15: cannot use featureGates (type *"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".featureGate) as type "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".MutableFeatureGate in field value:
	*"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".featureGate does not implement "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".MutableFeatureGate (wrong type for Add method)
		have Add(map["github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) error
		want Add(map["sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".Feature]"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/component-base/featuregate".FeatureSpec) error
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:82:43: cannot use k8sconfig (type *"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/rest".Config) as type *"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/rest".Config in argument to versioned.NewForConfig
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:89:26: cannot use kubeClient (type "github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface) as type "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface in argument to controller.New:
	"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface does not implement "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes".Interface (wrong type for AdmissionregistrationV1beta1 method)
		have AdmissionregistrationV1beta1() "github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes/typed/admissionregistration/v1beta1".AdmissionregistrationV1beta1Interface
		want AdmissionregistrationV1beta1() "sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/client-go/kubernetes/typed/admissionregistration/v1beta1".AdmissionregistrationV1beta1Interface
../../kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator/server.go:102:56: cannot use []"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object literal (type []"github.com/kubernetes-sigs/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object) as type []"sigs.k8s.io/aws-iam-authenticator/vendor/k8s.io/apimachinery/pkg/runtime".Object in argument to "sigs.k8s.io/aws-iam-authenticator/pkg/generated/clientset/versioned/fake".NewSimpleClientset
The command '/bin/sh -c go get -u -v github.com/kubernetes-sigs/aws-iam-authenticator/cmd/aws-iam-authenticator' returned a non-zero code: 2

no CA certs in image

Seems like we need volume mount CA certs or provide some in the docker image:

level=info msg="Couldn't get value for aws-ssm/test-secret: RequestError: send request failed\ncaused by: Post https://ssm.us-east-2.amazonaws.com/: x509: failed to load system roots and no roots provided"

Switch to go.mod and update the aws-sdk

Hey @cmattoon -

The current version of the code uses an outdated aws-sdk which doesn't support the AssumeRoleWithWebIdentity IAM call. The architecture also depends on the node running the container having access to the secrets.

I'm using your container on EKS with the OIDC identity stuff enabled which allows pods to assume roles in AWS, thus I can limit the permissions i need to set on a node.

I updated that with some other packages as well as the dependency system you were using before hand ๐Ÿ˜Š

I think i'm also using a newer version of golang but that's not as important.

I'll write up the PR for this shortly.

K8s secret key naming

I'm a bit confused regarding the mapping of values to keys in the created k8s secret. The README.md states that an AWS securestring value of "foo = bar" would set a k8s value of "foo:bar"

However I always get

data:
SecureString: bar

Where bar is base64 encoded as expected.

I've tried setting the parameter value as follows

bar # K8s secret get 'SecureString: bar'
foo=bar # K8s secret gets 'SecureString: foo=bar'

Again the values are base64 encoded as expected.

If this is expected behavior then I think it would be nice to provide an annotation that overrides the default data.SecureString name so that I can adjust the name to match what applications are expecting to find in their secret.

Multiple parameters in one secret

If I understand this correctly, currently it's only possible to map 1-1 parameters and secrets key (one SSM parameter per secret).
It would be nice if multiple parameters could be added as multiple key-value pairs in the same secret

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.