Giter VIP home page Giter VIP logo

my-aws-eks's Introduction

My AWS EKS

Description

I couldn't find any example Terraform projects to make an EKS cluster that I was happy with, so I cobbled this one together. This project spins up a decent EKS cluster for demos, development, or testing. In theory, you could scale it up to production too if your apps are stateful and can tolerate using spot instances - but it's really meant to be for short/medium term environments that you spin up or down at need.

It currently features:

  • Custom VPC Setup.
  • Kubernetes 1.21.
  • Secrets Encryption via a rotating customer-managed KMS key.
  • Cloudwatch Encryption via a rotating customer-managed KMS key.
  • Control Plane logging to Cloudwatch.
  • Common Tagging across all created resources for easy billing resolution.
  • Calico networking instead of "aws-node"
  • EC2 worker nodes with encrypted root volumes.
  • 2 Helm Charts at a minimum:
  • Configurable ausoscaling EC2 Pools. By default it runs:
    • 1 t3.small instance for safety. The autoscaler pod runs here.
    • 1 to 5 t3.medium spot instances. Ideally, most of the workload should run on these. The spot price is set to the on-demand price.
  • Configurable mapping of accounts, IAM roles, and IAM users to the aws-auth conifgmap.
  • (Occasionally) bleeding edge compatibility with Terraform 1.0.7
  • Generation of the Kubeconfig needed for kubectl, helm, etc.

Key Aims

  • Cost to remain as low as possible.
  • Ideally, I want this project to always run with the latest Terraform - though this requires compatibility with the public AWS terraform modules.
  • Helm is the tool of choice for installing into the cluster - Convince me otherwise.

Installation.

  • This was last run with Terraform 1.0.7
  • Just edit what you need to in provider.tf to allow you to connect, and put what you want into local.tf
  • Run a terraform apply.

This is what ends up running after your first install:

╰─❯ kubectl get all -A
NAMESPACE     NAME                                                            READY   STATUS    RESTARTS   AGE
kube-system   pod/aws-load-balancer-controller-54cf85b446-c8244               1/1     Running   1          43m
kube-system   pod/aws-load-balancer-controller-54cf85b446-vbpfz               1/1     Running   1          43m
kube-system   pod/aws-node-termination-handler-5jcfs                          1/1     Running   0          43m
kube-system   pod/aws-node-termination-handler-nqv7q                          1/1     Running   0          43m
kube-system   pod/calico-kube-controllers-784b4f4c9-qpfs8                     1/1     Running   0          43m
kube-system   pod/calico-node-r7lkj                                           1/1     Running   0          43m
kube-system   pod/calico-node-rmw6m                                           1/1     Running   0          43m
kube-system   pod/cluster-autoscaler-aws-cluster-autoscaler-d49c449d5-vlzt5   1/1     Running   0          43m
kube-system   pod/coredns-65ccb76b7c-g7cbz                                    1/1     Running   0          46m
kube-system   pod/coredns-65ccb76b7c-nzcd4                                    1/1     Running   0          46m
kube-system   pod/kube-proxy-qs2r5                                            1/1     Running   0          43m
kube-system   pod/kube-proxy-svnh5                                            1/1     Running   0          43m

NAMESPACE     NAME                                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
default       service/kubernetes                                  ClusterIP   10.100.0.1      <none>        443/TCP         46m
kube-system   service/aws-load-balancer-webhook-service           ClusterIP   10.100.25.81    <none>        443/TCP         43m
kube-system   service/cluster-autoscaler-aws-cluster-autoscaler   ClusterIP   10.100.115.16   <none>        8085/TCP        43m
kube-system   service/kube-dns                                    ClusterIP   10.100.0.10     <none>        53/UDP,53/TCP   46m

NAMESPACE     NAME                                          DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/aws-node-termination-handler   2         2         2       2            2           kubernetes.io/os=linux   43m
kube-system   daemonset.apps/calico-node                    2         2         2       2            2           kubernetes.io/os=linux   43m
kube-system   daemonset.apps/kube-proxy                     2         2         2       2            2           <none>                   46m

NAMESPACE     NAME                                                        READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/aws-load-balancer-controller                2/2     2            2           43m
kube-system   deployment.apps/calico-kube-controllers                     1/1     1            1           43m
kube-system   deployment.apps/cluster-autoscaler-aws-cluster-autoscaler   1/1     1            1           43m
kube-system   deployment.apps/coredns                                     2/2     2            2           46m

NAMESPACE     NAME                                                                  DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/aws-load-balancer-controller-54cf85b446               2         2         2       43m
kube-system   replicaset.apps/calico-kube-controllers-784b4f4c9                     1         1         1       43m
kube-system   replicaset.apps/cluster-autoscaler-aws-cluster-autoscaler-d49c449d5   1         1         1       43m
kube-system   replicaset.apps/coredns-65ccb76b7c

Todo

  • Setup pre-commit tooling, including a Checkov security scan.
  • I wanted to use Launch Templates instead of Launch Configs - but there seems to be a bug in the EKS terraform modules where it's ignoring the Spot configuration.
  • Testing Framework?
  • Build a list of must-have Helm charts you'd tend to put into an EKS/K8S cluster. I'm thinking it would start with:
    • Vault
    • Consul ?
    • Prometheus (via its Operator)
    • Cert Manager
    • Keycloak
  • How can this integrate with Route53? Should it?

Requirements

Name Version
terraform >= 0.13.1
aws >= 3.42.0
cloudinit ~> 2.2.0
kubernetes ~> 2.2.0
local >= 2.1.0
null ~> 3.1.0
random >= 3.1.0
template ~> 2.2.0

Providers

Name Version
aws >= 3.42.0
helm n/a
null ~> 3.1.0

Modules

Name Source Version
alb_controller git::github.com/GSA/terraform-kubernetes-aws-load-balancer-controller?ref=v4.1.0
eks terraform-aws-modules/eks/aws
iam_assumable_role_admin terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc
vpc terraform-aws-modules/vpc/aws

Resources

Name Type
aws_iam_policy.cluster_autoscaler resource
aws_kms_alias.eks resource
aws_kms_alias.ekslogs resource
aws_kms_key.eks resource
aws_kms_key.ekslogs resource
helm_release.autoscaler resource
helm_release.aws_node_termination_handler resource
null_resource.install_calico_plugin resource
null_resource.kube_config resource
aws_availability_zones.available data source
aws_caller_identity.current data source
aws_eks_cluster.cluster data source
aws_eks_cluster_auth.cluster data source
aws_iam_policy_document.cluster_autoscaler data source
aws_iam_policy_document.logging data source

Inputs

No inputs.

Outputs

Name Description
cloudwatch_log_group_name Cloudwatch Log Group Name for this Cluster
cluster_endpoint Endpoint for EKS control plane.
cluster_security_group_id Security group ids attached to the cluster control plane.
config_map_aws_auth A kubernetes configuration to authenticate to this EKS cluster.
kubectl_config kubectl config as generated by the module.
region AWS region.

my-aws-eks's People

Contributors

bocan avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

luisgradossalinas rbrto kylechou0755 kchou94 bridgecrew-perf4 babutamil

my-aws-eks's Issues

The Autoscaler pod isn't tied to the on-demand node yet.

This issue I think you can use node selector to run Autoscaler pod on on-demand node

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    env: test
spec:
  containers:
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
  nodeSelector:
    node.kubernetes.io/lifecycle=normal

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.