Giter VIP home page Giter VIP logo

10-mircoservices-cicd's Introduction

CI/CD Pipeline for 10-Tier Microservice Application

This repository contains the CI/CD pipeline for a 10-tier microservice application. The pipeline is designed to work with Amazon EKS and includes the following steps:

1. IAM User Setup

An IAM user was created with specific policies tailored for working with Amazon EKS.

2. VM Preparation

A Virtual Machine (EC2 instance) was set up on AWS. The following essential CLIs were installed:

  • awscli
  • kubectl
  • eksctl

Additionally, SonarQube was installed for static code analysis and Docker for containerizing the microservices.

3. AWS Connection

The VM was connected to AWS using AWSCLI, utilizing the IAM user security credentials.

4. EKS Cluster Creation

An EKS cluster was created using the EKSCTL CLI. The process started with the Master Node and then added Worker Nodes with Autoscaling enabled.

5. Service Account and Role Configuration

A Service Account was established within EKS and a ROLE with specific access permissions for CRUD operations in EKS. The role was properly bound to the service account.

6. Jenkins Setup

A Jenkins server was configured to streamline the deployment process. SonarQube, Docker, and Kubernetes were configured inside Jenkins.

7. Pipeline Creation

A pipeline was developed to perform the following tasks:

  • Perform static code analysis using SonarQube.
  • Containerize the microservices using Docker.
  • Push the images to DockerHub.
  • Deploy the 10-tier application to the EKS cluster.

Usage

Creating an IAM user with required permissions

iam-policies

Add above permissions to the IAM user so that it can interact with the EKS cluster

Configuring EC2 instance and installing required tools

Create a t2.large EC2 instance with 30GB of EBS volume. Use the below set of commands to setup the required tools

  • Install AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip
unzip awscliv2.zip
sudo ./aws/install
  • Install Kubectl for EKS
curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/1.28.3/2023-11-14/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin
  • Install EKS
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
  • Install JDK 17 and Jenkins
sudo apt install fontconfig openjdk-17-jre -y
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins -y
  • Install Docker and add jenkins user to docker group
sudo apt install docker.io -y
sudo usermod -aG docker jenkins
sudo systemctl restart docker
  • Install SonarQube using a Docker image
docker run -d -p 9000:9000 sonarqube:lts-community
installations-1 jenkins-install docker-sonarqube-install

Verifying the installations of the required tools

Connecting the EC2 instance to AWS

The EC2 instance was connected to the AWS using AWS CLI and then using the command aws configure to enter the access key ID and secret access key details

Creating EKS cluster

Create the EKS cluster using the below command:

eksctl create cluster --name=cluster-name \
		      --region=ap-south-1 \
		      --zones=ap-south-1a,ap-south-1b \
		      --without-nodegroup

eksctl utils associate-iam-oidc-provider \
    --region ap-south-1 \
    --cluster cluster-name \
    --approve

eksctl create nodegroup --cluster=cluster-name \
			--region=ap-south-1 \
			--name=node2 \
			--node-type=t3.medium \
			--nodes=2 \
			--nodes-min=2 \
			--nodes-max=3 \
			--node-volume-size=20 \
			--ssh-access \
			--ssh-public-key=pem-file-name \
			--managed \
			--asg-access \
			--external-dns-access \
			--full-ecr-access \
			--appmesh-access \
			--alb-ingress-access

It will create the master and the worker nodes seperately with auto scaling and load balancing enabled

Jenkins Setup

  • Login into the Jenkins
  • Install the plugins related to SonarQube, Docker and Kubernetes
jenkins-plugins
  • Go to Dashboard > Manage Jenkins > Tools and add SonarQube Scanner and Docker installations
tool-config-1 tool-config-2
  • Go to Dashboard > Manage Jenkins > System and setup the SonarQube server with the server URL and the authentication token generated from SonarQube
sonar-server-congif-jenkins

Modifying the security group in EKS cluster

Go to the addtional security group inside the EKS cluster and allow All Traffic under inbound rules eks-SG

Service Account and Role

In order to allow jenkins user to be able to communicate with the EKS cluster and perform deployments, create a service account and a role with specific access permissions and then bind that service account to the role. The service account, role and bind manifest files are located inside k8s folder. Apply those files using the kubectl apply -f <filename> command.

Creating a secret for the service account

Create a secret.yaml file for jenkins so that it can authenticate with the cluster. The file can be accessed from the k8s folder. Generate the secret by using the following command:

kubectl describe secret mysecretname -n webapps

Setting up the CI/CD pipeline

In order to communicate with the DockerHub repository, store the credentials inside jenkins. Similarily, store the secret token for the service account. The Jenkinsfile for the pipeline is present inside the repository. Once the pipeline is passed successfully use kubectl get pods and kubectl get svc to get the list of running pods and the load balancer IP. Hit the IP in the browser to view the application deployed.

pipeline pods services app

10-mircoservices-cicd's People

Contributors

devops-maestro17 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.