Dockerized CI/CD Pipeline for Microservices

Objective

Automate the build, test, and deployment process of a microservices-based application using a Dockerized CI/CD pipeline. The pipeline will integrate Jenkins, GitHub Actions, or GitLab CI to build and test Docker images, then push them to Docker Hub or AWS ECR, and finally deploy them to AWS ECS or Kubernetes.

Technology Stack

Version Control: GitHub / GitLab
CI/CD Tools: Jenkins, GitHub Actions, or GitLab CI
Containerization: Docker, Docker Compose
Container Registry: Docker Hub or AWS ECR
Deployment: AWS ECS (Fargate) / Kubernetes (EKS)
Infrastructure as Code: Terraform (Optional)
Monitoring: Prometheus & Grafana
Logging: ELK Stack (Elasticsearch, Logstash, Kibana)

Task Architecture

Developers push code to GitHub/GitLab repository.
CI Pipeline triggers automatically:
- Clones the repo.
- Builds a Docker image.
- Runs tests inside a container.
If tests pass:
- Image is tagged and pushed to Docker Hub / AWS ECR.
CD Pipeline triggers automatically:
- Deploys the new image to AWS ECS / EKS.
Monitoring & Logging track application health.

Implementation Steps

1. Set Up the Git Repository

Initialize a GitHub / GitLab repository.
Create a Dockerized microservice (Node.js, Python, or Java).
Define a
```
Dockerfile
```
and
```
docker-compose.yml.
```

2. Write the Dockerfile

Example Dockerfile for a Node.js app:

FROM node:18
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]

3. Write the CI/CD Pipeline Configuration

Option 1: Jenkins CI/CD Pipeline

Install Jenkins, Docker, and AWS CLI.
Configure Jenkinsfile to automate:
- Building the Docker image.
- Running tests.
- Pushing the image to Docker Hub or AWS ECR.
- Deploying to ECS / EKS.

pipeline {
    agent any
    environment {
        IMAGE_NAME = "myapp"
        DOCKERHUB_USER = credentials('dockerhub-credentials')
        AWS_ACCOUNT_ID = '123456789012'
        AWS_REGION = 'us-east-1'
        ECR_REPO = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${IMAGE_NAME}"
    }
    stages {
        stage('Checkout Code') {
            steps {
                git 'https://github.com/user/repo.git'
            }
        }
        stage('Build Docker Image') {
            steps {
                sh 'docker build -t $IMAGE_NAME .'
            }
        }
        stage('Run Tests') {
            steps {
                sh 'docker run --rm $IMAGE_NAME npm test'
            }
        }
        stage('Push to Docker Hub') {
            steps {
                sh 'echo $DOCKERHUB_USER | docker login --username myuser --password-stdin'
                sh 'docker tag $IMAGE_NAME myuser/$IMAGE_NAME:latest'
                sh 'docker push myuser/$IMAGE_NAME:latest'
            }
        }
        stage('Deploy to AWS ECS') {
            steps {
                sh 'aws ecs update-service --cluster my-cluster --service my-service --force-new-deployment'
            }
        }
    }
}

Option 2: GitHub Actions Workflow

Create

.github/workflows/ci-cd.yml

name: Dockerized CI/CD

on:
  push:
    branches:
      - main

jobs:
  build-test-push:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3

      - name: Login to Docker Hub
        run: echo "${{ secrets.DOCKERHUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKERHUB_USERNAME }}" --password-stdin

      - name: Build and Test
        run: |
          docker build -t myapp .
          docker run --rm myapp npm test

      - name: Push to Docker Hub
        run: |
          docker tag myapp myuser/myapp:latest
          docker push myuser/myapp:latest

      - name: Deploy to AWS ECS
        run: aws ecs update-service --cluster my-cluster --service my-service --force-new-deployment

Option 3: GitLab CI/CD

Create

.gitlab-ci.yml

stages:
  - build
  - test
  - push
  - deploy

build:
  script:
    - docker build -t myapp .

test:
  script:
    - docker run --rm myapp npm test

push:
  script:
    - echo "$DOCKERHUB_PASSWORD" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
    - docker tag myapp myuser/myapp:latest
    - docker push myuser/myapp:latest

deploy:
  script:
    - aws ecs update-service --cluster my-cluster --service my-service --force-new-deployment

4. Push Image to Docker Hub or AWS ECR

Push to Docker Hub


docker login -u myuser
docker tag myapp myuser/myapp:latest
docker push myuser/myapp:latest

      Push to AWS ECR 
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
docker tag myapp 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp:latest
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp:latest

5. Deploy to AWS ECS

Create ECS Service Definition

(ecs-task.json)

{
    "family": "myapp-task",
    "containerDefinitions": [
        {
            "name": "myapp",
            "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/myapp:latest",
            "memory": 512,
            "cpu": 256,
            "essential": true,
            "portMappings": [
                {
                    "containerPort": 3000,
                    "hostPort": 3000
                }
            ]
        }
    ]
}

Deploy to ECS:

aws ecs update-service --cluster my-cluster --service my-service --force-new-deployment

Optional: Monitoring & Logging

Prometheus & Grafana for real-time monitoring.
ELK Stack (Elasticsearch, Logstash, Kibana) for logging and analytics.

Conclusion

With this setup, you have a fully automated Dockerized CI/CD pipeline that builds the Docker image, tests it inside a container, pushes it to Docker Hub / AWS ECR, deploys it to AWS ECS / EKS, and monitors the application health using Prometheus/Grafana.

Builds the Docker image.
Tests it inside a container.
Pushes the image to Docker Hub / AWS ECR.
Deploys it to AWS ECS / EKS.
Monitors the application health using Prometheus/Grafana.