End-to-End Multi-Branch Pipeline Project Using Jenkins, GitHub, Docker, AWS, and Terraform

Project Overview

The goal of this project is to implement a Multi-Branch Pipeline in Jenkins that automatically triggers builds for different branches (main, dev, and feature/*). Each branch will have a distinct build, test, and deployment process.

Technology Stack

Project Architecture

  1. Developers push code to GitHub on different branches (main, dev, feature/*).
  2. Jenkins Multi-Branch Pipeline detects branch changes and triggers the corresponding job.
  3. Pipeline Stages:
  4. Terraform provisions AWS infrastructure dynamically if required.
  5. Monitoring setup using Prometheus and Grafana.

Implementation Steps

1. Set Up GitHub Repository

Create a repository with three branches: main, dev, and feature/test-feature.

Add a sample Java-based application along with a Jenkinsfile in each branch.

2. Install and Configure Jenkins

Install Jenkins on an EC2 instance (or use a containerized version).

Install necessary plugins

Configure Jenkins global credentials for:

3. Create a Multi-Branch Pipeline in Jenkins

4. Write Jenkinsfile for Multi-Branch Pipeline

Each branch executes different steps based on the environment.

Jenkinsfile for main branch (Production Deployment)

pipeline {
    agent any

    environment {
        DOCKER_IMAGE = "yourdockerhub/image:${BUILD_NUMBER}"
    }

    stages {
        stage('Checkout Code') {
            steps {
                git branch: 'main', url: 'https://github.com/yourrepo.git'
            }
        }

        stage('Static Code Analysis') {
            steps {
                script {
                    withSonarQubeEnv('SonarQube') {
                        sh 'mvn clean verify sonar:sonar'
                    }
                }
            }
        }

        stage('Build & Package') {
            steps {
                sh 'mvn clean package'
            }
        }

        stage('Push Artifact to Nexus') {
            steps {
                sh 'mvn deploy'
            }
        }

        stage('Docker Build & Push') {
            steps {
                sh "docker build -t ${DOCKER_IMAGE} ."
                sh "docker login -u youruser -p yourpassword"
                sh "docker push ${DOCKER_IMAGE}"
            }
        }

        stage('Deploy to Kubernetes (EKS)') {
            steps {
                sh "kubectl apply -f k8s/deployment.yaml"
            }
        }
    }

    post {
        success {
            slackSend channel: '#deployments', message: "✅ Build Successful: ${env.JOB_NAME} - ${env.BUILD_NUMBER}"
        }
        failure {
            slackSend channel: '#deployments', message: "❌ Build Failed: ${env.JOB_NAME} - ${env.BUILD_NUMBER}"
        }
    }
}

Jenkinsfile for dev branch (Staging Deployment)


      pipeline {
    agent any

    stages {
        stage('Checkout Code') {
            steps {
                git branch: 'dev', url: 'https://github.com/yourrepo.git'
            }
        }

        stage('Build & Unit Tests') {
            steps {
                sh 'mvn clean test'
            }
        }

        stage('Push Docker Image') {
            steps {
                sh "docker build -t yourdockerhub/dev-image:${BUILD_NUMBER} ."
                sh "docker push yourdockerhub/dev-image:${BUILD_NUMBER}"
            }
        }

        stage('Deploy to Staging') {
            steps {
                sh "kubectl apply -f k8s/dev-deployment.yaml"
            }
        }
    }
}

Jenkinsfile for feature/* branch (Testing)

pipeline {
    agent any

    stages {
        stage('Checkout Code') {
            steps {
                git branch: 'feature/test-feature', url: 'https://github.com/yourrepo.git'
            }
        }

        stage('Run Tests') {
            steps {
                sh 'mvn test'
            }
        }
    }
}

5. Create Terraform Script for AWS Infrastructure