CI/CD Pipeline for Terraform Deployment

Task Goal

Automate Terraform infrastructure deployment using a robust CI/CD pipeline with Jenkins, GitHub Actions, or GitLab CI. Ensure all infrastructure changes are version-controlled, reviewed via pull requests (PRs), and deployed automatically.

Architecture Overview

1. Tools and Technologies

• Version Control: GitHub / GitLab
• CI/CD Orchestration: Jenkins, GitHub Actions, or GitLab CI
• Infrastructure as Code (IaC): Terraform
• Cloud Provider: AWS (Alternatively Azure/GCP)
• State Management: Terraform Cloud / S3 + DynamoDB
• Secrets Management: AWS Secrets Manager / HashiCorp Vault
• Monitoring & Logging: AWS CloudWatch, Prometheus, Grafana

Task Workflow

1. Infrastructure as Code (IaC) Setup

• Define Terraform configurations to provision AWS resources (e.g., EC2, VPC, RDS, ALB).
• Store Terraform state in AWS S3 (with DynamoDB locking) or Terraform Cloud.
• Use Terraform modules for reusability and standardization.

2. Version Control & PR Workflow

• Developers create a feature branch to modify Terraform code.
• Submit a pull request (PR) for review.
• GitHub/GitLab enforces a pre-merge validation check:
• Terraform formatting check (terraform fmt)
• Terraform validation (terraform validate)
• Terraform plan to preview changes (terraform plan).
• After approval, merge PR into main branch.

3. CI/CD Pipeline Automation

Pipeline Triggers

• Pre-Merge (PR checks): Run terraform fmt, terraform validate, and terraform plan.
• Post-Merge (Deployment): Apply infrastructure changes automatically.

Jenkins Pipeline (Alternative: GitHub Actions/GitLab CI)

• Trigger: Push to main branch
• Stages:
1. Checkout Code (from GitHub/GitLab)
2. Lint & Validate (terraform fmt, terraform validate)
3. Plan Deployment (terraform plan)
4. Manual Approval (if needed for critical changes)
5. Apply Changes (terraform apply)

Implementation Steps

Step 1: Set Up Terraform Code Repository

1. Initialize a GitHub/GitLab repo for Terraform configurations.
2. Define Terraform modules:
• VPC module
• EC2 instance module
• RDS module
3. Store Terraform backend configuration in S3/Terraform Cloud.

Step 2: Configure CI/CD Pipeline

For Jenkins

1. Install Jenkins plugins: Pipeline, GitHub Branch Source, Terraform.
2. Create a >Jenkinsfile:

   pipeline {
       agent any
       environment {
           AWS_ACCESS_KEY_ID = credentials('aws-access-key')
           AWS_SECRET_ACCESS_KEY = credentials('aws-secret-key')
       }
       stages {
           stage('Checkout Code') {
               steps {
                   git 'https://github.com/org/repo.git'
               }
           }
           stage('Terraform Init') {
               steps {
                   sh 'terraform init'
               }
           }
           stage('Terraform Validate') {
               steps {
                   sh 'terraform validate'
               }
           }
           stage('Terraform Plan') {
               steps {
                   sh 'terraform plan'
               }
           }
           stage('Terraform Apply') {
               when {
                   branch 'main'
               }
               steps {
                   sh 'terraform apply -auto-approve'
               }
           }
       }
   }
   

name: Terraform Deployment
on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v2

      - name: Terraform Init
        run: terraform init

      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        run: terraform plan

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve

stages:
  - validate
  - plan
  - apply

variables:
  TF_IN_AUTOMATION: "true"

validate:
  stage: validate
  script:
    - terraform init
    - terraform validate
  only:
    - merge_requests

plan:
  stage: plan
  script:
    - terraform plan
  only:
    - merge_requests

apply:
  stage: apply
  script:
    - terraform apply -auto-approve
  only:
    - main