Terraform Infrastructure State Locking with Terraform Cloud or S3 Backend

Task Overview

This task focuses on setting up remote state management using either Terraform Cloud or an S3 backend with DynamoDB for state locking. It ensures that infrastructure state files are centrally managed, secure, and versioned to prevent conflicts during concurrent deployments.

Objectives

1. Store the Terraform state remotely in Terraform Cloud or AWS S3 backend.
2. Enable state locking to prevent conflicts in multi-user environments.
3. Implement state versioning to recover from accidental changes.
4. Secure Terraform secrets using AWS IAM roles and policies.
5. Automate Terraform deployments using a CI/CD pipeline.

Task Architecture

• Terraform Cloud Option
• Uses Terraform Cloud as the backend for state storage.
• Provides built-in state locking and collaboration features.
• S3 + DynamoDB Option
• Stores state files in an S3 bucket.
• Uses DynamoDB table for state locking to prevent race conditions.

Implementation Steps

Step 1: Setup Terraform Cloud or AWS S3 Backend

Option 1: Terraform Cloud Backend

1. Sign up for a Terraform Cloud account.
2. Create a new Terraform Cloud workspace.
3. Update the Terraform configuration to use Terraform Cloud as the backend.

terraform {
  cloud {
    organization = "my-terraform-org"
    workspaces {
      name = "my-infra-workspace"
    }
  }
}

4. Login to Terraform Cloud using the CLI:

terraform login

5. Initialize Terraform:

terraform init

Option 2: S3 Backend with DynamoDB

1. Create an S3 bucket for storing

Terraform state:

aws s3 mb s3://my-terraform-state-bucket --region us-east-1

2. Enable versioning on the S3 bucket:

aws s3api put-bucket-versioning --bucket my-terraform-state-bucket --versioning-configuration Status=Enabled

3. Create a DynamoDB table for state locking:

aws dynamodb create-table \
  --table-name terraform-lock \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

4. Update Terraform backend configuration:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-lock"
  }
}

5. Initialize Terraform to configure the backend:

terraform init

Step 2: Define Infrastructure Resources

Now that remote state management is set up, define Terraform configurations for AWS infrastructure:

Example: Deploy an EC2 Instance

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-12345678"
  instance_type = "t2.micro"

  tags = {
    Name = "Terraform-EC2"
  }
}

Step 3: Implement IAM Policies for Secure Access

• Create an IAM policy to grant Terraform read/write access to the S3 bucket and DynamoDB

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-terraform-state-bucket",
        "arn:aws:s3:::my-terraform-state-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:DeleteItem",
        "dynamodb:Scan"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/terraform-lock"
    }
  ]
}

• Attach this policy to an IAM role used by Terraform.

Step 4: Apply Terraform Configuration

1. Plan the infrastructure:

terraform plan

2. Apply the configuration:

terraform apply -auto-approve

3. Verify the state locking by running

terraform plan

If another user tries to apply changes at the same time, Terraform will show a lock error.

Step 5: Enable CI/CD for Terraform

1. Use GitHub Actions, Jenkins, or GitLab CI/CD for Terraform automation.
2. Example GitHub Actions pipeline:

name: Terraform CI/CD

on:
  push:
    branches:
      - main

jobs:
  terraform:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        run: terraform plan

      - name: Terraform Apply
        if: github.ref == 'refs/heads/main'
        run: terraform apply -auto-approve

Project Validation

Testing State Locking

• Run terraform apply from two terminals and observe the lock error.
• Check DynamoDB table entries for lock records:

aws dynamodb scan --table-name terraform-lock

Testing Versioning

• Modify the Terraform configuration and re-run terraform apply.
• Restore an older state version from S3 if needed.

Best Practices

• Use Terraform Cloud for enterprise collaboration.
• Enable S3 versioning for rollback support.
• Use DynamoDB locking to prevent conflicts.
• Store Terraform secrets in AWS Secrets Manager or Vault.
• Automate Terraform deployment via CI/CD pipelines.

Final Deliverables

• Terraform scripts for infrastructure provisioning.
• Remote state storage in Terraform Cloud or S3.
• State locking & versioning implemented.
• CI/CD pipeline to automate Terraform deployments.

Next Steps

• Extend the project to include RDS, ALB, and VPC networking.
• Integrate security scanning using tfsec.
• Deploy monitoring with AWS CloudWatch.

Conclusion

By implementing Terraform Cloud or S3 backend with DynamoDB, we ensure that the Terraform state is secure, locked, and versioned, preventing conflicts in multi-user environments. This setup enhances collaboration and stability in AWS infrastructure automation.