Automated Multi-Region File and Directory Management Using Ansible on AWS

Task Overview

This task automates multi-region deployment and backup automation for file and directory management using Ansible and Terraform on AWS. The automation ensures:

• Deployment of EC2 instances across multiple AWS regions
• Automated creation of required directories
• Copying and managing files
• Setting proper file and directory permissions
• Deploying a sample configuration file (nginx.conf)
• Ensuring configurations are applied and services restarted
• Backup automation using Amazon S3 and AWS Backup

This project helps ensure high availability, disaster recovery, and streamlined configuration management.

1. Tools & Technologies

• Infrastructure Provisioning: Terraform
• Configuration Management: Ansible
• Cloud Platform: AWS (EC2, IAM, VPC, Security Groups, CloudWatch, S3, AWS Backup)
• Version Control: Git/GitHub
• CI/CD: Jenkins, GitHub Actions
• Monitoring: AWS CloudWatch, Prometheus, Grafana

2. Task Architecture

1. Terraform provisions AWS infrastructure across multiple regions:
   • EC2 instances in primary and secondary AWS regions
   • Security Groups, IAM roles, and networking
   • S3 bucket for backup storage
   • AWS Backup policies for scheduled file backups
2. Ansible manages file and directory configurations:
   • Creates required directories
   • Copies configuration files (nginx.conf)
   • Sets correct file and directory permissions
   • Restarts services after configuration updates
3. Backup automation using AWS Backup & S3:
   • Periodic backups of important files and directories
   • Retention policies and automated snapshot creation
4. CI/CD pipeline automates deployments:
   • GitHub Actions/Jenkins triggers Ansible playbook
   • Ensures continuous configuration enforcement

3. Multi-Region Infrastructure Setup using Terraform

Step 1: Create Multi-Region Terraform Configuration (main.tf)

provider "aws" {
  alias  = "primary"
  region = "us-east-1"
}

provider "aws" {
  alias  = "secondary"
  region = "us-west-2"
}

resource "aws_vpc" "main_primary" {
  provider   = aws.primary
  cidr_block = "10.0.0.0/16"
}

resource "aws_vpc" "main_secondary" {
  provider   = aws.secondary
  cidr_block = "10.1.0.0/16"
}

resource "aws_instance" "web_primary" {
  provider      = aws.primary
  ami           = "ami-12345678"
  instance_type = "t2.micro"
  key_name      = "aws-key"

  tags = {
    Name = "WebServerPrimary"
  }
}

resource "aws_instance" "web_secondary" {
  provider      = aws.secondary
  ami           = "ami-87654321"
  instance_type = "t2.micro"
  key_name      = "aws-key"

  tags = {
    Name = "WebServerSecondary"
  }
}

resource "aws_s3_bucket" "backup_bucket" {
  provider = aws.primary
  bucket   = "nginx-config-backup"

  lifecycle {
    prevent_destroy = true
  }
}

output "primary_instance_ip" {
  value = aws_instance.web_primary.public_ip
}

output "secondary_instance_ip" {
  value = aws_instance.web_secondary.public_ip
}

Run Terraform:

terraform init
terraform apply -auto-approve

4. Configuring Ansible for Multi-Region EC2 Instances

Step 2: Define Ansible Inventory (hosts)

[primary]
web_primary ansible_host=<PRIMARY_EC2_IP> ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/aws-key.pem

[secondary]
web_secondary ansible_host=<SECONDARY_EC2_IP> ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/aws-key.pem

Step 3: Create Ansible Playbook (file_management.yml)

---
- name: Manage Directories and Files on Multi-Region AWS EC2
  hosts: all
  become: yes
  tasks:

    - name: Create necessary directories
      file:
        path: "/etc/nginx/custom-config"
        state: directory
        mode: '0755'

    - name: Copy the nginx configuration file
      copy:
        src: files/nginx.conf
        dest: /etc/nginx/nginx.conf
        owner: root
        group: root
        mode: '0644'
      notify: Restart nginx

    - name: Backup nginx.conf to S3
      aws_s3:
        bucket: nginx-config-backup
        object: "configs/nginx-{{ inventory_hostname }}.conf"
        src: /etc/nginx/nginx.conf
        mode: put

    - name: Ensure nginx service is running
      systemd:
        name: nginx
        state: started
        enabled: yes

  handlers:
    - name: Restart nginx
      systemd:
        name: nginx
        state: restarted

Run the playbook:

ansible-playbook -i hosts file_management.yml

5. Automating Backups with AWS Backup

Step 4: Create AWS Backup Plan

resource "aws_backup_vault" "nginx_backup" {
  name = "nginx-backup"
}

resource "aws_backup_plan" "daily_backup" {
  name = "daily-backup"

  rule {
    rule_name         = "daily-backup-rule"
    target_vault_name = aws_backup_vault.nginx_backup.name
    schedule          = "cron(0 12 * * ? *)" # Runs daily at noon UTC

    lifecycle {
      delete_after = 30 # Keep backups for 30 days
    }
  }
}

resource "aws_backup_selection" "backup_selection" {
  name         = "nginx-backup-selection"
  plan_id      = aws_backup_plan.daily_backup.id
  iam_role_arn = aws_iam_role.backup_role.arn

  resources = [
    aws_instance.web_primary.arn,
    aws_instance.web_secondary.arn
  ]
}

Run Terraform:

terraform apply -auto-approve

6. CI/CD Pipeline Integration

Step 5: Jenkins Pipeline Configuration

pipeline {
    agent any
    stages {
        stage('Clone Repo') {
            steps {
                git 'https://github.com/your-repo.git'
            }
        }
        stage('Deploy Infrastructure') {
            steps {
                sh 'terraform init && terraform apply -auto-approve'
            }
        }
        stage('Configure EC2 with Ansible') {
            steps {
                sh 'ansible-playbook -i hosts file_management.yml'
            }
        }
    }
}

7. Project Deliverables

• ✅ Terraform script for AWS multi-region infrastructure
• ✅ Ansible playbook for file and directory management
• ✅ Jenkins pipeline for automation
• ✅ AWS S3 backup setup for configuration files
• ✅ AWS Backup automation for EC2 snapshots
• ✅ CI/CD integration

8. Future Enhancements

• 🚀 Implement Multi-Region Load Balancing with AWS ALB
• 🚀 Integrate AWS Lambda for backup validation
• 🚀 Use AWS Systems Manager for remote management
• 🚀 Monitor backups using CloudWatch and set alerts

Conclusion

This project ensures high availability and disaster recovery by deploying multi-region infrastructure, automating file management with Ansible, and integrating backup automation using AWS S3 and AWS Backup. The CI/CD pipeline ensures seamless deployments and updates. 🚀