Automating Cloud Infrastructure Deployment with Terraform and Ansible: A Comprehensive Guide

Infrastructure as Code (IaC) is the cornerstone of automation. With the right tools, it’s possible to manage cloud resources, deploy applications, and configure servers without needing to manually perform these tasks each time.

In this article, we will walk you through an end-to-end project involving Terraform and Ansible to provision cloud infrastructure, deploy application and monitoring stacks, and configure everything automatically. This project will provide a deep dive into Docker, Terraform, and Ansible, as well as how they work together to automate your infrastructure setup and deployment pipeline.

Overview of the Project

This project involves:

1. Building Docker images for a frontend and backend application.


2. Designing and provisioning cloud infrastructure using Terraform.


3. Using Terraform to trigger Ansible playbooks for deploying the application stack and setting up the monitoring stack.


4. Creating a reverse proxy using Traefik or Nginx for routing between services.


5. Configuring monitoring using Prometheus, Grafana, cAdvisor, and Loki.

The ultimate goal is to run a single Terraform command that provisions the infrastructure, deploys all services, and configures routing automatically.

Step 1: Building and Pushing Docker Images

Before we can deploy anything, we need Docker images for our frontend and backend applications. This step ensures that we have prebuilt images ready for deployment on any cloud instance.

Frontend Application

For the frontend, we use React to build a single-page application. The Dockerfile for the frontend might look like this:

#-----------------

# Building the app

#-----------------

FROM node:17-alpine AS build



WORKDIR /app/frontend



COPY package*.json ./



RUN npm install



COPY . .



ARG VITE_API_URL



ENV VITE_API_URL=$VITE_API_URL



RUN npm run build



#----------------

# Serving the app

#----------------

FROM nginx:alpine



COPY --from=build /app/frontend/dist /usr/share/nginx/html



COPY nginx.conf /etc/nginx/conf.d/default.conf



EXPOSE 80



CMD [ "nginx", "-g", "daemon off;" ]

Backend Application

The backend is built using FastAPI, which is a modern, fast web framework for building APIs with Python. The Dockerfile for the backend would look something like this:

FROM python:3.11 AS build



WORKDIR /app/backend



COPY pyproject.toml poetry.lock ./



RUN pip install poetry



RUN poetry config virtualenvs.create false && \

    poetry install --no-root



COPY . .



RUN pip install sqlalchemy sqlmodel alembic



ENV PYTHONPATH=/app/backend



RUN chmod +x /app/backend/prestart.sh



EXPOSE 8000



CMD ["sh", "-c", "./prestart.sh && uvicorn app.main:app --host 0.0.0.0 --port 8000"]

Once the Dockerfiles are in place, you build and push the images to Docker Hub (or any other container registry of your choice):

# Build the frontend image

docker build -t <your-username>/frontend:latest .



# Build the backend image

docker build -t <your-username>/backend:latest .



# Push the images to Docker Hub

docker push <your-username>/frontend:latest

docker push <your-username>/backend:latest

This will make the Docker images available for deployment later on when using Terraform and Ansible.

Step 2: Designing the Infrastructure

Before provisioning resources, it’s essential to define the architecture. In this project, the architecture involves:

• A virtual machine (VM) that hosts the application stack (frontend, backend, and reverse proxy).


• 
  Networking to ensure the components can communicate with each other.


• 
  A reverse proxy (either Traefik or Nginx) to route traffic between services.


• 
  Prometheus, Grafana, cAdvisor, Loki, and Promtail to monitor the applications and collect logs.

Architecture Diagram

Here’s a simple architecture diagram to help visualize the components:

Step 3: Writing Terraform Configuration

Terraform is used to provision the infrastructure, such as the cloud instance (VM), and to trigger the execution of the Ansible playbook. The main.tf file will define the infrastructure, and it will include:

• The creation of an AWS EC2 instance.


• An Elastic IP (EIP) to give the instance a static public IP.


• A security group that allows SSH access.


• 
  Outputs to show important information such as the public IP of the instance.

Here is a simple Terraform configuration for provisioning an AWS EC2 instance:

provider "aws" {

  region = "us-east-1"

}



resource "aws_instance" "app" {

  ami           = "ami-12345678"  # Replace with an appropriate AMI ID

  instance_type = "t2.micro"

  key_name      = "your-key-pair"



  tags = {

    Name = "App Instance"

  }



  associate_public_ip_address = true

}



resource "aws_eip" "dojo-eip" {

  instance = aws_instance.app.id

}



output "public_ip" {

  value = aws_instance.app.public_ip

}

Terraform and Ansible Integration

Terraform will use Ansible to configure the application after the EC2 instance is created. The null_resource and remote-exec provisioner are used to execute commands on the instance via SSH.

Here is the relevant Terraform configuration that integrates Ansible with Terraform:

resource "null_resource" "ansible" {

  provisioner "remote-exec" {

    connection {

      host        = aws_instance.app.public_ip

      user        = "ubuntu"

      private_key = file("/path/to/your/private-key.pem")

    }



    inline = ["echo 'connected!'"]

  }



  provisioner "local-exec" {

    command = "ansible-playbook -vvv -T 180 -i /path/to/inventory /path/to/ansible/playbook.yml --extra-vars '@/path/to/ansible/vars/docker_hub.yml'"

  }



  depends_on = [aws_instance.app]

}

Step 4: Writing Ansible Playbooks

Ansible is used for configuring the application and monitoring stacks. The playbook.yml file will define the steps to set up the system.

1. 
   Server Setup: Install necessary dependencies and configure the environment.


2. 
   Application Setup: Pull Docker images from Docker Hub and run the frontend and backend services.


3. 
   Monitoring Setup: Install Prometheus, Grafana, and other monitoring tools.


4. 
   Reverse Proxy Setup: Configure Traefik or Nginx to route traffic.

Here’s an example of the playbook:

---

- name: Configure app server

  hosts: all

  become: yes

  tasks:

    - name: Install Docker

      apt:

        name: docker.io

        state: present



    - name: Create a shared Docker network

      command: docker network create app-network



    - name: Pull frontend Docker image

      docker_image:

        name: "{{ docker_hub.frontend_image }}"

        source: pull



    - name: Pull backend Docker image

      docker_image:

        name: "{{ docker_hub.backend_image }}"

        source: pull



    - name: Run frontend container

      docker_container:

        name: frontend

        image: "{{ docker_hub.frontend_image }}"

        state: started

        restart_policy: always

        networks:

          - name: app-network



    - name: Run backend container

      docker_container:

        name: backend

        image: "{{ docker_hub.backend_image }}"

        state: started

        restart_policy: always

        networks:

          - name: app-network



    - name: Install Traefik for routing

      docker_container:

        name: traefik

        image: traefik:v2.5

        state: started

        restart_policy: always

        ports:

          - "80:80"

          - "443:443"

        command: "--api.insecure=true --providers.docker"



    - name: Install Prometheus

      docker_container:

        name: prometheus

        image: prom/prometheus

        state:



 started

        restart_policy: always

        ports:

          - "9090:9090"

        volumes:

          - /prometheus.yml:/etc/prometheus/prometheus.yml

Step 5: Finalizing the Deployment

Once the Terraform configuration and Ansible playbooks are set up:

1. Run terraform apply to provision the infrastructure.


2. Terraform will automatically trigger Ansible, which will configure the server and deploy the application.


3. After execution, you can access the application via the public IP provided by Terraform, and the monitoring stack (Grafana, Prometheus, etc.) will be up and running.

Testing the Setup

You can test the entire setup by visiting the public IP address in your browser. Ensure that:

• The frontend and backend applications are accessible.


• The monitoring dashboard in Grafana is functioning.


• Logs are being collected by Loki.

Conclusion

This project demonstrated how to automate cloud infrastructure provisioning and application deployment using Terraform and Ansible. By combining IaC and CM, you can ensure repeatable, consistent, and scalable deployments. In this case, we also integrated monitoring into the workflow, which is an essential part of maintaining cloud-based systems.

Through this guide, you learned how to:

1. Build and push Docker images for your applications.


2. Write Terraform configurations to provision cloud resources.


3. Use Ansible to configure servers and deploy applications.


4. Set up monitoring using Prometheus, Grafana, and other tools.

This approach can be adapted to suit more complex applications and environments, making it an invaluable skill for any DevOps engineer or cloud automation specialist. Happy automating!