How to Use Terraform with Python for Infrastructure

Imagine you’re staring at a terminal, manually running terraform init, plan, and apply for the third time today because your CI/CD pipeline failed again. You know Python is your superpower for automation, testing, and glue logic—but Terraform feels like a rigid, HCL-only black box. What if you could bring Terraform under Python’s control, orchestrating infrastructure deployments with the same elegance you use to build APIs or data pipelines? That’s not just possible; it’s practical, powerful, and ready for you to use today.

Why Combine Terraform and Python?

Terraform excels at declarative infrastructure-as-code (IaC), letting you define cloud resources in human-readable HCL files. Python, meanwhile, shines in automation, scripting, and integrating disparate systems. Together, they unlock a hybrid workflow: write your infrastructure in .tf files (keeping Terraform’s strengths), then drive the entire lifecycle—init, plan, apply, output parsing—from Python scripts.

This approach is ideal for:

• Automating repetitive Terraform workflows in CI/CD


• Dynamically generating Terraform variables based on application state


• Chaining Terraform modules with conditional logic


• Extracting and transforming Terraform outputs for downstream systems

You don’t need to rewrite your Terraform configs in Python. Instead, you use Python as the control layer.

The Go-To Tool: python-terraform

The most straightforward way to bridge Python and Terraform is the python-terraform package. It provides a Pythonic wrapper around Terraform CLI commands, letting you call init(), plan(), apply(), and output() directly from your scripts.

Step 1: Install the Package

First, install the library:

pip install python-terraform

Make sure Terraform CLI is already installed and accessible in your environment’s PATH. You can verify with:

terraform -v

Step 2: Prepare Your Terraform Configuration

Create a standard Terraform file (e.g., main.tf) in a dedicated directory. Here’s a minimal AWS EC2 example:

provider "aws" {

  region = "us-west-2"

}



resource "aws_instance" "example" {

  ami           = "ami-0c94855ba95c71c99"

  instance_type = "t2.micro"



  tags = {

    Name = "Python-Terraform-Example"

  }

}

Save this in a folder like ./terraform/aws-ec2.

Step 3: Drive It from Python

Now, write a Python script to manage the lifecycle:

import terraform



# Initialize the Terraform working directory

tf = terraform.Terraform(working_dir='./terraform/aws-ec2')



# Initialize providers and plugins

tf.init()



# Plan the infrastructure changes (optional for preview)

tf.plan()



# Apply changes, skipping interactive approval

tf.apply(skip_plan=True)



# Fetch and print Terraform outputs

output = tf.output()

print("Terraform outputs:", output)

Run this script:

python deploy_ec2.py

You’ll see Terraform initialize providers, plan the change, apply it, and print any outputs (like instance IDs if you define them).

Note: python-terraform doesn’t handle provider authentication (e.g., AWS keys). You must configure credentials via environment variables, ~/.aws/config, or Terraform’s built-in auth mechanisms before running the script [1].

Advanced: Dynamic Variables and Conditional Logic

What if you want to spin up different instance types based on environment (dev vs. prod)? Python lets you inject variables dynamically:

import terraform

import os



env = os.getenv("ENV", "dev")

ami = "ami-0c94855ba95c71c99" if env == "dev" else "ami-0d94855ba95c71c88"

instance_type = "t2.micro" if env == "dev" else "t2.small"



tf = terraform.Terraform(working_dir='./terraform/aws-ec2')



# Pass variables dynamically

tf.apply(

    skip_plan=True,

    var={

        "ami": ami,

        "instance_type": instance_type,

        "env": env

    }

)



print(tf.output())

You’d also update main.tf to accept these as variables:

variable "ami" {}

variable "instance_type" {}

variable "env" {}



resource "aws_instance" "example" {

  ami           = var.ami

  instance_type = var.instance_type



  tags = {

    Name = "Python-Terraform-${var.env}"

  }

}

This pattern scales beautifully for multi-environment deployments, feature-flagged infra, or integration with application state.

Alternative Approach: CDK for Terraform (cdktf)

If you prefer writing infrastructure in Python instead of HCL, HashiCorp offers CDK for Terraform (cdktf). It lets you define resources using Python classes, then synthesizes them into Terraform JSON before deploying.

Install prerequisites:

npm install -g cdktf-cli

pip install cdktf

Initialize a Python project:

cdktf init --template=python --project=my-project

Define resources in Python:

from cdktf import App, TerraformStack

from aws_ec2 import AwsEc2Instance  # hypothetical provider module



class MyStack(TerraformStack):

    def __init__(self, scope: App, id: str):

        super().__init__(scope, id)

        instance = AwsEc2Instance(self, "example", {

            "ami": "ami-0c94855ba95c71c99",

            "instance_type": "t2.micro"

        })



app = App()

MyStack(app, "my-stack")

app.synth()

Then deploy:

cdktf deploy

While powerful, cdktf adds complexity and a Node.js dependency. For most teams, python-terraform + HCL files is simpler and more maintainable [2].

Real-World Use Cases You Can Implement Today

Here are three actionable scenarios:

1. CI/CD Pipeline Integration: Wrap your deploy_ec2.py script in a GitHub Actions or GitLab CI job. Pass environment variables dynamically and parse outputs to update deployment dashboards.




2. Infrastructure Testing: Use Python’s pytest to validate Terraform outputs. For example, assert that an instance is created with the correct tags or that a database endpoint is reachable.




3. Multi-Cloud Orchestration: Chain multiple Terraform modules (AWS, Azure, GCP) in one Python script. Conditionally deploy based on user input or system state.

Common Pitfalls and How to Avoid Them

• 
  Missing Credentials: Always set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (or use aws configure) before running scripts [1].


• 
  Wrong Working Directory: Ensure working_dir points to the folder containing main.tf. A missing config will cause init() to fail.


• 
  Interactive Prompts: Use skip_plan=True in apply() to avoid blocking on the “Proceed with terraform apply?” prompt.


• 
  Provider Auth: python-terraform doesn’t handle OAuth or API tokens. Configure them via Terraform’s native mechanisms or environment variables.

Conclusion: Your Infrastructure, Now Python-Controlled

You don’t need to abandon HCL to bring Python into your Terraform workflow. With python-terraform, you get a clean, scriptable interface to drive the full IaC lifecycle—init, plan, apply, and output parsing—while keeping your infrastructure definitions declarative and reusable.

Start small: wrap your next terraform apply in a Python script. Add dynamic variables. Chain modules. Test outputs. Within an hour, you’ll have a reusable automation layer that saves you time, reduces errors, and scales with your team.

Your call to action: Pick one Terraform module you use regularly, write a python-terraform script to deploy it, and run it today. Share your script on Dev.to or in your team’s Slack—someone else is probably struggling with the same manual workflow. Let’s automate smarter, together.

If you found this helpful, consider — free AI tools for developers.