There have been a few articles on Terraform stacks and how some of the core concepts with Stacks help you manage the deployment/provisioning of those multi region/multi account scenarios. I had one of those scenarios which I wanted to test to get a better understanding of Stacks.
Scenario
S3 Cross region replication
The name itself suggests the challenge you have with this stack or configuration you are trying to stand up.
- An S3 bucket in one region
- A replication bucket in another region
- The replication rules and necessary IAM roles in the first region referencing the second bucket.
Considering the way the Terraform workspaces are setup, this can be a pain to manage the relationships and dependencies you have here.
Add in bidirectional replication and you have a bigger challenge with all those dependencies you will need to manage and orchestrate.
Enter Stacks
Terraform stacks are a configuration layer in HCP Terraform that simplifies the management of your infrastructure modules with some of the following use cases
- Deploy an entire application with components like networking, storage, and compute as a single unit without worrying about dependencies
- Deploy across multiple regions, availability zones, and cloud provider accounts without duplicating effort/code
How is this done, though ? There are a few pieces to the puzzle.
Components, Providers and variables.
Components are an alternative to module invocations. If you are familiar with using a module with a source and variable inputs, you will feel right at home here.
component "cluster" {
source = "" # Can be local or from a source you can pull from
inputs = {
...... # variable inputs you would want to provide to the undelying module
}
providers = {
... # Not entirely new, but providers via components are a necessity as modules used for components cannot have provider definition in them. So this is your way to add provider details.
}
}
- Defined under a
<name>.tfstacks.hclfile.
Providers in a high level are your conifguration mechanisms to interact with a target plaform ( Cloud provider, Artifact registry etc..)
- They are different from your existing provider definitions in that they resemble any of your resource/datasource blocks you may currently have with the named blocks.
provider "aws" "<name_you_can_remember>" {
config {
region = <region>
assume_role_with_web_identity {
role_arn = .. # ARN of the role of the target platform you are deploying to
web_identity_token = .. # Workload identity token generated by a run in HCP Terraform
}
}
}
- Defined under a
providers.tfstacks.hclfile.
Variables are not a new concept and they mirror your root module variables if you have set one before.
- Defined under a
variables.tfstacks.hclfile. - Reference:
Orchestration rules
Rules written in HCL which allow you to automation some of the repetitive actions based on some context information your stacks have based on the deployments. Terraform has been slowly easing us into this concept of checks with a condition and error_message to review a certain rule. This looks very similar to that in view, but helps to orchestrate the automation of approvals of your stack.
- One of the orchestration rules available is
auto_approvewhich can be based on some check in the format below.
orchestrate "auto_approve" “safe_plans” {
check {
#check that there are no resources being removed
condition = context.plan.changes.remove == 0
reason = "Plan has ${context,plan.changes. remove} resources to be removed."
}
}
Additional ones
store block
A store block can be used to reference a variableset in your HCP Terraform project which can include some credentials you want to read on runtime. Specify the variables you are assigning this value to as ephemeral and you can avoid having your variable value in your logs or state. I wish these could reference a name than an id since they are anyway scoped to the project.
store "varset" "roles" {
id = "<variables-set-id>"
category = "env"
}
# Access your variable set's value using your store and pass them into your
# deployment's inputs.
deployment "dev" {
inputs = {
role_arn = store.varset.roles.role_dev
}
}
identity_token block
The identity_token block gives you a token that you will be used to authorize the AWS provider in this case. This bases itself on the dynamic credentials mechanism which HCP Terraform has with AWS. There are few updates to the existing workspace based one if you compare. The trust relationship on the role would look something like below:
"Condition": {
"StringEquals": {
"app.terraform.io:aud": "aws.workload.identity"
},
"StringLike": {
"app.terraform.io:sub": "organization:<org_name>:project:<proj_name>:stack:<stack_name>:deployment:<deployment_name>:operation:<operation_type>"
}
}
Read more about the OIDC based authentication
.
├── README.md
├── components.tfstack.hcl
├── deployments.tfdeploy.hcl
├── providers.tfstack.hcl
└── variables.tfstack.hcl
├── modules
│ ├── replication
│ │ ├── main.tf
│ │ ├── outputs.tf
│ │ └── variables.tf
│ └── s3
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
Modules
There are two modules:
- An S3 bucket module which deploys an S3 bucket and enables versioning with the bucket arn and id as outputs.
- A replication module with an IAM role, policy and an S3 replication rule referencing a source and destination bucket.
Providers
Providers are defined in the root of the stack configuration here with the required inputs and region references. You could even add a for_each loop to the provider block if you so desire to make some components be deployed across multiple regions and referenced once ( Like in an IPAM Hub and spoke model)
required_providers {
aws = {
source = "hashicorp/aws"
version = "5.72.1"
}
}
provider "aws" "source" {
config {
region = "us-east-1"
assume_role_with_web_identity {
role_arn = var.role_arn
web_identity_token = var.identity_token
}
}
}
provider "aws" "dest" {
config {
region = "us-west-2"
assume_role_with_web_identity {
role_arn = var.role_arn
web_identity_token = var.identity_token
}
}
}
Components
We are dividing this entire operation into 3 components.
- One to provision the source S3 bucket
- Second to provision the destination S3 bucket
- Third to create the IAM roles and replication rules.
Note that the provider references for source and desintation are different.
component "source" {
source = "./modules/s3"
inputs = {
bucket_name = "${var.source_bucket_name}-${var.suffix}"
}
providers = {
aws = provider.aws.source
}
}
component "destination" {
source = "./modules/s3"
inputs = {
bucket_name = "${var.destination_bucket_name}-${var.suffix}"
}
providers = {
aws = provider.aws.dest
}
}
component "replication" {
source = "./modules/replication"
inputs = {
role_name = var.replication_role
policy_name = var.replication_policy
source_bucket_arn = component.source.bucket_arn
destination_bucket_arn = component.destination.bucket_arn
source_bucket = component.source.bucket_id
}
providers = {
aws = provider.aws.source
}
}
Deployments
We have a deployment below which provisions the above infrastructure stack into those two regions. If we wanted to provision this to another account, all we need is another deployment block with a scoped role_arn.
identity_token "aws" {
audience = ["aws.workload.identity"]
}
store "varset" "roles" {
id = "varset-6pcUK8q4FQVLBRJY"
category = "env"
}
deployment "dev" {
inputs = {
identity_token = identity_token.aws.jwt
role_arn = store.varset.roles.role_dev
source_bucket_name = "manu-2024-source"
destination_bucket_name = "manu-2024-dest"
suffix = "dev"
replication_role = "stacks-replication"
replication_policy = "stacks-replication"
}
}
tfstacks CLI
The terraform-stacks-cli is a command-line tool for validating, initializing, and testing Stack configurations. I will skip the installation steps here which you can easily find in the documentation below.
More about the tfstacks CLI
Plan details :
When the plan is executed, it does follow the source->destination->replication flow. I wish the plan structure shown in the applied configuration showed that order.
Conclusion
The idea of being able to provision this across multiple accounts and regions easily with a single infrastructure construct is a thing of beauty. I have had to deal with managing those dependencies in many ways to get a cross region replication setup configured for a source S3 bucket before this. We will dive into deferred changes and other stack specific details in a future article. Hopefully this gave you some more details into what Stacks offer from an infrastructure management standpoint.
You can infact extend the replication component to make this a bidirectional replication configuration with very little changes.
SOCIAL SHARE CARD GENERATOR