🔧 Containers, Containers, Containers!
Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to
Yep, you already know how to build your container. But now you want to get that container running on AWS with as little fuss as possible, right? No problem. ⚡
In this quick guide, I'll show you how to deploy your container to AWS Fargate
using CloudFormation
. It’s fast, easy, and doesn’t require you to wade through endless steps. Just the essentials to get your container online.
Before We Begin
We’re using CloudFormation
here because it’s perfect for quickly spinning up your infrastructure with minimal hassle. Sure, there are other tools like Terraform or CDK, but for a straightforward ECS setup, CloudFormation
is your go-to.
We’re keeping it simple: we’ll create a VPC
, a couple of public subnets
, an ECS cluster
, and a load balancer
to get your containers up and running. No need for extra complexity—just the basics to get your containerized app deployed fast. And yes, we’ve got IAM roles
to keep everything secure. Let’s dive in!
Table of Contents
- Architecture Boring Stuff
-
LET'S KICK IT OFF!
- Step-01: Deploy CloudFormation Stack for ECS Fargate
- Step-02: Create AWS IAM User for Pipeline
- Step-03: Add Secrets to GitHub Secrets
- Step-04: Github Actions
- Step-05: Cleaning Up
- Pricing
- Conclusion
This table of contents should make it easy to navigate through the guide.
Architecture Boring Stuff
User Interaction
-
User Request: A user sends an HTTP request to the Application Load Balancer (ALB) (
ALBDNSName
). - ALB Routing: The ALB routes the request to the ECS service, which runs your containerized application.
- ECS Task Handling: The ECS tasks, running within the Fargate cluster, handle the request, with containers managed by ECS.
ECS and Load Balancer
-
ECS Fargate Cluster (
ECSFargateCluster
): Manages and scales the containers based on defined parameters. -
Application Load Balancer (
ALB
): Distributes incoming traffic to the appropriate ECS tasks, ensuring availability and responsiveness. -
Target Group (
ALBTargetGroup
): Monitors the health of your ECS tasks and directs traffic only to healthy instances.
IAM Roles
-
IAMRoleForECS
: Grants permissions for ECS tasks to pull images from ECR, send logs to CloudWatch, and perform other necessary actions. -
ECRCleanupLambdaRole
: Allows the Lambda function to delete images from the ECR repository during the cleanup process.
Pipeline User for GitHub Actions
The GitHub Actions workflow uses an IAM user (PipelineUser
) to deploy updates to the ECS service. The workflow includes steps for:
- Checking out the code.
- Building the container image.
- Pushing the image to ECR.
- Configuring AWS credentials.
- Updating the ECS task definition with the new image.
- Triggering a new deployment of the ECS service.
Flow
- User -> Application Load Balancer -> ECS Service -> ECS Task (Container)
- ECS Tasks -> Fargate Cluster (for running containers)
- Pipeline User -> GitHub Actions -> ECR and ECS (for deployment)
cloudformation-template.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: AWS ECS Infrastructure setup using a new VPC
Parameters:
VpcCidr:
Description: "CIDR block for the VPC"
Type: String
Default: "30.0.0.0/16"
PublicSubnet1Cidr:
Description: "CIDR block for the public subnet 1"
Type: String
Default: "30.0.1.0/24"
PublicSubnet2Cidr:
Description: "CIDR block for the public subnet 2"
Type: String
Default: "30.0.2.0/24"
ECRRepositoryName:
Description: The name of the ECR repository
Type: String
Default: "application-repository"
ClusterName:
Description: The name of the ECS Fargate Cluster
Type: String
Default: "EcsFargateCluster"
ServiceName:
Description: The name of the ECS Service
Type: String
Default: "EcsService"
TaskFamilyName:
Description: The family name of the ECS Task Definition
Type: String
Default: "TaskDefinition"
ContainerName:
Description: The name of the container in the ECS Task Definition
Type: String
Default: "application-container"
Resources:
# VPC and Networking resources
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref VpcCidr
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: vpc
# Internet Gateway
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: igw
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
# Public Subnets
PublicSubnet1:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
CidrBlock: !Ref PublicSubnet1Cidr
AvailabilityZone: !Select
- 0
- Fn::GetAZs: !Ref "AWS::Region"
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: public-subnet-1
PublicSubnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VPC
CidrBlock: !Ref PublicSubnet2Cidr
AvailabilityZone: !Select
- 1
- Fn::GetAZs: !Ref "AWS::Region"
MapPublicIpOnLaunch: true
Tags:
- Key: Name
Value: public-subnet-2
# Route Tables and Routes
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref VPC
Tags:
- Key: Name
Value: public-rt
PublicRoute:
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref InternetGateway
PublicSubnet1RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet1
RouteTableId: !Ref PublicRouteTable
PublicSubnet2RouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnet2
RouteTableId: !Ref PublicRouteTable
# Security Group
SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Default security group for the VPC"
VpcId: !Ref VPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 443
ToPort: 443
CidrIp: 0.0.0.0/0
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
SecurityGroupEgress:
- IpProtocol: "-1"
CidrIp: 0.0.0.0/0
Tags:
- Key: Name
Value: default-sg
# ECS Fargate Cluster
ECSFargateCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Ref ClusterName
# ECR Repository
ECRRepository:
Type: AWS::ECR::Repository
Properties:
RepositoryName: !Ref ECRRepositoryName
# Custom Resource Hook for ECR Cleanup
ECRCleanupHook:
Type: "Custom::ECRCleanup"
Properties:
ServiceToken: !GetAtt ECRCleanupLambdaFunction.Arn
RepositoryName: !Ref ECRRepositoryName
DependsOn: ECRRepository
# ECS Task Definition
ECSTaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Ref TaskFamilyName
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: "512"
Memory: "1024"
ExecutionRoleArn: !Ref IAMRoleForECS
TaskRoleArn: !Ref IAMRoleForECS
ContainerDefinitions:
- Name: !Ref ContainerName
Image: "nginx:latest"
PortMappings:
- ContainerPort: 80
Environment:
- Name: ENVIRONMENT
Value: "production"
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref CloudWatchLogGroup
awslogs-region: !Ref "AWS::Region"
awslogs-stream-prefix: ecs
# ECS Service
ECSService:
Type: AWS::ECS::Service
DependsOn: ALB
Properties:
Cluster: !Ref ECSFargateCluster
ServiceName: !Ref ServiceName
DesiredCount: 2
LaunchType: FARGATE
TaskDefinition: !Ref ECSTaskDefinition
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- !Ref SecurityGroup
Subnets:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
LoadBalancers:
- ContainerName: !Ref ContainerName
ContainerPort: 80
TargetGroupArn: !Ref ALBTargetGroup
# ALB
ALB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: "ALB"
Subnets:
- !Ref PublicSubnet1
- !Ref PublicSubnet2
SecurityGroups:
- !Ref SecurityGroup
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: "60"
ALBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
LoadBalancerArn: !Ref ALB
Port: 80
Protocol: HTTP
DefaultActions:
- Type: forward
TargetGroupArn: !Ref ALBTargetGroup
ALBTargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
VpcId: !Ref VPC
Protocol: HTTP
Port: 80
TargetType: ip
HealthCheckEnabled: true
HealthCheckPath: "/" # Set to a valid endpoint in your application
HealthCheckIntervalSeconds: 30
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
UnhealthyThresholdCount: 3
Matcher:
HttpCode: "200"
Name: "TargetGroup"
# IAM Role for ECS Task
IAMRoleForECS:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- ecs-tasks.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: "ECRPullPolicy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ecr:GetAuthorizationToken
- ecr:GetDownloadUrlForLayer
- ecr:BatchGetImage
- ecr:BatchCheckLayerAvailability
Resource: "*"
- PolicyName: "CloudWatchLogsPolicy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
- logs:CreateLogGroup
Resource: "*"
# CloudWatch Log Group
CloudWatchLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: "/ecs/logs"
RetentionInDays: 7
# Lambda Function to Delete ECR Images
ECRCleanupLambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: LambdaCloudWatchLogsPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- arn:aws:logs:*:*:*
- PolicyName: ECRDeletePolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ecr:BatchDeleteImage
- ecr:ListImages
Resource: !Sub "arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${ECRRepositoryName}"
ECRCleanupLambdaFunction:
Type: AWS::Lambda::Function
Properties:
FunctionName: ecr-cleanup-lambda
Handler: index.handler
Runtime: python3.9
Role: !GetAtt ECRCleanupLambdaRole.Arn
Timeout: 60
Code:
ZipFile: |
import boto3
def handler(event, context):
if event['RequestType'] == 'Delete':
ecr = boto3.client('ecr')
images_to_delete = ecr.list_images(repositoryName=event['repository_name'])['imageIds']
if images_to_delete:
ecr.batch_delete_image(
repositoryName=event['repository_name'],
imageIds=images_to_delete
)
return 'Cleanup complete'
Outputs:
VpcId:
Value: !Ref VPC
Description: The VPC Id where the ECS Cluster is deployed.
ECSFargateCluster:
Value: !Ref ECSFargateCluster
Description: The ECS Fargate Cluster created by the template.
ALBDNSName:
Value: !GetAtt ALB.DNSName
Description: The DNS name of the Application Load Balancer.
ALBEndpoint:
Value: !Sub "http://${ALB.DNSName}"
Description: The HTTP endpoint of the Application Load Balancer.
The same template without creating a new vpc:
cloudformation-template-with-existing-vpc.yaml
Note: It's important to edit the task's port if needed. By default, this is set up with a placeholder using nginx
listening on port 80
and the health check
is set to /
. However, you can change these settings to fit your requirements.
LET'S KICK IT OFF!
Alright, buckle up! If you've got an AWS account and the AWS CLI locked and loaded, we're ready to roll. (Pro tip: jq is handy, but hey, we won't judge if you skip it.)
Step-01: Deploy CloudFormation Stack for ECS Fargate
export CLUSTER_NAME=EcsFargateCluster
export SERVICE_NAME=EcsService
export TASK_FAMILY_NAME=TaskDefinition
export REPOSITORY_NAME=application-repository
export CONTAINER_NAME=application-container
export STACK_NAME=aws-ecs-fargate-stack
aws cloudformation deploy \
--stack-name $STACK_NAME \
--template-file ecs-fargate-cloudformation.yaml \
--parameter-overrides \
ClusterName=$CLUSTER_NAME \
ServiceName=$SERVICE_NAME \
TaskFamilyName=$TASK_FAMILY_NAME \
ECRRepositoryName=$REPOSITORY_NAME \
ContainerName=$CONTAINER_NAME \
--capabilities CAPABILITY_NAMED_IAM \
--region us-east-1
Note: The stack uses predefined default names for easy identification in the pipeline, u can override it:
-
ClusterName:
EcsFargateCluster
-
ServiceName:
EcsService
-
TaskFamilyName:
TaskDefinition
-
ECRRepositoryName:
application-repository
-
ContainerName:
application-container
Step-02: Create AWS IAM User for Pipeline
In this step, we'll create a Pipeline User that GitHub Actions will use. This user will have the necessary policies attached to push container images to ECR and update our ECS service. Here's how to do it:
# Set user name
export USER_NAME="codepipeline"
# Create IAM user
aws iam create-user --user-name $USER_NAME
# Attach necessary policies
# ⚠️ THIS IS NOT PRODUCTION READY - USE A LEAST PRIVILEGE ROLE INSTEAD
aws iam attach-user-policy --user-name $USER_NAME --policy-arn arn:aws:iam::aws:policy/AmazonECS_FullAccess
aws iam attach-user-policy --user-name $USER_NAME --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess
# Create access keys for the user
ACCESS_KEYS=$(aws iam create-access-key --user-name $USER_NAME)
# Extract AccessKeyId and SecretAccessKey
ACCESS_KEY_ID=$(echo $ACCESS_KEYS | jq -r '.AccessKey.AccessKeyId')
SECRET_ACCESS_KEY=$(echo $ACCESS_KEYS | jq -r '.AccessKey.SecretAccessKey')
# ⚠️ Output the Access Key details
echo "Access Key ID: $ACCESS_KEY_ID"
echo "Secret Access Key: $SECRET_ACCESS_KEY"
Step-03: Add Secrets to GitHub Secrets
After creating the IAM user and generating the access keys, follow these steps to add these credentials to your GitHub
repository secrets for use in GitHub Actions:
-
Navigate to Your GitHub Repository:
- Go to the main page of your repository on GitHub.
-
Access Repository Settings:
- Click on the Settings tab at the top of the repository page.
-
Navigate to Secrets:
- In the left sidebar, click on Secrets and variables > Actions.
-
Add New Repository Secrets:
- Click New repository secret and add the following:
-
AWS_ACCESS_KEY_ID: Enter your
AccessKeyId
value. -
AWS_SECRET_ACCESS_KEY: Enter your
SecretAccessKey
value.
By doing this, you'll securely store the necessary credentials for your GitHub Actions workflow.
Step-04: Github Actions
Time to automate the deployment with GitHub Actions! Here's how you can set up your pipeline:
Copy the following YAML content into .github/workflows/deploy.yml
:
name: Build, Push to ECR, and Deploy to ECS Fargate
on:
push:
branches:
- main
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
# Step 1: Checkout the latest code from the main branch
- name: Checkout code
uses: actions/checkout@v3
# Step 2: Configure AWS credentials for the GitHub Actions runner
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1 # Change to your region
# Step 3: Login to Amazon ECR to push Docker images
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
# Step 4: Build the Docker image, tag it, and push it to Amazon ECR
- name: Build, tag, and push image to Amazon ECR
id: build-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: application-repository # Change this to your ECR repository name
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
# Step 5: Download the current ECS task definition to update it with the new image
- name: Download current task definition
run: |
aws ecs describe-task-definition --task-definition TaskDefinition \ # Change this to your Task Family Name
--query taskDefinition > ecs-task-definition.json
# Step 6: Replace the image in the ECS task definition with the newly built image
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: ecs-task-definition.json
container-name: application-container # Change this to your container name
image: ${{ steps.build-image.outputs.image }}
# Step 7: Deploy the updated task definition to the ECS Fargate service
- name: Deploy updated task definition to ECS Fargate
uses: aws-actions/amazon-ecs-deploy-task-definition@v2
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: EcsService # Change this to your ECS service name
cluster: EcsFargateCluster # Change this to your ECS Fargate cluster name
wait-for-service-stability: true
Important: The container image in the ECR repository is updated with each push to the main
branch. The service name EcsService
, cluster name EcsFargateCluster
, and other values are predefined in your CloudFormation stack, ensuring smooth integration with your AWS infrastructure.
Now, commit and push your changes to trigger the pipeline:
git commit -am “feat(ci): deployment”
git push origin main
Once pushed, the pipeline will automatically build, push your Docker image to ECR, and update your ECS Fargate service with the latest code. 🚀
Cleaning Up
To delete the CloudFormation stack, use the following command:
# Set the stack name
export STACK_NAME="aws-ecs-fargate-stack"
# Delete the CloudFormation stack
aws cloudformation delete-stack --stack-name $STACK_NAME
# Delete the IAM user
aws iam delete-user --user-name $USER_NAME
BLAZINGGG ENJOYY 🎉🔥
You can see your pipeline triggered, the Docker image built, pushed to ECR, and your ECS Fargate service updated. This workflow will automatically deploy your changes to the Fargate cluster every time you push to the main branch. 🚀
To check if everything's working, grab the ALB DNS Name from the stack output or the AWS console and make an HTTP request to your application. 🌐
~/edu curl http://your-alb-dns-name/ # Replace with your actual ALB DNS name
Pricing
Here’s an estimated cost breakdown for the AWS ECS infrastructure setup you provided, considering that you have a single ECS Fargate task running:
1. VPC and Subnets
- VPC: No additional cost.
- Subnets: No direct cost for subnets.
2. Internet Gateway
- Cost: $0.045 per hour.
3. ECS Fargate
- vCPU (0.5 vCPU): $0.04048 per vCPU-hour.
- Memory (1 GB): $0.004445 per GB-hour.
For a single task running 24/7:
- vCPU Cost (0.5 vCPU): 0.5 * 24 * 30 * $0.04048 = $14.58 per month.
- Memory Cost (1 GB): 1 * 24 * 30 * $0.004445 = $3.20 per month.
4. Application Load Balancer (ALB)
- ALB: $0.0225 per hour.
- Data Processed by ALB: $0.008 per GB.
For 24/7 operation:
- ALB Cost: 24 * 30 * $0.0225 = $16.20 per month.
5. ECR Storage
- Cost: $0.10 per GB per month.
6. CloudWatch Logs
- Storage: $0.03 per GB.
- Ingested Logs: $0.50 per GB.
Total Estimated Monthly Cost
Adding up the individual components, here’s an estimated monthly cost:
- ECS Fargate: ~$17.78
- ALB: ~$16.20
- Internet Gateway: ~$32.40 (if used)
- ECR Storage: Variable depending on image size.
- CloudWatch Logs: Variable depending on log volume.
Estimated Total: ~$66.38 per month (excluding data transfer and storage variations).
Alternative Pricing without Internet Gateway:
If you use the default AWS network configuration and don’t need an Internet Gateway:
- Estimated Total: ~$34.58 per month (excluding Internet Gateway cost).
Conclusion
While serverless and Fargate offer great convenience, costs can escalate quickly. For instance, running a 0.5 vCPU
and 1GB
memory task costs around $18/month just for compute. Additional services like the Application Load Balancer (ALB) and CloudWatch Logs further increase expenses. If you're working on smaller projects or development environments, consider skipping features like the Internet Gateway or ALB to save costs. Always tailor your setup to your specific needs to optimize cloud spending.
And there you have it! A blazing fast, simple way to deploy your containers on AWS ECS Fargate. Enjoy!
Happy deploying :)
...
🔧 Containers, Containers, Containers!
📈 27.09 Punkte
🔧 Programmierung
🐧 Containers vs no containers
📈 18.06 Punkte
🐧 Linux Tipps
🐧 Linux containers vs docker containers
📈 18.06 Punkte
🐧 Linux Tipps
🔧 AWS Terraform for Single Containers
📈 9.03 Punkte
🔧 Programmierung
🔧 Dev Containers in Visual Studio
📈 9.03 Punkte
🔧 Programmierung
🐧 Linux containers
📈 9.03 Punkte
🐧 Linux Tipps
🔧 Build and sign application containers
📈 9.03 Punkte
🔧 Programmierung
🐧 I want to hear about how you use containers
📈 9.03 Punkte
🐧 Linux Tipps