Cookie Consent by Free Privacy Policy Generator ๐Ÿ“Œ How To Automate Your Deployments To AWS EC2 Using CircleCI And Ansible

๐Ÿ  Team IT Security News

TSecurity.de ist eine Online-Plattform, die sich auf die Bereitstellung von Informationen,alle 15 Minuten neuste Nachrichten, Bildungsressourcen und Dienstleistungen rund um das Thema IT-Sicherheit spezialisiert hat.
Ob es sich um aktuelle Nachrichten, Fachartikel, Blogbeitrรคge, Webinare, Tutorials, oder Tipps & Tricks handelt, TSecurity.de bietet seinen Nutzern einen umfassenden รœberblick รผber die wichtigsten Aspekte der IT-Sicherheit in einer sich stรคndig verรคndernden digitalen Welt.

16.12.2023 - TIP: Wer den Cookie Consent Banner akzeptiert, kann z.B. von Englisch nach Deutsch รผbersetzen, erst Englisch auswรคhlen dann wieder Deutsch!

Google Android Playstore Download Button fรผr Team IT Security



๐Ÿ“š How To Automate Your Deployments To AWS EC2 Using CircleCI And Ansible


๐Ÿ’ก Newskategorie: Programmierung
๐Ÿ”— Quelle: dev.to

Introduction

In today's world of fast paced releases and distributed teams working across flexible hours, the importance of automated deployments cannot be overstated. In this tutorial I will guide you through the steps to configure CircleCi and Ansible for automated deployments to AWS EC2.

Prerequisite

To get the most out of this tutorial and follow along it will be nice to have:

  • Basic programming experience and be comfortable with the command line.
  • A Github account and access to command line with git installed.
  • An AWS account with EC2 server set up.
  • A CircleCI account.
  • A code editor.

This tutorial will focus on configuration and deployment only, we won't be setting up new servers. I will be using a backend server I wrote a while ago. You can find the starter code here. Fork and clone the repository. Switch to the starter branch to follow along.

CircleCI Set-up

Head over to circleci.com and login. Ensure you are within your personal organization. On the "Projects" page search for aws_ec2_auto_deploy repo and just click "Set Up Project". Select "Use the . circleci/config. yml in my repo" option and set up project. Set up your project environment variables by copying the default values in .env.example file over to Project Settings > Environment Variables on CircleCI project dashboard. Now, let's get down to brass tacks.

Writing config.yml

On your editor, open the config.yml file inside .circleci folder.

CircleCI uses YAML files to configure pipelines which is usually made up of the following blocks of YAML codes:

YAML Block Description
version This specifies the version of CircleCI API you wish to use
commands Contains list of reusable commands
orbs Makes it possible to use certain prewritten functionalities in a job
jobs List of jobs
workflow Defines the order to run jobs

Before deploying your code to the servers, it is good practice to always ensure that all tests are passing. Update your config.yml file with the code below.

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:16.17.0
      - image: redis
        name: redis
      - image: mysql:8.0
        environment:
          MYSQL_ROOT_PASSWORD: 12345
          MYSQL_DATABASE: lend
          MYSQL_USER: abuchikings
          MYSQL_PASSWORD: 12345
    steps:
      - checkout
      - restore_cache:
          keys: [packages]
      - run:
          name: Test MYSQL connection
          command: |
            for i in `seq 1 30`;
            do
              nc -z 127.0.0.1 3306 && echo Success && exit 0
              echo -n .
              sleep 1
            done
            echo Failed waiting for MySQL && exit 1

      - run:
          name: Run tests
          command: |

            npm install
            npm run migrate:latest
            npm run test

      - save_cache:
          paths:
            - "./node_modules"
          key: packages

workflows:
  default:
    jobs:
      - test

Here is a break down of what is happening in the code above:

  • The jobs keyword indicates that the code block is going to contain jobs.
  • The very first job is named test and the docker keyword tells CircleCI that this job is going to be executed within a docker environment.
  • cimg/node is CircleCI's official nodejs image on docker hub. Node version 16.17.0 is used for this project. MySQL 8.0 database and redis are required to test the application.
  • steps: As its name suggests defines the steps required to run a specific job.
  • checkout: This checks out the source code from your version control repository.
  • restore_cache: Restores cached dependencies. Caching is a mechanism that allows you to save and reuse dependencies (e.g., npm packages) to speed up the build process. In this case, it restores cached npm packages.
  • run: This step runs a series of commands. It's responsible for running the actual CI tasks.
  • name: This provides a descriptive name for this step, indicating that the following commands are related to running tests.
  • command: Specifies the commands to be executed. The | symbol is used to define a multi-line command block, allowing the execution of multiple commands within this step. The first command utilizes nc (netcat) utility to check if a MySQL database server running on the localhost (127.0.0.1) and listening on port 3306 is available. The second command runs migration and automated tests on the code.
  • save_cache: This is a built-in CircleCI command used to save cached dependencies or artefacts. It ensures that the specified files or directories are cached for future builds.
  • workflows: This section defines one or more workflows. A workflow is a series of jobs that need to be executed in a particular order or under specific conditions.

Now you can commit your changes and push them to remote repository. Head over to CircleCI dashboard to see your pipeline run.

Next update your config.yml with the setup_ansible job below. Remember to to also update your workflow as shown in the code snippet below.

  setup_ansible:
    docker:
      - image: amazon/aws-cli
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            yum install -y tar gzip
      - run:
          name: Add Backend IP To Ansible
          command: |
            backend_ip=$(aws ec2 describe-instances \
            --query "Reservations[*].Instances[*].PublicIpAddress" \
            --filters "Name=tag:Name,Values=ec2_auto_deploy_test" \
            --output text)
            echo "$backend_ip" >> ~/project/.circleci/ansible/inventory.txt
            cat ~/project/.circleci/ansible/inventory.txt
      - persist_to_workspace:
          root: ~/
          paths:
            - project/.circleci/ansible/inventory.txt

workflows:
  default:
    jobs:
      - test
      - setup_ansible:
          requires: [test]

Remember to change ec2_auto_deploy_test to the name of your EC2 server on AWS.

The setup_ansible job uses aws-cli to get the IP address of an EC2 server with name tag ec2_auto_deploy_test. The IP address is saved to the inventory.txt file within the Ansible folder and persisted to the workspace for use by the next job. Utilities tar and gzip are required to persist file to workspace.

Before pushing your changes to github, head over to the project page on CircleCI => Project Settings => Environment Variables and add the following environment variables:

  • AWS_SECRET_ACCESS_KEY
  • AWS_DEFAULT_REGION
  • AWS_ACCESS_KEY_ID

which can be obtained from your aws account.

The final job configuration on the config.yml file will be responsible for executing an Ansible playbook that uses ssh to access the server, pulls code from github and builds an updated docker image. See code below.

  deploy_server:
    docker:
      - image: python:3.11.6-alpine3.18
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints: ["9f:66:26:48:fd:251:62:5d:73:c4:d5:90:2b"]
      - attach_workspace:
          at: ~/
      - run:
          name: Install dependencies
          command: |
            apk add --update ansible openssh-client
            eval $(ssh-agent -s)
      - run:
          name: Deploy Server
          command: |
            cd ~/project/.circleci/ansible
            echo "Contents  of the inventory.txt -------"
            cat inventory.txt
            ansible-playbook -i inventory.txt deploy-server.yml

On your CircleCI project settings page, go to SSH Keys, add a new ssh key by copying and pasting the ssh key for your EC2 server. This will generate a key fingerprint. Copy the SSH key fingerprint and replace fingerprints on the deploy_server job.

Also update your workflow with the following code:

workflows:
  default:
    jobs:
      - test
      - setup_ansible:
          requires: [test]
      - deploy_server:
          requires: [setup_ansible]

Ansible Set-up

Ansible is a powerful open-source automation tool and configuration management system that streamlines and simplifies the automation of routine IT tasks and the management of complex systems. It provides a framework for IT professionals to define and execute automation tasks through human-readable playbooks.

With Ansible, you can automate the provisioning, configuration, deployment, and maintenance of servers, applications, and various infrastructure components. You should check out Ansible docs for more on this interesting automation tool.

Writing A Playbook

To automate Ansible, we use simple YAML files called playbooks to define automation tasks. The following shows the tree structure for the Ansible folder which contains the playbook to be executed.

โ”œโ”€โ”€ ansible
โ”‚   โ”œโ”€โ”€ ansible.cfg
โ”‚   โ”œโ”€โ”€ deploy-server.yml
โ”‚   โ”œโ”€โ”€ inventory.txt
โ”‚   โ””โ”€โ”€ roles
โ”‚       โ””โ”€โ”€ deploy
โ”‚           โ””โ”€โ”€ tasks
โ”‚               โ””โ”€โ”€ main.yml
โ””โ”€โ”€ config.yml

Here is a breakdown of each file and its function:

  • ansible.cfg: A configuration file used to customize the behaviour of Ansible. This allows you to override built-in configuration settings like SSH connection options, default inventory file, define remote user privileges and so on.
  • deploy-server.yml: This is an Ansible playbook. It defines a set of tasks or a task to be executed on remote hosts. In this project, I used roles to define playbook tasks.
  • inventory.txt: This serves as a critical component in Ansible for defining the hosts and groups of hosts that Ansible will manage. It is a configuration file that specifies the remote servers or systems where Ansible tasks and playbooks will be executed. In this file, IP addresses of the servers or machines to be managed by Ansible are listed.
  • roles: Roles are a way to organize and structure tasks, handlers, variables, and other content for better code reuse and maintainability. This contains the tasks to be executed in playbooks. These tasks are defined within the main.yml file.

Update deploy-server.yml with the following code:

---
- name: "Deploy Play"
  hosts: web
  user: ubuntu
  gather_facts: false
  vars:
    - ansible_python_interpreter: /usr/bin/python3
    - ansible_host_key_checking: false
    - ansible_stdout_callback: yaml
  roles:
    - deploy
  • name: This is a user-defined name for this playbook that makes it easy for us to understand its function.
  • hosts: This specifies the target hosts or hosts group on which the tasks in this playbook will be executed. In this case it is set to web. When you open the inventory.txt file, it contains a single line, [web]. This defines a host group called web and all IP Addresses listed under [web] are the hosts or remote machines we wish to manage with Ansible. To add an IP address to the inventory we are using aws-cli in the setup_ansible job to pull the IP Address of our EC2 server and dynamically update the inventory file.
  • user: This specifies the remote user that Ansible should use when connecting to the target hosts. In this case it is ubuntu. You may need to change that if your ssh user is different. gather_facts: This tells Ansible whether to collect facts about the target hosts, such as hardware details, network configuration, and more. Here it is set to false to improve playbook execution speed.
  • vars: This defines variables to be used in the playbook. ansible_python_interpreter specifies the python intepreter Ansible should use when running jobs on the host. ansible_host_key_checking is set to false to disable host key checking. Host key checking is a prompt that comes up on your command line when you are executing ssh login to a server with a key for the first time. This is a security measure to ensure the remote host identity.
  • roles: This specifies which roles should be included in this job. In this case, it's a role named deploy. When the playbook is executed, Ansible will look for a directory named "deploy" within the roles path and execute the tasks and configurations defined in that role. Let's write the tasks for the deploy role.

Copy the code below to main.yml within the deploy/tasks role folder.

---
- name: Perform Git Pull
  become: false
  git:
    repo: [email protected]:AbuchiKings/aws_ec2_auto_deploy.git # Replace with github repo you prefer
    dest: ~/apps/aws_ec2_auto_deploy
    accept_hostkey: yes
    clone: false
    version: main # Replace with the desired branch or tag

- name: Run Docker build
  become: true
  shell: 
    chdir: apps/aws_ec2_auto_deploy
    cmd: make "{{ item }}"
  loop: 
      - down
      - up-d

The first task uses Ansible's git module to perform a git pull from the project's git repository.

  • become is a boolean field that tells Ansible to perform a task as a root user.
  • dest is the destination or directory where the fetched items should be added.
  • To prevent a fresh cloning of the repo, clone is set to false.
  • version defines the branch to pull from.

In the second task, the make command is used on two different arguments defined inside a Makefile in the project root folder:

  • down which maps to docker-compose -f docker-compose.yml -f docker-compose.prod.yml down takes down the current running container.
  • up-d which maps to docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build starts up the container with a new build in a detached mode.

Finally, commit your changes and push to remote repository.

Conclusion

This is by no means means an exhaustive tutorial on CircleCi and Ansible. There are a wide range of automation tasks you could perform with Ansible. Nonetheless, I hope you find this helpful. Thank you for your time.

...



๐Ÿ“Œ How To Automate Your Deployments To AWS EC2 Using CircleCI And Ansible


๐Ÿ“ˆ 98.79 Punkte

๐Ÿ“Œ Automate container and pod deployments with Podman and Ansible


๐Ÿ“ˆ 46.6 Punkte

๐Ÿ“Œ Build and Deploy a ReactJS App to AWS EC2 with Docker, NGINX, and Automate with GitHub Actions.


๐Ÿ“ˆ 42.29 Punkte

๐Ÿ“Œ Installing Ansible on AWS EC2 Instance


๐Ÿ“ˆ 37.94 Punkte

๐Ÿ“Œ Fast Deployments of Microservices Using Ansible and Kubernetes


๐Ÿ“ˆ 36.57 Punkte

๐Ÿ“Œ How to Select the Right EC2 Instance โ€“ A Guide to EC2 Instances and Their Capabilities


๐Ÿ“ˆ 35.99 Punkte

๐Ÿ“Œ EC2 Configuration using Ansible & GitHub Actions


๐Ÿ“ˆ 35.31 Punkte

๐Ÿ“Œ CircleCI acquires Vamp to help engineering teams automate software delivery at scale


๐Ÿ“ˆ 33.54 Punkte

๐Ÿ“Œ GPN19 - Automate your virtual server deployments


๐Ÿ“ˆ 33.53 Punkte

๐Ÿ“Œ Automate Your Kubernetes Deployments With Helm


๐Ÿ“ˆ 33.53 Punkte

๐Ÿ“Œ AWS unveils sixth generation of Amazon EC2 instances powered by AWS Graviton2 processors


๐Ÿ“ˆ 32.99 Punkte

๐Ÿ“Œ Automate every Industry using Power Automate with Roma Gupta | #LessCodeMorePower


๐Ÿ“ˆ 32.28 Punkte

๐Ÿ“Œ How to Create EC2 Instance (Ubuntu 22.04) on AWS and Connect Via SSH using PEM


๐Ÿ“ˆ 32.26 Punkte

๐Ÿ“Œ Automate Your SSO With Ansible and Keycloak


๐Ÿ“ˆ 31.95 Punkte

๐Ÿ“Œ Automate and Manage AWS Outposts Capacity Across Multi-Account AWS Setup


๐Ÿ“ˆ 31.32 Punkte

๐Ÿ“Œ How to Secure Your Web Server with Continuous Integration Using NGINX and CircleCI


๐Ÿ“ˆ 30.97 Punkte

๐Ÿ“Œ Optimizing Your AWS EC2 Windows Instance: A Comprehensive Guide to Extending Root Volumes and Adding Extra Storage


๐Ÿ“ˆ 30.58 Punkte

๐Ÿ“Œ How To Understand and Choose Your First EC2 Instance on AWS


๐Ÿ“ˆ 30.58 Punkte

๐Ÿ“Œ Detailed instructions to setup hadoop cluster using Hortonworks distribution on AWS EC2 instances


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ Migrating MySQL from EC2 to RDS using AWS DMS


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ How to Set up VPN on AWS EC2 Using Ubuntu


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ Access EC2 Instances Privately Using AWS Systems Manager


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ Trace EC2 Console Activity in Multiple Code Format Using AWS Console-to-Code


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ Logitech: SSRF allows reading AWS EC2 metadata using "readapi" variable in Streamlabs Cloudbot


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ IPV6 Migration Guide for Developers using AWS EC2 -- A Primer


๐Ÿ“ˆ 30.36 Punkte

๐Ÿ“Œ Ansible tutorial - Automate your systems


๐Ÿ“ˆ 30.05 Punkte

๐Ÿ“Œ Ansible Playbook writing for beginners: Automate your morning routine


๐Ÿ“ˆ 30.05 Punkte

๐Ÿ“Œ CircleCI updates platform, partners with AWS, HashiCorp, and others


๐Ÿ“ˆ 29.95 Punkte

๐Ÿ“Œ Three resources to improve your understanding of AWS security and harden cloud deployments


๐Ÿ“ˆ 29.94 Punkte

๐Ÿ“Œ Automate Always On availability group deployments with SQL Virtual Machine resource provider


๐Ÿ“ˆ 29.87 Punkte

๐Ÿ“Œ Loft Labs integrates with Argo CD to automate deployments with virtual clusters


๐Ÿ“ˆ 29.87 Punkte

๐Ÿ“Œ Download Power Automate Desktop for Windows 10 to automate tasks and processes at no additional cost


๐Ÿ“ˆ 28.84 Punkte

๐Ÿ“Œ You can now use the Automate tab in Excel for Windows and Mac to automate tasks


๐Ÿ“ˆ 28.84 Punkte

๐Ÿ“Œ Pros, Cons, and traps of EC2 Instance Start and Stop Schedules with AWS Lambda


๐Ÿ“ˆ 28.82 Punkte











matomo