Lädt...


🔧 EKS Observability Infrastructure as Code - AWS Observability Accelerator


Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to

Observability is an essential part of any cloud-native environment. When it comes to Amazon EKS clusters, observability is an even more critical factor. This blog post will explore how you can provision an observability landscape to monitor and manage your EKS clusters using AWS Observability Accelerator for Terraform.

The AWS Observability accelerator for Terraform is a powerful terraform module that enables developers to quickly and easily set up observability solutions for their Amazon Elastic Container Service for Kubernetes (EKS) clusters using Amazon Web Services (AWS) observability services. This tool includes a core module that helps you configure your cluster with the AWS Distro for OpenTelemetry (ADOT) Operator for EKS, Amazon Managed Service for Prometheus, and Amazon Managed Grafana.

In addition to the core module, the AWS Observability accelerator also includes a set of workload modules that provide curated ADOT collector configurations, Grafana dashboards, Prometheus recording rules, and alerts. These modules allow you to leverage the power of these tools to monitor and troubleshoot your applications and infrastructure in real-time.

%[https://github.com/aws-observability/terraform-aws-observability-accelerator]

In this post, we will set up the EKS cluster using terraform. Then we will provision AWS Managed service for Prometheus, AWS Managed service for Grafana, Install AWS Distro for Open Telemetry in Cluster to scrape the metrics and necessary recording rules, and dashboards for Prometheus and Grafana - all these in only a few lines of Infrastructure as Code.

Let’s start with the setup of initializing the EKS cluster using the EKS Blueprints terraform module.

  1. Initialize the EKS cluster using the EKS Blueprints terraform module. EKS blueprints are modular constructs available in CDK and as terraform modules that can provision fully functional production-ready kubernetes cluster and modular add-ons such as service mesh, autoscalers, networking, storage, gitops, etc.

    module "eks_blueprints" {
      source          = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.19.0"
      cluster_name    = var.cluster_name
      cluster_version = "1.20"
    
      vpc_id             = var.vpc_id
      private_subnet_ids = var.private_subnet_ids
      public_subnet_ids  = var.public_subnet_ids
    
      managed_node_groups = {
        t3_small = {
          node_group_name = "node-group-t3-small"
          instance_types  = ["t3.small"]
          subnet_ids      = var.private_subnet_ids
          min_size        = 1
          max_size        = 2
          desired_size    = 1
        }
      }
    
      tags = var.tags
    }
    
  2. You can then specify which observability services you want to enable, such as the ADOT operator, Amazon Managed Prometheus, and Amazon Managed Grafana.

    module "eks_observability_accelerator" {
      source = "github.com/aws-observability/terraform-aws-observability-accelerator?ref=v1.5.0"
    
      aws_region     = var.aws_region
      eks_cluster_id = module.eks_blueprints.eks_cluster_id
    
        enable_amazon_eks_adot    = true
      enable_managed_prometheus = true
      enable_alertmanager       = true
        enable_managed_grafana    = true
    
      tags = var.tags
    }
    
  3. You can also specify your own instance IDs if you want to reuse existing managed services or disable the creation of managed services altogether.

    module "eks_observability_accelerator" {
      source = "github.com/aws-observability/terraform-aws-observability-accelerator?ref=v1.5.0"
    
      aws_region     = var.aws_region
      eks_cluster_id = module.eks_blueprints.eks_cluster_id
    
        enable_amazon_eks_adot    = true
      enable_managed_prometheus = true
      enable_alertmanager       = true
    
        # Reuse existing setup by disabling the creation of managed grafana service
        # and pass the existing service workspace id
      enable_managed_grafana       = false
      managed_grafana_workspace_id = var.managed_grafana_workspace_id
      grafana_api_key              = var.grafana_api_key
    
      tags = var.tags
    }
    
  4. Set up the default Grafana dashboards, alerting rules, and metrics scraping configuration from AWS Observability Accelerator for Terraform modules. You can customize them as needed.

    module "workloads_infra" {
      source = "github.com/aws-observability/terraform-aws-observability-accelerator/modules/workloads/infra"
    
      eks_cluster_id = module.eks_observability_accelerator.eks_cluster_id
    
      dashboards_folder_id            = module.eks_observability_accelerator.grafana_dashboards_folder_id
      managed_prometheus_workspace_id = module.eks_observability_accelerator.managed_prometheus_workspace_id
    
      managed_prometheus_workspace_endpoint = module.eks_observability_accelerator.managed_prometheus_workspace_endpoint
      managed_prometheus_workspace_region   = module.eks_observability_accelerator.managed_prometheus_workspace_region
    
      # optional, defaults to 60s interval and 15s timeout
      prometheus_config = {
        global_scrape_interval = "60s"
        global_scrape_timeout  = "15s"
      }
    
      depends_on = [
        module.eks_observability_accelerator
      ]
    }
    
  5. Once you have included the AWS Observability Accelerator in your Terraform configuration and specified which services you want to enable, you can use Terraform to provision the observability landscape for your EKS cluster.

    terraform plan -var-file env/dev.tfvars
    terraform apply -var-file env/dev.tfvars
    

The complete Observability Landscape has been set up for you - including ADOT (AWS Distro for Open Telemetry collectors), AWS Managed Services for Prometheus (Prometheus endpoints where ADOT writes the metrics), AWS Managed Services for Grafana (where it can query from Prometheus data sources to build dashboards and display alerts).

You can view the Prometheus Endpoint URL, alerting, and recording rules in the AWS console.

Navigate the Grafana Dashboard in the AWS Managed Services for the Grafana workspace, and configure the user to view/edit the dashboards. The necessary dashboards are already created for the user to view.

  • Kubernetes - Cluster

  • Kubernetes - Namespace(Workloads)

  • Kubernetes - Node (Pods)

  • Kubernetes - Workload

  • Kubernetes - Kubelet

  • Node Exporter - Nodes

You can navigate to the individual dashboard and view the charts based on the metrics scraped from the infrastructure and workloads.

Conclusion

In conclusion, the AWS Observability Accelerator for Terraform is a powerful terraform module that allows you to quickly and easily set up observability solutions for your EKS clusters. It includes a core module and a set of workload modules that provide curated configurations, dashboards, recording rules, and alerts to help you monitor and troubleshoot your applications and infrastructure in real-time. Using the AWS Observability Accelerator, you can leverage the power of tools like Prometheus and Grafana to visualize and analyze your metrics and logs, enabling you to optimize and improve the performance and reliability of your cloud-native environments.

I'm Siva - working as Director, DevOps & Principal Architect at Computer Enterprises Inc from Orlando. I'm an AWS Community builder. I will write a lot about Cloud, Containers, IoT, and DevOps. If you are interested, please follow me @@ksivamuthu on Twitter or check out my blogs at sivamuthukumar.com

...

🔧 Amazon EKS and Amazon EKS Distro now supports Kubernetes version 1.31


📈 32.52 Punkte
🔧 Programmierung

🔧 Bootstrap Complete Amazon EKS Clusters with EKS Blueprints for Terraform


📈 32.52 Punkte
🔧 Programmierung

🔧 Bootstrap Complete Amazon EKS Clusters with EKS Blueprints for Terraform


📈 32.52 Punkte
🔧 Programmierung

🔧 New-Amazon EKS and Amazon EKS Distro now support Kubernetes version 1.26


📈 32.52 Punkte
🔧 Programmierung

🔧 Simplify your EKS Cluster Observability with Prometheus & Grafana


📈 27.88 Punkte
🔧 Programmierung

🔧 A Step-by-Step Guide to Easily Deploying EKS Infrastructure and Applications Using Terraform


📈 24.62 Punkte
🔧 Programmierung

🔧 From Observability To Architectural Observability — Shifting Left for Resiliency


📈 23.24 Punkte
🔧 Programmierung

🔧 Revolutionizing Observability: How AI-Driven Observability Unlocks a New Era of Efficiency


📈 23.24 Punkte
🔧 Programmierung

📰 Mezmo Observability Pipeline empowers teams to control their observability data


📈 23.24 Punkte
📰 IT Security Nachrichten

🔧 From Observability to Architectural Observability—Shifting Left for Resiliency


📈 23.24 Punkte
🔧 Programmierung

🔧 Infrastructure as Code (IaC) in DevOps: The Key to Streamlined DevOps Infrastructure Management


📈 20.44 Punkte
🔧 Programmierung

🔧 Demystifying Infrastructure as Code: Provisioning Infrastructure with Terraform


📈 20.44 Punkte
🔧 Programmierung

🔧 Infrastructure as Code: The Evolution of Cloud Infrastructure Management


📈 20.44 Punkte
🔧 Programmierung

🔧 Master API Observability: Enhancing Reliability and Performance in Your Digital Infrastructure


📈 19.98 Punkte
🔧 Programmierung

🔧 OpenTelemetry: Unifying Application and Infrastructure Observability


📈 19.98 Punkte
🔧 Programmierung

📰 Datadog President Amit Agarwal on Trends in Infrastructure Observability


📈 19.98 Punkte
📰 IT Nachrichten

📰 SolarWinds’ Observability offers visibility into hybrid cloud infrastructure


📈 19.98 Punkte
📰 IT Security Nachrichten

📰 Splunk’s new APM solution unlocks real-time observability for cloud infrastructure


📈 19.98 Punkte
📰 IT Security Nachrichten

📰 Low-code: An Accelerator for Digital Transformation


📈 19.18 Punkte
📰 IT Security Nachrichten

🕵️ SAP TREX/Business Warehouse Accelerator Code Injection privilege escalation


📈 19.18 Punkte
🕵️ Sicherheitslücken

📰 Public cloud IT infrastructure spending exceeds that for non-cloud IT infrastructure


📈 16.73 Punkte
📰 IT Security Nachrichten

🔧 Containerized Application Deployment on Amazon EKS with Jenkins Pipeline


📈 16.26 Punkte
🔧 Programmierung

🔧 EKS Setup using EKSCTL


📈 16.26 Punkte
🔧 Programmierung

🔧 Back2Basics: Monitoring Workloads on Amazon EKS


📈 16.26 Punkte
🔧 Programmierung

🔧 How to Deploy Wordpress to Amazon EKS using Helm


📈 16.26 Punkte
🔧 Programmierung

🔧 Expose EKS tekton pipeline dashboard with ssl enabled


📈 16.26 Punkte
🔧 Programmierung

🔧 Karpenter on EKS Fargate


📈 16.26 Punkte
🔧 Programmierung

🔧 Best Practices for Seamless EKS Cluster Upgrades with Fargate: A Hands-On Guide


📈 16.26 Punkte
🔧 Programmierung

matomo