Cookie Consent by Free Privacy Policy Generator 📌 OTEL Tracing demo with AWS X-Ray and Grafana


✅ OTEL Tracing demo with AWS X-Ray and Grafana


💡 Newskategorie: Programmierung
🔗 Quelle: dev.to

Introduction

Hello 👋, In this post we'll see about sending traces from a demo app to AWS X-Ray via the ADOT(AWS Distro for OpenTelemetry) collector. We would then visualize this on Grafana. Note that we'd deploy the workloads on a kubernetes cluster.

Here is a picture of what we are trying to accomplish:
Block diagram for the lab

Alright, let's get started!!!

Namespace

We shall deploy the workloads on a separate namespace. Let's create one.

kubectl create ns adot-traces-demo

Credentials

Store the AWS credentials as a kubernetes secret.

kubectl create secret generic aws-credentials \
    --from-literal=AWS_ACCESS_KEY_ID=<access-key-id> \
    --from-literal=AWS_SECRET_ACCESS_KEY=<aws-secret-access-key> \
    -n adot-traces-demo

ADOT Config

Set the ADOT config in a file.

$ cat adot-config.yaml
exporters:
  awsxray:
    region: ap-south-2
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
service:
  pipelines:
    traces:
      exporters:
        - awsxray
      receivers:
        - otlp

And create a config map with this file.

kubectl create configmap adot-config --from-file=adot-config.yaml -n adot-traces-demo 

ADOT Deployment

Setup deployment spec in a file, that injects the secret we created earlier as environment variables and the configmap as a volume.

$ cat adot-deploy.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: adot-collector
  name: adot-collector
spec:
  replicas: 1
  selector:
    matchLabels:
      app: adot-collector
  template:
    metadata:
      labels:
        app: adot-collector
    spec:
      containers:
        - args:
            - '--config=/etc/adot-config.yaml'
          envFrom:
            - secretRef:
                name: aws-credentials
          image: public.ecr.aws/aws-observability/aws-otel-collector:latest
          name: adot-collector
          volumeMounts:
            - mountPath: /etc/adot-config.yaml
              name: config-volume
              subPath: adot-config.yaml
      volumes:
        - configMap:
            name: adot-config
          name: config-volume

Create the deployment.

kubectl create -f adot-deploy.yaml -n adot-traces-demo

The pod in the deployment should be running.

$ kubectl get po -n adot-traces-demo
NAME                              READY   STATUS    RESTARTS   AGE
adot-collector-7cbf849b89-b4bkl   1/1     Running   0          3m26s

ADOT Service

We can expose the ADOT deployment with a service spec that exposes the grpc port 4317 as follows.

$ cat adot-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: adot-collector-service
spec:
  selector:
    app: adot-collector
  ports:
    - protocol: TCP
      port: 4317
      targetPort: 4317

We can now create the service.

kubectl create -f adot-svc.yaml -n adot-traces-demo

The endpoint IP should match with the pod IP.

$ kubectl get ep -n adot-traces-demo
NAME                     ENDPOINTS         AGE
adot-collector-service   10.1.3.187:4317   22s

$ kubectl get po -n adot-traces-demo -o wide
NAME                              READY   STATUS    RESTARTS   AGE     IP           NODE             NOMINATED NODE   READINESS GATES
adot-collector-7cbf849b89-b4bkl   1/1     Running   0          7m11s   10.1.3.187   docker-desktop   <none>           <none>

Demo app

We can now deploy the sample demo app which can send traces to ADOT collector, with the following manifest.

$ cat k6-tracing-deploy.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: xk6-tracing
spec:
  replicas: 1
  selector:
    matchLabels:
      app: xk6-tracing
  template:
    metadata:
      labels:
        app: xk6-tracing
    spec:
      containers:
        - env:
            - name: ENDPOINT
              value: adot-collector-service:4317
          image: ghcr.io/grafana/xk6-client-tracing:v0.0.2
          name: xk6-tracing

Let's create the deployment.

kubectl create -f k6-tracing-deploy.yaml -n adot-traces-demo

Both the ADOT collector and k6-tracing pods should now be running.

$ kubectl get po -n adot-traces-demo   
NAME                              READY   STATUS    RESTARTS   AGE
adot-collector-7cbf849b89-b4bkl   1/1     Running   0          14m
xk6-tracing-69b48fcfd9-bjzbd      1/1     Running   0          24s

X-Ray

We can now headover to AWS X-Ray, in ap-south-2 region that we mentioned in the adot-config.
Traces in AWS X-Ray

The nodes(services) shown in the screenshot belong to our demo application. We could filter for traces that passes through a particular service name for ex. article service, like below.
Traces

If we click on a single trace we should be able to see a complete service map for that trace, that shows all the services that trace traverses.
Service map for a trace

If we go a further down on this we should be able to see the details for segments/spans with in this trace.
Spans in trace

Grafana

So we far we were able to see the traces in AWS X-Ray, we can do a similar exercise on Grafana. I am using a Grafana Cloud Free subscription for this lab.

Go to Connections, Add a new connection and search for X-Ray and install it.
Install X-Ray

You can then go to datasources, add a new X-Ray datasource with the access key id, secret access key, and default region(I have chosen ap-south-2 which matches with adot config).

Add X-Ray datasource

All good, we can try adding a new panel, go to dashboards > new dashboard and a new visualization with table as panel type and a sample query for ex. service(id(name: "article-service" ))
Traces in tabular format

We can click on one of the traces we should take us to the explore view where we can see the node graph(service map)
Node graph for trace in grafana

We should also see the trace explorer that shows the individual spans.
Trace explorer in Grafana

Okay so we reached this far, that was some fun exploring traces on AWS and Grafana with Open Telemetry. Thank you for reading !!!

CleanUp

Just delete the namespace with kubectl delete ns adot-traces-demo and it should remove the workloads we deployed from kubernetes and stop sending any new data to the cloud.

...

✅ Logging demo with OTEL Collector, CloudWatch and Grafana


📈 55.97 Punkte

✅ Grafana OTEL App Observability Demo


📈 54.44 Punkte

✅ Grafana Labs releases Grafana 11, Loki 3.0, and a new open source project called Grafana Alloy


📈 44.06 Punkte

✅ Mehrere Probleme in grafana-piechart-panel, grafana-status-panel und grafana (SUSE)


📈 42.53 Punkte

✅ Grafana up to 6.3.6 on Red Hat /etc/grafana/grafana.ini information disclosure


📈 42.53 Punkte

✅ otel-tui: A TUI Tool for Viewing OpenTelemetry Traces


📈 31.26 Punkte

✅ Get started quickly with AWS Trainium and AWS Inferentia using AWS Neuron DLAMI and AWS Neuron DLC


📈 30.08 Punkte

✅ The open source projects Grafana Phlare and Pyroscope will be merged under the name Grafana Pyroscope.


📈 29.89 Punkte

✅ Low CVE-2019-15635: Grafana Grafana


📈 28.35 Punkte

✅ Grafana On Docker | Simple Beginner Tutorial On Grafana


📈 28.35 Punkte

✅ Low CVE-2020-12052: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2020-13430: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2018-18624: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2018-18623: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2018-18625: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2020-11110: Grafana Grafana


📈 28.35 Punkte

✅ Low CVE-2019-19499: Grafana Grafana


📈 28.35 Punkte

✅ Grafana up to 6.7.3 /var/lib/grafana Credentials cleartext


📈 28.35 Punkte

✅ Low CVE-2020-24303: Grafana Grafana


📈 28.35 Punkte

✅ Security: Mehrere Probleme in grafana und system-user-grafana (SUSE)


📈 28.35 Punkte

✅ CVE-2023-5123 | Grafana grafana-json-datasource prior 1.3.11 path traversal


📈 28.35 Punkte

✅ heise+ | Microservices: Distributed Tracing mit Grafana Tempo im Test


📈 25.74 Punkte

✅ Live Demo of Azure Managed Grafana


📈 23.18 Punkte

✅ Amazon Grafana demo with EKS


📈 23.18 Punkte

✅ Privacy-Friendly Contact Tracing – How Do the Protocols Work in Corona Tracing Apps?


📈 23.13 Punkte

✅ EKS cluster Monitoring for AWS Fargate with Prometheus and managed Grafana


📈 22.46 Punkte

✅ Guide to Installing and Configuring Grafana on AWS EC2 with MySQL Database Integration


📈 22.46 Punkte

✅ Visualizing Amazon S3 Storage Usage with Grafana and AWS SDK for Python (Boto3)


📈 22.46 Punkte

✅ Test Observability for AWS Lambda with Grafana Tempo and OpenTelemetry Layers


📈 22.46 Punkte

✅ DevSecOps with AWS – ChatOps with AWS and AWS Developer Tools – Part 1


📈 21.79 Punkte

✅ AWS Resume Challenge using Pulumi, Golang, AWS S3 and AWS CloudFront


📈 21.79 Punkte

✅ Simple & Secure Access to Grafana on AWS


📈 20.93 Punkte

✅ Building an AWS Lambda Telemetry API extension for direct logging to Grafana Loki


📈 20.93 Punkte

✅ Stack Grafana deployed on AWS ECS -Part 1 - Create a Cluster


📈 20.93 Punkte











matomo

Datei nicht gefunden!