Traditionally, we rely on print or log statements in our code to debug or understand the system’s current status. But does that really solve the problem?
AWS offers tools like CloudWatch Insights, dashboards, and alarms to meet basic monitoring needs. However, it doesn’t allow us to visualise the time taken for a particular part of the function or where there are bottlenecks. We could use AWS X-Ray. But what if your application interacts with services owned by other teams — possibly in different AWS accounts or entirely different environments?
What if you could see the entire path (or trace) of events that happened when a user clicked on a website, to all the backend services that were triggered as part of it? This is where OpenTelemetry comes to the rescue, ushering in the concept of observability.
Observability is the ability to see the current state or context of the system using the data it generates. We would be able to see all the incoming and outgoing requests in our application.
What is OpenTelemetry (OTel)?
OpenTelemetry is an open-source observability framework for instrumenting, gathering and exporting telemetry data like traces, metrics and logs from our application.
OTel is a vendor-agnostic framework, which means that we can export the logs to various observability tools like Honeycomb, New Relic, Grafana, Datadog, etc.
OpenTelemetry is a CNCF (Cloud Native Computing Foundation) project and is the result of a merger of two projects, OpenCensus and OpenTracing, unified to create a global standard for telemetry
AWS Distro For OpenTelemetry (ADOT)
ADOT is a secure, production-ready, AWS-supported distribution for OTel provided by AWS. It abstracts the manual instrumentation required for OpenTelemetry. Using Auto instrumentation in ADOT, without writing any extra lines of code, we can publish basic telemetry data to any Observability tool. It also collects and includes metadata from our AWS Lambda like function name and version.
In AWS Lambda, we can just add an ADOT Lambda Layer to our function and just by adding a few environment variables, we would be able to collect telemetry data and export it to any observability tool.
Now let’s understand more about OpenTelemetry and its core concepts.
Core Concepts of OpenTelemetry
Traces
Traces provide end-to-end visibility into request flows across our systems. Think of a trace as the journey of a single request as it travels through your application ecosystem. Each step taken by it is called a Span.
Trace Contains Unique identifiers known as trace ID, and each Span also has a unique ID known as Span ID.
A Span contains metadata like timestamp, status, faas (Function As a Service) name and versions.
Key Moments, like exceptions in a span, are captured as Events.
We can add custom Key-Value Pairs to span, which are application-related, and it is called span attributes.
If any external calls are made, this Trace ID is propagated across systems, and this is called as context propagation.
Metrics
Metrics are numerical measurements collected to monitor the system. Counter is the most common type of metric. For example, you can have a metric for the number of users who clicked on a button or the number of products that were purchased.
Logs
Logging mechanisms are available in all programming languages. OpenTelemetry helps us to correlate logs with Traces and Metrics. And, a centralised location to observe logs from various systems.
Instrumentation
It is the process of adding observability to our application code. There are two types available one is Manual instrumentation and another is Auto Instrumentation.
In Auto Instrumentation, minimal code changes are required, and it enables us to easily set up observability for our application, whereas in manual instrumentation, we need to write custom code to set up collection and transportation of telemetry data.
OpenTelemetry Collectors
OpenTelmetry collectors can be run as an agent (sidecar) alongside our application or as a standalone service.
Receiver: Ingests data from sources
Processor: Transforms, filters telemetry data
Exporters: Send data to various Observability tools like AWS X-Ray, Honeycomb, New Relic, Etc.
In AWS Lambda, this can be run as a Lambda extension just by adding the ADOT Lambda Layer.
Now that we’ve covered the basics of OpenTelemetry, the next part will walk through implementing it in AWS Lambda and exporting telemetry data to an observability tool.
Until then, Happy Learning!