Introduction: Rethinking API Gateway
If you ask many developers what Amazon API Gateway is, they’ll often say,
“It’s a way to trigger Lambda functions with HTTP requests.”
While that’s a common use case, this view undersells the service dramatically.
In reality, API Gateway is a complete API management platform. It’s your first and most critical line of defense, capable of handling validation, transformation, authentication, throttling, and observability before your Lambda function even runs.
It shifts the burden of common API concerns from your code to a managed, scalable layer.
To prove this, we’ll build TrailLog — a serverless API for logging hiking trips.
We won’t just connect endpoints; we’ll use API Gateway’s full potential to create a robust, secure, and observable API.
By the end of this guide, you will learn how to:
Validate & Transform: Enforce data contracts at the gateway, keeping Lambda code clean.
Secure Endpoints: Implement a custom Lambda authorizer for token-based access control.
Achieve Observability: Gain deep insights into API performance with structured CloudWatch logs.
Orchestrate a Governed API: Deploy a production-ready API with full visibility and control.
The Blueprint: TrailLog System Architecture
Our fully serverless architecture is designed for scalability and simplicity.
Each AWS service plays a clear role in this ecosystem.
| Component | Responsibility |
|---|---|
| Amazon API Gateway (REST API) | The unified entry point — handles routing, validation, auth, and logging |
| AWS Lambda | Executes backend logic for creating and retrieving trips |
| Amazon DynamoDB | Stores all hiking data in a serverless NoSQL table |
| Amazon CloudWatch | Centralizes logs and metrics for monitoring and debugging |
This stack scales automatically with traffic, without managing or patching servers.
Architecture Diagram
Client application sends incoming HTTPS REST requests to Amazon API Gateway, which triggers AWS Lambda functions for GET, POST, and DELETE operations. Lambda interacts with Amazon DynamoDB for data storage and sends logs to Amazon CloudWatch.
Foundation: Setting Up Permissions with IAM
All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role,
configured via an inline policy that grants access to DynamoDB and CloudWatch logs.
IAM Role: lambda-traillog-role
All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role.
This role is granted access to the DynamoDB table TrailLog through the following inline policy.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DDBWriteTrailLog",
"Effect": "Allow",
"Action": [
"dynamodb:PutItem"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
},
{
"Sid": "DDBReadTrailLog",
"Effect": "Allow",
"Action": [
"dynamodb:GetItem",
"dynamodb:Scan",
"dynamodb:Query"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
},
{
"Sid": "DDBDescribeForChecks",
"Effect": "Allow",
"Action": [
"dynamodb:DescribeTable"
],
"Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
}
]
}
This role is attached to all Lambda functions in the project.
Deep Dive 1: The POST /trips Endpoint — Validation & Transformation
The first endpoint demonstrates API Gateway’s ability to offload validation and data transformation
before invoking the backend logic.
DynamoDB Table: TrailLog
| Property | Value |
|---|---|
| Partition Key | tripId (String) |
| Billing Mode | Pay-per-request |
Lambda Function: create-trip (Python 3.12)
This Lambda function remains simple because API Gateway enforces all validation upfront.
import json, os, uuid, boto3
from datetime import datetime
from decimal import Decimal, InvalidOperation
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])
def lambda_handler(event, context):
# API Gateway has already validated input; focus on core logic.
trip_id = str(uuid.uuid4())
item = {
"tripId": trip_id,
"userId": event["userId"],
"trailName": event["trailName"],
"date": event["date"],
"distanceKm": event["distanceKm"],
"durationMin": event["durationMin"],
"elevationGainM": event["elevationGainM"],
"notes": event.get("notes", ""),
"createdAt": datetime.utcnow().isoformat() + "Z"
}
table.put_item(Item=item)
return {
"statusCode": 201,
"headers": {"Content-Type": "application/json"},
"body": json.dumps({"ok": True, "tripId": trip_id})
}
The API Gateway Configuration: Non-Proxy Integration
Disabling Lambda Proxy Integration gives us control over request handling and validation.
| Setting | Value |
|---|---|
| Integration Type | Lambda Function |
| Proxy Integration | Disabled |
| Validator | Body & Params |
| Model | TripCreate |
Request Model: TripCreate
{
"type": "object",
"required": ["userId", "trailName", "date", "distanceKm", "durationMin", "elevationGainM"],
"properties": {
"userId": {"type": "string"},
"trailName": {"type": "string"},
"date": {"type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"},
"distanceKm": {"type": "number"},
"durationMin": {"type": "integer"},
"elevationGainM": {"type": "integer"},
"notes": {"type": "string"}
}
}
Now, any malformed input (for example, an invalid date format or incorrect type)
is rejected at the gateway, saving Lambda invocations and improving reliability.
Successfully created a new hiking trip via the POST /trips endpoint, returning a 201 status and generated tripId.
Deep Dive 2: The GET /trips/{tripId} Endpoint — Simple Proxy Integration
For simple reads, we use Lambda Proxy Integration to pass the entire request context directly to Lambda.
Lambda Function: get-trip
This function extracts tripId from the path and fetches the corresponding item from DynamoDB.
import json, os, boto3, decimal
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])
def _to_native(x):
if isinstance(x, decimal.Decimal):
return int(x) if x % 1 == 0 else float(x)
return x
def lambda_handler(event, context):
trip_id = (event.get("pathParameters") or {}).get("tripId")
if not trip_id:
return {"statusCode": 400, "body": "Missing tripId"}
res = table.get_item(Key={"tripId": trip_id})
item = res.get("Item")
if not item:
return {"statusCode": 404, "body": "Not found"}
return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps(_to_native(item))
}
Test Request
curl https://<api-id>.execute-api.us-east-1.amazonaws.com/dev/trips/<tripId>
Fetched detailed hiking trip data successfully using the GET /trips/{tripId} endpoint.
Deep Dive 3: Security and Observability — /secure/admin/stats
This administrative endpoint combines two advanced API Gateway features:
custom token authorization and centralized CloudWatch observability.
1. Securing the Route with a Lambda Authorizer
We use a custom Lambda authorizer to protect the /secure/admin/stats route.
Authorizer Function: lambda-authorizer
import json
def lambda_handler(event, context):
DEMO_TOKEN = "Bearer trail123"
token = event.get("authorizationToken", "").strip()
if token == DEMO_TOKEN:
effect, role = "Allow", "admin"
else:
effect, role = "Deny", "guest"
return {
"principalId": "trail-admin",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": effect,
"Resource": event["methodArn"]
}]
},
"context": {"role": role}
}
Any request without X-Auth-Token: Bearer trail123 will receive a 403 Forbidden response.
2. Stats Aggregation Function: admin-stats
This function aggregates trip statistics across all records.
It’s invoked only if the authorizer grants access.
import json, os, boto3, decimal
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])
def _n(x):
if isinstance(x, decimal.Decimal):
return int(x) if x % 1 == 0 else float(x)
return x
def lambda_handler(event, context):
scan = table.scan(ProjectionExpression="tripId, distanceKm, durationMin, elevationGainM")
items = scan.get("Items", [])
total = len(items)
km = sum(_n(i.get("distanceKm", 0)) for i in items)
mins = sum(_n(i.get("durationMin", 0)) for i in items)
gain = sum(_n(i.get("elevationGainM", 0)) for i in items)
stats = {
"totalTrips": total,
"totalKm": round(float(km), 2),
"totalHours": round(float(mins) / 60.0, 1),
"totalElevationM": int(gain)
}
return {
"statusCode": 200,
"headers": {"Content-Type": "application/json"},
"body": json.dumps(stats)
}
Test Request
curl -H "Authorization: Bearer trail123" "$BASE/secure/admin/stats"
Access denied without a valid token — API Gateway blocking unauthorized requests using the custom Lambda authorizer.
Authorized admin access to /secure/admin/stats returning aggregated trail statistics.
The Crown Jewel: Deep Observability with CloudWatch
To gain complete visibility into API behavior, enable both Execution Logs and Access Logs in Amazon API Gateway.
These logs provide the foundation for debugging, performance monitoring, and request analytics.
Create a Log Group for Access Logs
Before enabling access logging, create a dedicated CloudWatch Log Group where API Gateway will push structured log entries.
This ensures all API requests are tracked consistently and stored centrally.
Steps:
- Open CloudWatch Console → Log groups → Create log group.
- Enter the name:
/apigw/traillog/dev
- (Optional) Set the retention period to 7, 30, or 90 days depending on your monitoring needs.
- Note down the log group ARN — you’ll use it in your API Gateway stage configuration:
arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev
- Ensure API Gateway has permission to write logs by attaching this managed policy to its execution role:
arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
CloudWatch log group /apigw/traillog/dev created to store structured access logs from API Gateway.
1. Execution Logs
Execution logs trace the internal request flow through API Gateway —
from input validation and mapping templates to Lambda invocation and response transformation.
They’re your primary tool for debugging and tracing how API Gateway processes a request.
| Property | Description |
|---|---|
| Log Group | API-Gateway-Execution-Logs_{restApiId}/dev |
| Purpose | Traces internal request flow and mapping templates |
| Best For | Debugging authorization, validation, or mapping issues |
Ensure your API stage has CloudWatch execution logging enabled and that API Gateway can push logs.
Attach the following managed policy to grant permissions:
arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
2. Access Logs
Access logs summarize each API request and response in a structured JSON format.
These are invaluable for analytics, performance tracking, and auditing user behavior.
| Property | Description |
|---|---|
| Log Group ARN | arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev |
| Purpose | Capture per-request analytics and response metadata |
| Best For | Monitoring latency, traffic, and client behavior |
Access Log Format Template
To enable structured access logs, open your API Gateway stage and set the log format using the following JSON template.
{
"requestId": "$context.requestId",
"ip": "$context.identity.sourceIp",
"time": "$context.requestTime",
"method": "$context.httpMethod",
"path": "$context.resourcePath",
"status": $context.status,
"integrationStatus": $context.integration.status,
"userAgent": "$context.identity.userAgent",
"authorizer": "$context.authorizer.role",
"error": "$context.error.message"
}
Once redeployed, every request will produce:
Execution Logs: Detailed traces for debugging.
Access Logs: High-level JSON entries for analytics.
API Gateway Execution Logs showing Lambda invocation flow, request mapping, and response transformation details.
Structured JSON access logs capturing requestId, method, path, status, and authorizer context for analytics.
Conclusion: You’ve Built More Than an API
TrailLog isn’t just a CRUD backend. It’s a governed, secure, and observable API platform built with managed AWS services.
Key Takeaways
Pre-Lambda Validation: Offload schema checks to API Gateway for cleaner, cheaper code.
Custom Authorization: Implement fine-grained security without burdening business logic.
Structured Logging: Use CloudWatch for comprehensive visibility and debugging.
By embracing these patterns, you can use Amazon API Gateway not just as a trigger mechanism, but as a complete API management engine — the foundation of scalable, production-ready serverless systems.