AWS Cloud Path Week 15: AWS Networking Workshop Part 5 - Hybrid DNS with Route 53 Resolver, Network Monitoring and VPC Flow Logs

Missed the session? Catch up here:

Welcome to Week 15 of the AWS Cloud Path series! In this comprehensive workshop, we dive deep into AWS network monitoring capabilities, focusing on Route 53 Resolver for hybrid DNS, CloudWatch monitoring, and VPC Flow Logs. This session builds upon the foundational AWS networking workshop available at https://networking.workshop.aws/.

Prerequisites

Before starting this workshop, ensure you have:

• AWS account with appropriate permissions


• Basic understanding of VPC and networking concepts


• Familiarity with CloudFormation templates


• Access to AWS CLI or AWS Console

Hybrid DNS with Route 53 Resolver

DNS resolution is fundamental to any internet-connected infrastructure. When working with hybrid cloud environments, you need seamless DNS resolution between your on-premises infrastructure and AWS environments without routing traffic through the public internet.

Understanding Route 53 Resolver

Route 53 Resolver makes hybrid cloud easier for enterprise customers by enabling DNS resolution across connected environments. The key components include:

Outbound Endpoints: These are Elastic Network Interfaces (ENIs) that forward DNS queries from your AWS environment to on-premises DNS servers.

Elastic Network Interfaces: Remember that ENIs always have IP addresses, so when planning your VPC subnets, account for the IP addresses that AWS resources using network interfaces will consume.

Setting Up Outbound Endpoints

To create an outbound endpoint:

1. Navigate to Route 53 in the AWS Console


2. Under Resolver, select "Outbound endpoints"


3. Click "Create outbound endpoint"


4. Configure the following:
   
   
   
   
   • 
     Endpoint name: Choose a descriptive name
   
   
   • 
     VPC: Select the VPC connected to your on-premises environment
   
   
   • 
     Security group: Note that you cannot change this after creation
   
   
   • 
     IP address type: Choose IPv4, IPv6, or dual stack (IPv4 recommended)
   
   
   • 
     Availability zones: Minimum of two for high availability
   
   
   • 
     IP addresses: Specify automatic assignment or manual IP addresses
   
   
   
   

The endpoint creates network interfaces in at least two availability zones for redundancy. This ensures your DNS resolution remains available even if one AZ experiences issues.

Creating Resolver Rules

After setting up the outbound endpoint, create resolver rules to define which domain names should be forwarded:

1. Go to Route 53 Resolver Rules


2. Click "Create rule"


3. Configure:
   
   
   
   
   • 
     Rule name: Descriptive identifier
   
   
   • 
     Rule type: Forward (most common)
   
   
   • 
     Domain name: The domain to resolve (e.g., example.corp)
   
   
   • 
     Outbound endpoint: Select your created endpoint
   
   
   • 
     Target IP addresses: IP address of your on-premises DNS server
   
   
   • 
     VPC associations: Select VPCs that should use this rule
   
   
   
   

This configuration ensures that whenever resources in associated VPCs need to resolve the specified domain, they'll forward the request through your outbound endpoint to the on-premises DNS server.

CloudWatch Monitoring for Networks

CloudWatch is the central monitoring service in AWS, supporting metrics from virtually every AWS service including networking components.

Automatic Dashboards

CloudWatch provides automatic dashboards for various services, including VPC networking. These pre-configured dashboards show key metrics without manual setup:

• Active connection counts


• Packet drop counts


• Network interface utilization


• Transit Gateway metrics

To access automatic dashboards:

1. Navigate to CloudWatch Console


2. Select "Dashboards" from the left menu


3. Choose "Automatic dashboards"


4. Select "VPC and Transit Gateway" or relevant service

Creating Custom Dashboards

While automatic dashboards are convenient, custom dashboards allow you to focus on specific metrics relevant to your environment.

Creating a Custom Network Dashboard:

1. Go to CloudWatch → Dashboards


2. Click "Create dashboard"


3. Add widgets by selecting metrics:
   
   
   
   
   • Navigate to EC2 → Per-Instance Metrics
   
   
   • Filter by "Network" to find network-related metrics
   
   
   • Select NetworkIn and NetworkOut for specific instances
   
   
   
   


4. Configure visualization (line chart, number, gauge)


5. Name your dashboard appropriately

Pro tip: Custom dashboards become essential when presenting metrics to management or when you need to monitor specific combinations of metrics not available in automatic dashboards.

Setting Up CloudWatch Alarms

Alarms help you proactively monitor your infrastructure and respond to issues before they impact users.

Creating a Network Alarm:

1. Navigate to CloudWatch → Alarms → All alarms


2. Click "Create alarm"


3. Select metric (e.g., EC2 → Per-Instance Metrics → NetworkIn)


4. Configure conditions:
   
   
   
   
   • 
     Statistic: Maximum, Average, or Sum (pay attention to which is appropriate)
   
   
   • 
     Period: 1 minute for immediate response, longer for trend analysis
   
   
   • 
     Threshold type: Static (you define value) vs Anomaly detection (AWS learns normal patterns)
   
   
   • 
     Condition: Greater than, less than specific value
   
   
   
   

SNS Integration:

Configure Simple Notification Service (SNS) for alarm notifications:

1. Create new SNS topic or use existing


2. Add email endpoints for notifications


3. Configure alarm states (In alarm, OK, Insufficient data)


4. Set up actions like auto-scaling or Lambda functions for automated responses

Important: Always confirm your email subscription to receive alarm notifications.

VPC Flow Logs

VPC Flow Logs capture information about IP traffic flowing through your VPC network interfaces, providing visibility into network communication patterns.

Understanding Flow Logs

Flow Logs monitor traffic to and from network interfaces within your VPC. Since every AWS resource with network connectivity uses an Elastic Network Interface, Flow Logs provide comprehensive network visibility.

Key capabilities:

• Monitor traffic patterns within your VPC


• Debug network connectivity issues


• Analyze security group and NACL effectiveness


• Track data transfer for cost optimization

Setting Up VPC Flow Logs

Step 1: Create CloudWatch Log Group

1. Navigate to CloudWatch → Log groups


2. Click "Create log group"


3. Configure:
   
   
   
   
   • 
     Name: networking-workshop-flow-logs
     
   
   
   • 
     Retention setting: Set realistic retention (1 week, 1 month) - avoid "Never expire" to control costs
   
   
   • 
     Log class: Standard for frequently accessed logs, Infrequent Access for cost savings
   
   
   
   

Step 2: Create Flow Log

1. Go to VPC Console


2. Select your VPC


3. Navigate to "Flow logs" tab


4. Click "Create flow log"


5. Configure:
   
   
   
   
   • 
     Name: Descriptive identifier
   
   
   • 
     Filter: All traffic (recommended for comprehensive monitoring)
   
   
   • 
     Maximum aggregation interval: 1 minute for detailed monitoring, 10 minutes for cost optimization
   
   
   • 
     Destination: CloudWatch Logs
   
   
   • 
     Log group: Select created log group
   
   
   • 
     IAM role: Use existing or create new service role
   
   
   
   

Analyzing Flow Logs with CloudWatch Insights

CloudWatch Logs Insights provides a query language for analyzing log data:

1. Navigate to CloudWatch → Logs Insights


2. Select your flow logs log group


3. Use sample queries or create custom queries:
   

fields @timestamp, srcaddr, dstaddr, srcport, dstport, bytes

| filter srcaddr like /10.0/

| stats sum(bytes) by srcaddr, dstaddr

| sort bytes desc

| limit 10

This query shows the top 10 source/destination pairs by bytes transferred.

Sample Use Cases:

• Identify top talkers in your network


• Analyze traffic patterns by time


• Debug connectivity issues


• Monitor for unusual network activity

Cost Optimization Tips

Flow Logs can generate significant costs:

• Set appropriate retention periods


• Use selective filtering (accepted/rejected traffic only)


• Consider log aggregation intervals


• Monitor log group size regularly


• Delete unnecessary log groups

Generating Test Traffic

To validate your monitoring setup, generate network traffic between instances:

Using Session Manager:

1. Connect to EC2 instance via Systems Manager Session Manager


2. Use networking tools like ping or iperf3


3. Generate traffic between VPCs through Transit Gateway


4. Monitor resulting metrics and logs

Security Group Configuration:

Ensure security groups allow the traffic you're trying to generate:

• ICMP for ping traffic


• Specific ports for application traffic


• Proper source/destination configurations

Key Takeaways

1. 
   Hybrid DNS: Route 53 Resolver enables seamless DNS resolution between AWS and on-premises environments


2. 
   Monitoring Strategy: Combine automatic dashboards with custom dashboards tailored to your specific needs


3. 
   Proactive Alerting: Set up CloudWatch alarms with appropriate thresholds and SNS notifications


4. 
   Network Visibility: VPC Flow Logs provide comprehensive insight into network traffic patterns


5. 
   Cost Management: Always set realistic retention periods and monitor log storage costs


6. 
   Testing: Generate test traffic to validate your monitoring and alerting setup

Cleanup Instructions

Don't forget to clean up resources to avoid unexpected charges:

• Delete EC2 instances


• Remove VPC Flow Logs


• Delete CloudWatch log groups with short retention needs


• Clean up SNS topics and subscriptions


• Remove unnecessary CloudWatch dashboards and alarms

What's Next

This completes the foundational AWS Networking Workshop series, we will have one more part for the AWS Network Workshop Parts (6 in total).

For AWS credits to support your learning and experimentation, check the community resources or reach out through the appropriate channels.

Remember: The foundation you've built here with VPC networking, monitoring, and DNS resolution forms the basis for more advanced AWS networking topics. Practice these concepts and experiment with different configurations to deepen your understanding.

This article is based on the AWS Networking Workshop available at https://networking.workshop.aws/. The visual demonstrations and hands-on examples from the video provide additional context beyond what can be represented in text format.