Understanding Envoy Proxy's Hot Restart Implementation

As modern distributed systems grow in complexity, the ability to update proxy configurations without dropping active connections has become crucial. In this post, I'll break down how Envoy Proxy implements its hot restart mechanism, a feature that allows seamless configuration updates and binary upgrades without disrupting existing connections.

What is Hot Restart?

Hot restart (or hot reload) is a mechanism that allows a proxy server to reload its configuration or upgrade its binary while maintaining existing client connections. This is achieved by having the new process take over the listening sockets and existing connections from the old process, ensuring zero connection drops during the transition.

Envoy's Approach

Envoy implements hot restart through a parent-child process model, where the parent process manages the handover of socket descriptors to the new child process. Here's how it works:

1. Shared Memory Architecture
   Envoy uses shared memory to facilitate communication between the old and new processes. This is implemented in the HotRestartImpl class:
   

class HotRestartImpl {

private:

  static constexpr uint64_t MAX_STAT_SEGMENTS = 256;

  SharedMemory* shmem_;

  Stats::StatDataAllocator* stats_allocator_;

};

1. Socket Passing Process
   The hot restart process follows these key steps:

Initialize Shared Memory: The parent process creates a shared memory segment that both processes can access.

Socket Duplication: The parent process duplicates its listening sockets.

Graceful Handover: Traffic is gradually transferred to the new process.

Here's a simplified version of how Envoy handles socket passing:

class HotRestartingChild {

public:

  void initialize(int argc, char** argv) {

    // Request parent's listen sockets

    std::vector<int> fds = parent_.retrieveListenSockets();



    // Initialize new server with inherited sockets

    for (int fd : fds) {

      Server::createListenerFromSocket(fd);

    }



    // Signal ready to parent

    parent_.sendReady();

  }

};

1. State Transfer
   One of the most critical aspects is transferring the state of existing connections:
   

void HotRestartImpl::drainListeners() {

  // 1. Stop accepting new connections

  for (auto& listener : listeners_) {

    listener->stopAcceptingConnections();

  }





  // 2. Wait for existing connections to complete

  while (hasActiveConnections()) {

    std::this_thread::sleep_for(std::chrono::milliseconds(100));

  }



  // 3. Signal completion to new process

  notifyNewProcess();

}

Key Implementation Challenges

1. File Descriptor Handling
   Envoy needs to carefully manage file descriptors to ensure they're properly transferred and not leaked:

Uses SCM_RIGHTS to pass file descriptors between processes

Maintains a registry of active file descriptors

Implements careful cleanup mechanisms

1. Connection State Management
   The proxy must maintain connection state during the transition:

TCP connection parameters

TLS session information

Protocol-specific state (HTTP/2 streams, etc.)

1. Configuration Compatibility
   Envoy ensures that configuration changes are compatible with existing connections:
   

bool HotRestartImpl::validateConfig(

  const envoy::config::bootstrap::v3::Bootstrap& new_config) {

  // Verify that critical fields haven't changed

  // Check listener compatibility

  // Validate cluster configurations

  return isCompatible;

}

Best Practices Learned from Envoy

• Incremental Testing: Implement comprehensive tests for each component of the hot restart system.




• Graceful Degradation: Have fallback mechanisms if hot restart fails.




• Monitoring: Add metrics to track hot restart success rates and duration.




• Documentation: Maintain clear documentation about the process and limitations.

Conclusion

Diving into Envoy's hot restart implementation has been quite the journey! It's fascinating to see how they've tackled the challenge of swapping out a running proxy without dropping connections. The elegant dance between parent and child processes, the careful handling of file descriptors, and the intricate state management all come together to make this possible.

What really stands out is how much thought went into making the system robust. It's not just about passing sockets around – it's about handling edge cases, ensuring configuration compatibility, and providing fallback options when things don't go as planned.

As I explore implementing similar features in Rust-based systems, I'm particularly excited about how Rust's ownership model could provide some interesting advantages in handling the file descriptor lifecycle and process coordination. The strict compiler checks could help catch potential issues that might lead to descriptor leaks or invalid states during the handover process.

Note: This is a high-level overview based on Envoy's open-source implementation. For the most up-to-date and detailed information, please refer to the official Envoy documentation and source.[https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/arch_overview]