WebSocket architecture best practices: Designing scalable realtime systems

If you’re building realtime application, you’re likely already familiar with the benefits of WebSockets—persistent, bidirectional, low-overhead communication, and near-universal availability.

But even if WebSocket is already on your radar, you might not yet have looked into WebSocket architecture best practices. Particularly if you’re more used to building request-response based applications with HTTP, then working with WebSocket requires some changes in your assumptions and the tools you use.

The good news is that this change is fairly straightforward. In this guide, we’ll explore the key considerations for designing an application architecture with WebSocket. We’ll cover both architectural and operational best practices to ensure scalability and long-term reliability. And, crucially, we’ll look at how WebSocket’s architectural needs differ from system design for HTTP.

A brief overview of WebSocket vs HTTP

HTTP’s stateless, request-response model is effectively the default pattern for client-server and intra-application communication. And so much of the tooling we use to build applications is designed around those assumptions.

So, before we get into WebSocket architecture best practices, let’s put WebSocket and HTTP side by side to underscore how they differ.

In summary, the key difference with WebSocket is that it provides persistent, stateful connections, allowing both the client and server to send messages at any time. This is what makes it great for realtime communication, but it also means we face new challenges.

Challenges of WebSocket architecture

Every protocol and design pattern comes with its own set of pros and cons. For instance, HTTP’s statelessness makes it easy to scale horizontally, but it also requires workarounds—such as

So, how does sharding WebSocket shape-up in practice?

Pros of sharding:

• 
  Load distribution: Balances load across multiple servers.


• 
  Horizontal scalability: It’s easy to add and remove shards as needs change.


• 
  Fault isolation: Failures in one shard shouldn’t impact others.

Cons of sharding:

• 
  Cross-shard complexity: Maintaining consistency between shards adds complexity.


• 
  Hot shards: Some shards may become overloaded and need rebalancing.


• 
  Shard redistribution: When you need to add or remove shards, the system needs to update the sharding scheme on the client-side and then redistribute sessions across shards.


• 
  Siloed state: Harder for clients connected to different servers to communicate. Also harder to move clients between shards without losing state.

Sharding’s biggest issue is the risk of creating silos within your system. For example, in a chat application, if users from the US all connect to shard 1 and users from Japan connect to shard 2, what happens when someone from Tokyo wants to chat with someone in Chicago? In effect, the sharded system means that you have two or more entirely separate chat servers and there’s no way for those two people to chat with each other.

The solution is to share state updates between shards. That way, users shouldn’t ever notice that the sharding exists. But it introduces additional complexity. Sharing state across shards can lead to data integrity and ordering issues. And, of course, some part of each shard’s resources will be dedicated to backend synchronization, meaning you’ll need more shards than otherwise.

Any time you introduce more than one WebSocket server, you’ll need to consider state synchronization between those servers. But issues like hot shards and resharding are more likely to occur in sharded systems. Using a load balancer, instead of a sharding scheme, is one way to address that.

Sticky sessions with stateful load balancing

If traditional HTTP load balancing isn’t well-suited to WebSocket because it’s designed for short-lived, stateless connections, is there a way to adapt the approach to suit WebSocket’s persistent connections? One way is to make sure that specific clients connect to the same server every time they reconnect. These are sticky sessions.

Sticky sessions are different to shards because the stickiness lasts only so long. Usually for the duration of a session. That way, the client doesn’t need to know the sharding scheme but just where to find the load balancer.

Here’s how it works: once a client establishes a connection with a server, any subsequent reconnections are directed to the same server. This avoids the need to synchronize session state across multiple servers. While this helps maintain session continuity, it also introduces some challenges. The load balancer needs to track which client belongs to which server, which adds complexity and can make that load balancer into a single point of failure. And if the assigned “sticky” server goes down, there’s a chance of some data loss depending on how robust the inter-server data sharing is.

So, what are the pros and cons of sticky sessions?

Pros of sticky sessions:

• 
  Scalability: Easily scale horizontally by adding or removing servers.


• 
  Session affinity: Clients reconnect to the same server, keeping session state intact.


• 
  Greater flexibility: There’s no need to maintain a sharding scheme or to reshard when scaling.

Cons of sticky sessions:

• 
  Session persistence overhead: Managing sticky sessions requires an additional tool in the form of the load balancer.


• 
  State sharing: Like sharded systems, there’s an additional resource overhead to share data between server instances.


• 
  Single point of failure: If the load balancer fails then that shouldn’t affect open connections but could prevent clients from reconnecting. You can add more load balancers to reduce this risk.


• 
  Load balancing inefficiency: Poor load distribution can lead to server overload.

Sticky sessions have a lot in common with sharding. However, the load balancer simplifies the architecture and makes it easier to scale up or down by shielding the clients from the precise make-up of the back-end.

Implementing a pub/sub architecture

Pub/sub (publish/subscribe) helps scale WebSocket architectures by taking the problem of inter-server communication and making it the hero of the story. Rather than thinking of keeping servers in sync as a complication of scaling, pub/sub makes it a feature. In this approach, instead of each server needing to constantly share state with others, messages are published to a central system and then distributed to subscribers. This way, WebSocket servers focus on managing connections, while the pub/sub system takes care of ensuring that messages get to the right clients.

For example, in a realtime chat app, rather than having every WebSocket server manage both connections and message routing, the servers handle the connections and leave the pub/sub system to broadcast messages to all clients subscribed to a chat room or channel. By offloading this responsibility, pub/sub allows your architecture to scale without requiring every server to stay perfectly synchronized.

come in three main forms:

• 
  At-most once: Fire-and-forget that’s easy to implement but only suited to messages you don’t care about.


• 
  At-least once: Your message will arrive but you might have to handle deduplication on the client.


• 
  shows that projects like this rarely come in on time or under budget.


• 
  It’s also hard: Without extensive prior experience, even the best engineering teams can find it hard to build realtime communications platforms that scale. As we saw above, this isn’t just about implementing WebSocket but it could be a case of building a pub/sub messaging system.


• 
  Control is an illusion: One reason for developing a system in-house might be to build precisely what you need. But time and resource constraints mean that teams often end-up compromising and building a lesser solution. Using a realtime platform-as-a-service (PaaS) you can have the best of both worlds, in that you build the solution you need with building blocks created and maintained by specialists.