The Big Picture

Even at the same moment, the feed you see on your screen differs from the one your friend sees. Why?

Because your activities—like follows, likes, comments, shares, and even blocks—are unique to you.

A feed system must tailor itself to show each user a personalized experience.

How Feed Systems Work

A feed system is generally divided into two main components to efficiently achieve personalized feeds:

1. 
   

   Feed Publishing System

   
   
   
   
   
   • Activated when a user creates a new post.
     
   
   
   • Includes tasks like authentication, rate limiting, and fanout services.
     
   
   
   
   


2. 
   

   Feed Reading System

   
   
   
   
   
   • Activated when a user opens the app to view their feed.
     
   
   
   • Handles dynamic updates, such as refreshing posts on-demand.
     
   
   
   • Powered by tools like Content Delivery Networks (CDNs) and caching systems.
     
   
   
   
   

Two Ways to Build Feeds

When a user creates a post, the system must determine how to distribute it to everyone who should see it (e.g., Instagram followers or Facebook friends).

This process, known as fanout, can happen in two ways:

1. Push Model (Fanout-on-Write)

Posts are distributed to followers as soon as they are created.

✔ Advantages:

• 
  Real-time updates: Feeds feel immediate.
  


• 
  Quick reads: Posts are preloaded into followers’ feeds.
  

✘ Disadvantages:

• 
  Hot Key Problem: Popular users with millions of followers (e.g., Elon Musk) require immense resources to update all feeds.
  > Example: One tweet from Elon Musk can trigger updates for over 100 million feeds.
  


• 
  Resource Waste: Updating inactive users' feeds consumes unnecessary computing power.
  

2. Pull Model (Fanout-on-Read)

Posts are fetched and added to a feed only when a user refreshes it.

✔ Advantages:

• Optimized for inactive users: Resources are spent only when users actively check their feed.
  


• 
  No Hot Key Problem: System avoids bottlenecks caused by distributing posts to millions of followers at once.
  

✘ Disadvantages:

• 
  Slower reads: Fetching posts on demand can increase loading time.
  

3. Hybrid Model

Combines the strengths of both the Push and Pull Models to optimize performance and scalability.

• 
  Push Model: Used for most users to ensure fast, real-time feed updates.
  


• 
  Pull Model: Applied to high-profile users to handle the Hot Key Problem.
  

Example Scenario


A follower subscribes to both an average user and a celebrity:

1. When the average user posts (Push Model), it instantly appears in the follower’s feed.
   


2. When the celebrity posts (Pull Model), the post is only added when the follower refreshes their feed.
   

Additional Considerations

Web Server Responsibilities

When a user creates a post, the web server handles the following:

1. 
   Authentication: Verify the user making the request.
   


2. 
   Rate Limiting: Prevent spam by restricting the frequency of posts.
   


3. 
   Post Handling:
   
   
   
   
   • Save the post (usually as metadata to conserve storage).
     
   
   
   • Distribute it to followers.
     
   
   
   • Send notifications about the new post.
     
   
   
   
   

Graph Databases

Graph databases are ideal for managing relationships (e.g., friends, followers) in feed systems. They allow:

• Efficient retrieval of user relationships.
  


• Seamless integration with both Push and Pull Models.
  

Caching and Message Queues

These tools enhance speed and efficiency:

1. 
   

   Caching Layers: Separate caches for:

   
   
   
   
   
   • 
     Feed IDs
     
   
   
   • 
     Post Content
     
   
   
   • 
     Relationship Data
     
   
   
   • 
     User Interactions (likes, comments)
     
   
   
   • 
     Metrics (follower counts, engagement stats)
     
   
   
   
   



2. CDNs: As a distributed caching layer, CDNs improve global feed delivery.

By blending the Push, Pull, and Hybrid Models with advanced data management techniques, feed systems deliver fast, personalized, and scalable user experiences. 🌟