System Design - EP 3.2 : Throughput

Speed vs. Volume

Most people confuse Latency and Throughput. Imagine a water pipe. Latency is how fast the first drop of water gets through the pipe. Throughput is how many gallons of water flow out of the pipe per minute.

What is Throughput?

Throughput is the number of requests a system can process in a given unit of time. It is usually measured in Queries Per Second (QPS) or Transactions Per Second (TPS).

The "Restaurant" Analogy

Back to our restaurant:

• 
  Throughput is how many total meals the kitchen can finish and serve in one hour.

The Latency-Throughput Paradox

Ideally, you want Low Latency and High Throughput. But often, increasing one affects the other.

• If you give your kitchen staff 50 orders at once, the Throughput goes up (more meals per hour), but the Latency for each individual customer also goes up (their food takes longer to arrive).

Why It Matters

When you are scaling an app like Netflix, you care deeply about throughput. You need to ensure your system can handle 10 million concurrent streams without the "pipe" bursting.