Load Balancing & Auto-Scaling Explained

How one URL serves millions

Type a popular site's address and millions of other people are doing the same thing right now. Yet no single computer on Earth could handle that, one server has finite CPU, memory, and network. So how does one URL serve a crowd that would crush any single machine? The answer is the most important pattern in scalable systems, and it's made of exactly two ideas.

Idea one: run many identical copies of your app and spread traffic across them, that's load balancing. Idea two: automatically add more copies when it's busy and remove them when it's quiet, that's auto-scaling. Together they turn "my one server fell over under load" into "the system grew to meet demand and shrank to save money." This article builds both from scratch.

Note: Who this is for: Anyone who's deployed an app to one server and wondered how real products handle traffic. No prior scaling experience needed. If you understand that a server can get overloaded, you're ready.

What a load balancer actually is

A load balancer is a single front door that receives all the traffic and spreads it across a pool of identical servers behind it, so no single server gets crushed, and a failing one is quietly skipped.

Users only ever see the load balancer's address. They have no idea whether there are two servers behind it or two hundred. You can add, remove, or replace servers and the public URL never changes. The everyday picture:

A load balancer is the host at a busy restaurant deciding which open table gets the next party.

See it as a picture

All traffic hits the load balancer. It health-checks each app instance and forwards each request to a healthy one. A separate component, the auto-scaler, watches metrics like CPU and adds or removes instances behind the load balancer as load changes.

This part is interactive in the original. discipline paying off.

Health checks: the part that prevents outages

A load balancer is only as good as its health checks. Every few seconds it asks each instance "are you OK?" by hitting an endpoint (like /healthz). Healthy instances get traffic; an instance that fails its checks is pulled out of rotation automatically, no human, no 3am page. When it recovers, it's added back. This is how one crashed server stops affecting users almost instantly.

Tip: Your health check should verify the app can actually do its job (e.g. reach the database), not just that the process is running. A check that always returns 200 even when the database is down will keep routing users into a broken instance, defeating the whole point.

How it decides where to send each request

When a request arrives, the load balancer picks an instance using an algorithm. The defaults are simple and usually right, but knowing the options is a classic interview question and occasionally matters for real performance.

Algorithm	How it picks	Best when
Round-robin	Next instance in order, one after another	Instances are equal and requests are similar, the default
Least connections	The instance with the fewest active connections	Requests vary a lot in duration (some slow, some fast)
Weighted round-robin	Round-robin, but bigger instances get more	Your instances aren't all the same size
IP hash / sticky	Same client always hits the same instance	App keeps per-user state in memory (avoid if you can)
Least response time	The instance answering fastest right now	You want to favour healthy, fast instances automatically

Common load balancing algorithms, round-robin is the sensible default; reach for others when you have a specific reason.

Warning: Sticky sessions are a trap: Sticky sessions (IP hash) pin a user to one instance so its in-memory state survives. It feels convenient, but it breaks even load distribution and means losing that instance logs the user out. The real fix is to make instances stateless and store session data in a shared cache. Reach for stickiness only when you genuinely can't.

Auto-scaling: matching servers to demand

Load balancing spreads traffic across the instances you have. Auto-scaling changes how many you have. You set a metric and thresholds, "keep average CPU around 50%", and the auto-scaler adds instances when you're over and removes them when you're under, between a minimum and maximum you define.

1. 
   Set a target and bounds: e.g. target 50% CPU, minimum 2 instances (for redundancy), maximum 10 (to cap the bill).


2. 
   The scaler watches the metric: Every minute or so it checks average CPU (or request count, or queue depth) across the pool.


3. 
   Busy → scale out: Sustained CPU above target adds instances. They boot, pass health checks, and the load balancer starts sending them traffic.


4. 
   Quiet → scale in: Sustained low CPU removes instances so you stop paying for idle capacity.

Always keep a minimum of two instances across two availability zones, even at zero traffic. One instance means one crash equals an outage. Two across AZs means you survive losing one, the cheapest reliability win there is, the same logic .

See these patterns at planetary scale: .

Practise the networking these sit on in the browser: the , where this guide is interactive, with in-browser terminal labs and diagrams. Learn cloud and DevOps by doing, no videos.

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen