Load Balancing: Algorithms & Layer 4 vs. Layer 7

A single server can only take so many requests before it runs out of CPU, memory, or open connections. The fix is almost never "buy a bigger server" forever — at some point it's cheaper and safer to run several smaller instances behind something that spreads traffic across them. That something is a load balancer, and the algorithm it uses to pick a server is one of those decisions that looks trivial until it isn't.

Why you need one at all

The moment you have more than one backend instance, someone has to decide which instance handles which request. Do that badly and you'll end up with one server pegged at 100% CPU while its neighbor sits idle. A load balancer sits in front of the fleet and makes that decision for every incoming request, which buys you three things at once: no single instance gets overwhelmed, you can scale horizontally by simply adding more instances behind the same virtual IP, and you can deploy new code by rolling instances out of rotation one at a time instead of taking the whole service down.

That last point is easy to undervalue until you've lived without it. Without a load balancer, a deploy means a maintenance window. With one, it means quietly draining connections from one instance, replacing it, and adding it back — zero downtime, because the balancer never sends new requests to an instance that isn't ready for them.

The algorithms, and what each one is actually optimizing for

"Load balancing algorithm" sounds like one concept, but each one is answering a slightly different question about how to pick the next server.

• Round robin — the simplest possible answer: cycle through the server list in order, one request each. Server A, then B, then C, then back to A. It requires no state beyond "whose turn is it," and it works well when every server has roughly equal capacity and every request costs roughly the same to serve.
• Weighted round robin — the same rotation, but servers with more capacity get more turns. If server A is on hardware twice as powerful as B and C, you give it a weight of 2 so it receives twice the requests in each cycle. This shows up constantly in real fleets, where instances get added over time on newer, beefier hardware.
• Least connections — instead of blindly rotating, route each new request to whichever server currently has the fewest active connections. This matters the moment request durations stop being uniform: round robin happily sends the next request to a server that's still grinding through a slow upload, even though a neighboring server finished its work seconds ago and is sitting idle. Least connections notices that and routes around it.
• Consistent hashing — route based on a hash of some key in the request (a client IP, a session ID, a cache key) so the same key always lands on the same server. This is valuable any time a server benefits from seeing a client repeatedly — an in-memory cache that stays warm, or a WebSocket session that needs to keep hitting the process that holds its state. The "ring" structure underneath consistent hashing also means that adding or removing a server only reshuffles the keys near it on the ring, not the entire keyspace — a huge advantage over naivehash(key) % serverCount, where changing the server count remaps almost everything at once.

function pickServer(key: string, ring: SortedMap<number, Server>): Server {
  const hash = hashFn(key);

  // Find the first server whose position on the ring is >= hash.
  // Wrap around to the first server if we've gone past the end.
  const position = ring.ceilingKey(hash) ?? ring.firstKey();

  return ring.get(position);
}

Layer 4: fast, but blind to the request

A Layer 4 load balancer operates at the transport layer — it looks at IP addresses and ports and nothing else. It doesn't parse HTTP, doesn't know what URL was requested, and doesn't terminate TLS. That narrow scope is exactly why it's so fast: routing a packet based on a tuple of source/destination IP and port is cheap enough to do in the kernel or even in dedicated hardware, with minimal latency overhead. The tradeoff is that it can't make any routing decision that depends on the content of the request, because it never looks.

Layer 7: slower, but it can actually read the request

A Layer 7 load balancer operates at the application layer, which means it terminates the connection, parses the HTTP request, and can route based on the URL path, headers, cookies, or even the request body. That unlocks routing decisions a Layer 4 balancer simply can't make: send everything under /api/* to one pool of servers, everything under /static/* to a CDN-backed pool, and requests carrying a particular auth cookie to a pool that's pinned to that user's region. The cost is more processing per request — parsing HTTP and inspecting headers is inherently more work than reading a packet header — but for most web applications that overhead is negligible next to the flexibility it buys.

Health checks: routing around failure automatically

None of these algorithms matter if the load balancer keeps sending traffic to a server that's already dead. That's what health checks are for: the load balancer polls each backend on a fixed interval — often an HTTP GET to a lightweight /health endpoint — and expects a fast, successful response. Miss a threshold number of checks in a row and the balancer pulls that instance out of rotation immediately, no human intervention required. Once the instance recovers and starts passing checks again, it's added back automatically.

This is what turns a partial outage into a non-event instead of a cascading failure. If one instance starts throwing errors because of a bad deploy, a disk filling up, or a downstream dependency timing out, health checks isolate the damage to that instance within seconds, while the rest of the fleet keeps serving traffic normally. Without them, a single unhealthy instance keeps receiving its full share of requests and drags down the overall error rate for every user unlucky enough to be routed to it.