Load Balancing

Load Balancing

A load balancer distributes incoming traffic across multiple backend servers to ensure no single server bears too much load. It's a critical component in any horizontally scaled system.

Layer 4 vs Layer 7 Load Balancing

Layer 4 (Transport) — routes based on IP and TCP/UDP port

Client → L4 LB → Server
         │
         └── Sees: source IP, dest IP, port
             Does NOT see: HTTP headers, cookies, URL path
             Faster: less processing

Layer 7 (Application) — routes based on HTTP headers, URL, cookies

Client → L7 LB → Server
         │
         └── Sees: URL path, headers, cookies, request body
             Can route: /api/* → API servers, /static/* → CDN
             Slower: terminates and re-establishes TCP connections

Load Balancing Algorithms

Round Robin — requests distributed cyclically

Request 1 → Server 1
Request 2 → Server 2
Request 3 → Server 3
Request 4 → Server 1 ...

Weighted Round Robin — servers with more capacity get more requests

Server 1 (weight=3), Server 2 (weight=1)
→ S1, S1, S1, S2, S1, S1, S1, S2 ...

Least Connections — route to the server with fewest active connections

// Pseudocode
int minConn = Integer.MAX_VALUE;
Server chosen = null;
for (Server s : servers) {
    if (s.activeConnections < minConn) {
        minConn = s.activeConnections;
        chosen = s;
    }
}

IP Hash — same client IP always routes to same server (session persistence without sticky sessions)

hash(clientIP) % numServers → server index

Consistent Hashing — handles server additions/removals with minimal redistribution (see dedicated lesson).

Health Checks

Load balancers continuously check if backend servers are healthy:

Active health check (every 30s):
  GET /health → 200 OK → server is healthy
  Timeout or 5xx → mark server down, stop sending traffic

Passive health check:
  Track real request failures
  If error rate > threshold → temporarily remove server

Session Persistence (Sticky Sessions)

Some applications need the same client to always hit the same server:

Cookie-based: LB injects SERVERID cookie
X-Forwarded-For header: client IP tracked

⚠️ Drawback: if a sticky server fails, all its sessions are lost
Preferred: make application stateless (store session in Redis) instead

Common Load Balancers

Geographic Load Balancing

Route users to the nearest data center using DNS-based load balancing:

User in EU → eu-west data center
User in US → us-east data center

Interview Tips

1. For most system design questions, describe L7 load balancing (more features, handles HTTP routing).
2. Health checks are essential — always mention them when discussing LB.
3. Stateless backends + external session store (Redis) is the recommended approach over sticky sessions.