Horizontal vs Vertical Scaling

Horizontal vs Vertical Scaling

Scaling is the process of handling increased load. The two fundamental strategies differ in whether you add more power to existing machines or add more machines.

Vertical Scaling (Scale Up)

Replace a server with a more powerful one: more CPU, RAM, faster disk.

Before:                    After:
┌─────────────────┐        ┌──────────────────────────┐
│ 4 cores, 16GB   │  →     │ 64 cores, 512GB RAM      │
│ 500GB SSD       │        │ 2TB NVMe                 │
└─────────────────┘        └──────────────────────────┘

Pros:

• No code changes needed
• Simple: one server to manage
• Low latency (no network hops between components)

Cons:

• Hard limit — there's a ceiling on how big a single machine can be
• Single point of failure — if the server dies, everything dies
• Expensive — high-end machines cost disproportionately more
• Downtime during upgrades

Horizontal Scaling (Scale Out)

Add more machines and distribute load across them.

           ┌────────────┐
Requests → │  Load      ├→ Server 1
           │  Balancer  ├→ Server 2
           └────────────┘→ Server 3

Pros:

• No theoretical limit — add as many servers as needed
• Fault tolerance — if one server dies, others serve requests
• Cost-effective — commodity hardware
• Zero-downtime scaling (add/remove servers without restart)

Cons:

• Statelessness required — servers can't hold session state locally
• Distributed system complexity — consistency, networking, orchestration
• Data partitioning — databases need sharding strategy

The Statelessness Requirement

For horizontal scaling to work, each server must be stateless — any server can handle any request.

❌ Stateful (broken with horizontal scaling):
User session stored in Server 1's memory
→ If load balancer routes next request to Server 2, session is lost!

✅ Stateless (works with horizontal scaling):
User session stored in Redis (shared cache)
JWT token carries auth state
All servers read from the same database
→ Any server can handle any request

When to Use Each

Auto-Scaling

Modern cloud platforms (AWS EC2 Auto Scaling, GKE, Kubernetes HPA) add/remove servers automatically based on CPU, memory, or custom metrics:

# Kubernetes Horizontal Pod Autoscaler
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Interview Tips

1. The answer is almost always horizontal scaling in system design interviews — but explain why: fault tolerance, no ceiling, cost efficiency.
2. Always mention statelessness when describing horizontal scaling — it's the prerequisite.
3. Vertical scaling still has a place: databases (especially write-heavy ones) are often harder to scale horizontally.