Vertical vs Horizontal Scaling

Explore the differences between upgrading a single server and adding multiple servers to a cluster.

Goal: See how one larger server differs from a pool of smaller nodes under rising traffic.Try: Increase traffic until the system strains, then try an upgrade in vertical mode or add nodes in horizontal mode.

Scaling controls

Change traffic and capacity live.

Live scaling model

One server gets bigger.

Scale up

Users

180 rps

Node 1
CPU100%
Memory80%

180 requests/sec

Avg CPU
100%
Latency
187ms
Availability
100%
Cost Estimate
$168/hr
Active Nodes
1
Peak Node CPU
100%
Headroom
0%
Dropped Requests
0

Insight

What changed

The same server receives more CPU and memory.

Why it matters

Scale-up is simple and keeps architecture small.

Tradeoff

Eventually hardware hits a ceiling and one server remains a failure point.

Mini challenges

Can the system scale safely?

Adjust the controls above. Cards solve as the system reaches the target.

Keep latency below 120ms

Handle high traffic

Avoid one-server failure

Keep peak CPU below 80%

Explain the tradeoff

Step 1: Start with one server and increase traffic.
Step 2: Upgrade the server to see vertical scaling improve capacity.
Step 3: Push traffic higher until one machine becomes risky.
Step 4: Switch to horizontal scaling and add nodes.
Step 5: Simulate a failure and observe remaining capacity.
Step 6: Compare cost, headroom, and availability.

Summary

  • Vertical scaling increases the power of one machine.
  • Horizontal scaling adds more machines and distributes traffic.
  • Scale-up is simpler, but has a hard ceiling and a single failure point.
  • Scale-out improves availability, but needs load balancing and coordination.
REST vs gRPCLoad Balancing

Why this exists

Vertical scaling makes one machine stronger. Horizontal scaling adds more machines. Vertical scaling is simpler but bounded. Horizontal scaling is more flexible but requires coordination, distribution, and stateless design.

Vertical scaling preserves simplicity

One stronger machine avoids load balancing and distributed coordination, but there is a hard ceiling and bigger boxes get expensive quickly.

Horizontal scaling changes architecture

Once you add nodes, you need routing, shared state strategy, health checks, and a plan for partial failure.

State is the real constraint

Stateless app servers scale horizontally easily. Stateful components require partitioning, replication, or coordination mechanisms.

Key takeaways

  • Vertical scaling is easier, but capped.
  • Horizontal scaling is more elastic, but operationally heavier.
  • Stateless tiers are the first good candidates for horizontal scaling.
  • Scaling decisions are really decisions about where state lives.