I added an in-memory cache to shield a slow external API. It worked perfectly in development. It worked fine with a single instance. Then we deployed multiple pods.
Suddenly:
- Cache misses became inconsistent
- Requests routed to different pods returned different behavior
- Pod restarts wiped cache state entirely and brought back latency
At first, it felt like I had "done caching wrong."
But the real issue wasn't the cache — it was my assumptions. I had assumed memory was shared, processes were long-lived, and state would stick around. Kubernetes assumes the opposite.
Scaling doesn't break systems — it exposes the assumptions we didn't know we were making.
Since this was meant to be a temporary mitigation, I focused on stabilizing instead of redesigning everything:
- Warming the cache on pod startup to reduce cold-start impact
- Making cache behavior deterministic across pods
- Introducing explicit invalidation so stale data didn't linger
- Accepting eventual consistency instead of chasing perfection
Once you scale horizontally, cache lifecycle, consistency, and invalidation become first-class concerns.
The biggest takeaway: Temporary solutions still need distributed thinking. If you've had something work flawlessly until production, you've probably had a similar moment.