Your cache is hiding a slow endpoint

We removed a cache last week to fix a stale-data bug, and the actual result was discovering that one of our endpoints had been taking two seconds for months. Nobody had filed a ticket about it. Nobody knew. The cache had been quietly absorbing it the entire time, and it took making the app worse to find out.

The cache was doing its job. That was the problem.

The setup was ordinary. A tab in our app needed to know which of several screens to show, so on boot we'd fetch that from the backend, write the answer to local storage, and on open we'd read it back and render instantly. Refresh in the background, correct if needed. It felt fast because it was fast — the screen appeared with no spinner at all.

When we finally ripped that path out so the screen would reflect the database instead of yesterday's answer, the tab started taking about two seconds to open. My first assumption was that I'd broken something. I hadn't. That was just how long the call had always taken. We'd built a very effective device for not noticing.

What it was actually doing

The endpoint answered a question that sounds trivial: which day of a seven-day setup period is this user on. A single small integer.

To produce it, the backend re-read a week of raw sensor events out of DynamoDB one day at a time, ran a couple of unindexed Cassandra scans per day on top of that, recomputed the entire week from scratch, wrote the result back, and then returned it. Fourteen-odd round trips and a write, to answer something that was already sitting in a row waiting to be read. The method on the client was called getCalibrationData, which is how I'd managed to look straight at it a dozen times without registering that it was a POST.

None of that was a secret. It was all right there in a file anyone could open. It just never cost anyone anything, so nobody opened it.

Slow is loud. Stale is silent.

This is the trade, and I don't think most of us make it deliberately: a cache converts a performance problem into a correctness problem.

Performance problems announce themselves. Someone waits, someone complains, it shows up in a dashboard, it has a number attached to it that goes in the wrong direction. Correctness problems from staleness do none of that. The app is fast. The data is wrong in a way that looks completely plausible - an old number is still a number. It surfaces weeks later as a support ticket that can't be reproduced, because reproducing it requires a device whose local storage is in a specific state you can't recreate.

We swapped a loud problem for a quiet one and recorded it as an optimization. The latency graph got better. The thing the latency graph was measuring got worse.

Then the cache becomes structural

Here's the part that makes this hard to walk back. Once the instant path exists, the product starts assuming it. Screens are designed without loading states. Nobody adds a skeleton because there's nothing to skeleton. Transitions get tuned against a render that happens in a single frame.

By the time you suspect the cache is covering for something, you can't take it out to check - not even briefly, not even in staging with a flag - because removing it makes the app visibly worse and now that's a regression with your name on it. The cache stops being a decision you made and becomes a constraint you inherited. We only got to see behind ours because we were removing it for an unrelated reason and had already accepted the slowdown as the cost.

What I'd do differently

Measure the thing before you cache it, and write the number down somewhere a human will see again. Not in a commit message. A comment on the cache itself: this exists because the call underneath takes 2.1s. That single sentence is the difference between a deliberate trade and an accident nobody remembers making.

Then be suspicious of any cache whose justification is only ever stated as "so it's fast." Fast compared to what? If nobody on the team can answer that in seconds, the cache isn't an optimization, it's anesthesia - and the thing it's numbing is still there, getting slower, with nobody watching it.

We're adding a read-only endpoint that returns that integer directly. It'll be about fifty milliseconds. The right fix was always cheap and always available. We just spent months unable to see the problem, because we'd built something very good at making sure it never hurt.