Why Does a Slow Model Feel Even Slower?

Here is a counterintuitive fact: when users complain that a product is "too slow," it is rarely because the total response time is actually long — it is because the experience of waiting broke down. Waiting three seconds for a short video is barely noticeable; waiting three seconds for an AI answer can make you want to hit refresh. Behind this lies a "latency perception ledger" that can be measured — and designed.

1. Human Time Perception Is Non-linear

The Weber-Fechner law of psychophysics tells us that perceived stimulus intensity is logarithmic in the actual stimulus — and time is no exception. The gap between 0.5s and 1s is trivial, but 5s to 10s is a qualitative shift: the brain reclassifies "waiting beyond expectation" as "stuck." Even subtler: uncertain waits feel longer than known ones. A hospital that says "about forty minutes" calms patients more than "almost ready," because the former gives the brain a concrete anchor.

For AI products this means shaving latency from 8s to 6s may register as nothing, while adding a "thinking…" progress indicator cuts complaints immediately. Not mysticism — expectation management.

2. Time To First Token Is the Real KPI

Classic web performance has TTFB; LLM apps have TTFT (Time To First Token). Repeated online A/B tests confirm that user satisfaction with AI answers tracks TTFT far more than total generation time. The reason is simple: before the first token appears, the brain sits in the anxiety of "is the system even alive?" Once text starts flowing, waiting becomes a reading-pace problem — and reading is pleasant.

This is why streaming output is now standard in nearly every LLM product. The backend saves not a single millisecond, yet perceived speed roughly doubles. Combine streaming with skeleton screens and typewriter rendering, and perceived latency can drop below half of actual latency.

3. Keep the Latency Ledger in Layers

An AI request's timeline is long: network round-trip, auth, prompt assembly, prefill (processing the input), and decode (token-by-token generation). A common engineering trap is watching only total time and optimizing the wrong layer. For example, prefill cost grows roughly with the square of input length; if a context-stuffed request burns two seconds in prefill alone, tuning decode speed is pointless — slim the context or reuse the KV cache first.

Another overlooked variable is output certainty. When users cannot tell whether they'll get half a sentence or three hundred words, the wait feels interminable. The clever fix is to emit a short conclusion frame first (even just "found it — three points"), then expand. You've split one uncertain long wait into several certain short ones.

4. When Fast Is Actually Bad

One last angle rarely discussed: instant replies can erode trust. A customer service bot that replies in zero seconds feels robotic; a report generated instantly feels like it wasn't read carefully. A modest amount of "thinking time" is itself a quality signal. So the goal of latency work is not squeezing every millisecond to zero, but making every wait meaningful, responsive, and predictable.

Latency perception is the craft of translating engineering metrics into psychological experience. Remember the three rules of this ledger: optimize first token before total time; give every wait an anchor; slice long tasks into feedback-rich segments. Follow them, and a six-second experience can beat a silent three-second one.