Serverless is the right default for a lot of workloads. It was the wrong one for three of ours, and it took real production traffic — not a benchmark — to prove it.
The pitch we bought
Zero servers to patch, scale-to-zero for spiky traffic, and a bill that (in theory) tracks usage instead of provisioned capacity. For an early-stage product with unpredictable load, that's a genuinely good trade. We shipped three client backends on managed functions between 2024 and 2025 on exactly that reasoning.
Where it broke down
Cold starts compounded with fan-out. One system called out to four downstream services per request. Each cold function added 200–400ms; on a bad rollout, the p95 latency chart looked like a heart monitor.
export async function handler(req: Request) {
const [user, billing, entitlements, usage] = await Promise.all([
fetchUser(req), // cold: +280ms
fetchBilling(req), // cold: +310ms
fetchEntitlements(req), // cold: +260ms
fetchUsage(req), // cold: +240ms
]);
return respond({ user, billing, entitlements, usage });
}The bill inverted. Past a steady baseline of traffic, per-invocation pricing costs more than a right-sized always-on instance. One client's function costs crossed their previous EC2 bill around month four of sustained growth — the exact point scale-to-zero stopped mattering, because they never scaled to zero.
Debugging got slower, not faster. Distributed tracing across a dozen small functions is a real discipline you have to build deliberately. We hadn't budgeted for it on any of the three projects, so incident response time went up, not down.
What we moved to
A small fleet of long-lived containers behind a load balancer, with the same code — we kept the handler signatures nearly identical and swapped the entrypoint. Connection pooling alone (something serverless makes awkward with per-invocation cold connections) cut our database latency in half.
const pool = createPool({ min: 4, max: 20 });
export async function handler(req: Request) {
const [user, billing, entitlements, usage] = await Promise.all([
fetchUser(req, pool),
fetchBilling(req, pool),
fetchEntitlements(req, pool),
fetchUsage(req, pool),
]);
return respond({ user, billing, entitlements, usage });
}What we'd do differently
Serverless still made sense for two of the six services in that same system — the ones with genuinely spiky, low-frequency traffic (webhooks, scheduled jobs). We'd split the decision per service from day one instead of picking one hosting model for the whole backend. That's the actual lesson: serverless isn't a verdict on your whole architecture, it's a per-workload decision, and treating it as an all-or-nothing platform choice is what cost us the migration later.
If you're mid-decision on this trade-off for a system with real traffic behind it, it's worth modeling the crossover point before you commit — not after your bill tells you.