The Prototype Got Cheap. The Product Didn’t.
On the widening distance between a working prototype and a production system.
The Prototype Got Cheap. The Product Didn’t.
On the widening distance between a working prototype and a production system.
Everyone is going somewhere
Three hours on Saturday afternoon, and I had a working data exploration tool allowing me to drill down into CMS public health care datasets. I sent the demo link to two friends. They said the appropriate things. What none of us said, and what I already knew by Saturday night, was that the thing was maybe a tenth of a product.
Two years ago, that same prototype would have taken a small team a week. The compression is real, and I’m not trying to talk anyone down from being excited about it. This is about what happens after the demo.
What the demo doesn’t ship
Most of the AI-makes-engineers-10x content gets the economics wrong. Building the prototype was never what cost you the calendar. Cheap in 2023, cheaper now, and cheap either way compared to what came after. The calendar got eaten by the work underneath.
In a production system, the happy path is roughly a tenth of the code. The rest is failure modes. A dependency goes slow. A user inputs something the designer never imagined. Malformed data shows up. The network blips. The database catches fire under load, the config is wrong in one region, the clock skews thirty seconds across a cluster, a feature flag is half-rolled-out, a permission check fires in the wrong order. The list has no end because new ways to fail keep getting invented.
Then there’s everything around the code: tests that actually exercise the failure modes, observability so someone notices the problem before a customer does, access controls that prevent the thing from becoming a liability. Documentation so the next on-call engineer can orient themselves at 3am. A review process structured to catch what the author missed.
Senior engineers have been making a version of this argument for twenty years. What changed is the ratio. Prototype hours collapsed while operational hours stayed the same, and suddenly the second number looks disproportionate to anyone without scars.
The hard category
The naive read goes: AI tooling makes coding faster, so all of engineering gets faster in equal proportion. Two hours to build instead of two weeks. Doesn’t the whole pipeline compress? No, and the reason is worth sitting with.
The operational work is only nominally about code. Most of it is thinking work, drawing on years of watching particular systems fail in particular ways. You need a model of your business, your customers, the shape of your risk surface, the compliance posture you inherit, the adjacent teams whose systems touch yours, and a willingness to make judgment calls with information that will never be complete.
AI tooling accelerates the work that looks like “take a specification and produce working code.” This is a huge category. There is real acceleration. But this is separate from the hard part.
The hard part is figuring out what the specification should be, given a messy real-world situation, and then what could go wrong with it, and then building in the right defenses at the right cost. That category hasn’t gotten dramatically faster. In some ways, it got harder because the prototype is so convincing so early that stakeholders lose patience with the rest of the work.
That’s the trap. When the demo is this good, this fast, the pressure to ship exactly what got demoed starts mounting inside the first week, and the pressure doesn’t know that the demo skipped the parts that take the longest.
What I’ve watched happen
Teams ship the prototype and call it done. A handful of patterns recur.
Load is where a lot of them die first. The prototype handles a hundred users and folds at a thousand. Nobody modeled the query pattern because nobody had to. The data layer gets rewritten, which nobody wanted to do before and nobody wants to do now, except now there’s also a migration attached.
Input. Someone types a character nobody planned for. A very long string. An empty field the happy path assumed was full. The fix is input validation all the way through the call graph, which is tedious and extensive and wasn’t in the three-hour plan.
Silent breakage is a third category. Nothing on the dashboard says anything is wrong, because there aren’t any dashboards. No alerts got wired up. Nothing traced through the calls. The first signal is a customer support ticket on a Tuesday about behavior that started on Sunday, and by the time someone figures out what changed, the team has lost most of a week.
Upstream dependencies break in their own way. One of the integrations starts returning slightly different data. The contract test that would have caught it was never built, the canary was never wired up, graceful degradation was never in the three-hour plan. The first signal comes from a revenue dashboard going flat.
The expensive category is data and permissions. The system leaks, or stores data it shouldn’t, or grants access it wasn’t supposed to grant. Nobody designed the security model because the prototype wasn’t real. The production deployment is real, and the bill for skipping that design arrives on its own timeline.
None of these issues are exotic. They’re the default failure modes of software shipped without the last 90%, and they show up more often now than three years ago, since the prototyping cost collapsed without taking the hardening cost with it.
The estimation problem
This creates a specific estimation pathology that’s going to get worse before it gets better.
A leader sees the prototype on Friday. It works. Asks, reasonably, when it will be in production. The engineer standing next to the thing that just ran wants to anchor on how long the build took. The accurate answer is that the prototype was around 10% of the work, and the production timeline needs to include hardening, testing, review, rollout, and coordination across adjacent teams. That timeline usually runs an order of magnitude longer.
The prototype looks done. What “looks done” and “is done” actually mean has drifted further apart than at any point I can remember, and most organizations still don’t have the vocabulary they need for status meetings.
If you’re an engineer, part of your job now is teaching stakeholders what the rest of the work looks like. This isn’t about protecting project slack. It’s a factual conversation about what hasn’t happened between the demo and production. A stakeholder who thinks the demo means 90% done will make budget and staffing decisions based on that number, and those decisions get harder to unwind once they’re made.
If you’re a leader, your job includes not anchoring on the demo. Treat the demo as a conviction-building artifact, useful for deciding whether to fund the next phase, rather than a measure of how close the thing is to shipping. Conflating the two costs the project weeks that nobody budgeted.
If you manage engineers directly, there’s a ratio worth watching. A team that ships a working demo in three days and a production version in three to four weeks is running a six-to-nine-times multiplier between concept and ship. That multiplier is usually a signal of a healthy bar. Someone on the team is writing the tests for the inputs the demo never saw, instrumenting the paths that go quiet under load, walking the security review down before the rollout gets blocked, and looping in the adjacent on-call before Friday. Teams that close the demo and the production version on the same timeline are usually the teams that shipped the demo and called it the production version. When your engineers slow down by six to nine times between the first working build and the final deploy, they’re holding a standard you want them to hold. Your job then is to name that out loud in staff meetings, and to give the stakeholders asking for timelines a concrete picture of what those extra weeks are paying for.
What the work actually looks like
I want to be concrete about what the operational 90% includes, because a lot of the hand-waving about “production readiness” leaves people fuzzy on what they’re actually supposed to do.
Hardening a system means walking through every dependency and defining system behavior when each one is slow, wrong, or missing. It means instrumentation so that on-call, in November, can answer “is this working right now” without guessing, and tests that exercise the failure paths instead of just the happy path from the demo. Data flows have to get mapped against the policies that govern them. Someone needs to write down enough of how the system works that the next engineer to inherit it doesn’t start discovery from scratch.
The work also lives in meetings. Sessions with security, platform, legal, ops, and whoever owns the upstream service you depend on. Review cycles that feel like friction in the moment and save the project two weeks later. A staged rollout plan that keeps the blast radius small while real traffic exposes the gaps the tests didn’t cover.
AI tooling doesn’t replace this work in any meaningful way yet. Some of it is assisted at the margins. Most of it is the same human engineering work it was three years ago.
Pushing AI into the 90%
If the argument stopped at “the operational work is still hard and AI doesn’t help,” the essay would be pessimistic and not very useful. The more interesting question is where AI tooling can start eating into the operational 90%, and what that takes.
Adversarial test generation. Not the happy-path unit tests that confirm what already works. The interesting application is adversarial: describe your system’s contracts and failure boundaries, and ask the tooling for the hundred worst things a caller could send. I’ve started feeding my interface definitions in and running that prompt. Results vary. The hit rate is high enough that it catches inputs I would otherwise have caught the first time a customer caught them for me.
Review augmentation. A lot of production judgment lives in code review, and senior engineers become the bottleneck. AI as a first-pass reviewer flags the gaps with clear names: missing timeouts, missing retry logic, error handlers that swallow silently, config that isn’t validated at startup. The human reviewer then focuses on the architectural and business-logic questions that need real context. The AI doesn’t replace the review, it raises the floor of what reaches human eyes.
Runbook drafting. Nobody wants to write runbooks. The tooling has gotten good enough that pointing it at a service, its dependencies, its alert definitions, and its recent incident history produces a draft that’s around 70% accurate. A human still has to review, fill gaps, and pressure-test the diagnostic steps. What changed is the starting point: editing a 70% draft takes less activation energy than staring at a blank page.
Failure-mode enumeration. This is the application I think has the most room to grow. You describe a system architecture and ask the tooling to enumerate the ways it breaks: slow dependencies, backed-up queues, stale caches, retries amplifying into thundering herds. The enumeration isn’t complete, because the tooling doesn’t have your traffic patterns or your operational history. What it produces is a starting checklist, and starting checklists are worth a lot when the alternative is relying on whoever in the room has seen the most outages.
Across all four applications, the pattern runs the same way. Judgment stays with humans. What the tooling changes is the activation energy: the draft already exists, and the human work starts from something concrete instead of nothing.
The failure mode to name clearly is this. Teams treat the AI-generated draft as the finished product. A test suite gets merged without review. The runbook sits in the repo untouched after generation. The failure-mode list never gets cross-referenced against actual outage data. You’ve just installed a layer of false confidence, which is worse than having nothing, because now the team thinks it’s covered when it isn’t.
The discipline hasn’t changed with the tooling. Get to the draft faster, then do the same review work you would have done without it. The part that makes the system safe happens after the draft.
The patience argument
There’s a version of this essay that scolds engineers for being too fast. That isn’t the argument I want to make.
The argument is that speed at the prototype stage and patience at the production stage aren’t opposites. They’re the same discipline operating under different constraints. The prototype stage is safe to move fast on because the blast radius is small. The production stage needs to move slower because the blast radius is every customer, and the tooling can trick you into feeling like you’re still in the prototype stage when you aren’t.
I use AI tooling constantly. I build prototypes in an afternoon that would have taken a week a few years ago. Then I spend weeks on the operational work around them, because the operational calendar didn’t compress when the prototyping calendar did, and customer tolerance for broken software stayed where it always was.
The last 10% has always been 90% of the engineering effort. What changed is the first 10% now takes an hour instead of a week, which makes the rest feel disproportionate and makes a lot of people want to skip it. Teams that skip end up with a demo running in production. It’s a different product than the one they thought they shipped, and the gap between those two products shows up in the on-call rotation a month later.
By Joshua McDonald on April 21, 2026.
Exported from Medium on August 26, 2026.
Reader discussion