The position, stated up front
Production has no such person. What looked like a high success rate on curated cases becomes an unknown success rate on everything, and nobody can say which because there is no eval set. The project does not fail. It just never gets approved, quarter after quarter.
Gap 1 · The inputs were chosen
Someone assembled the demo set. Not maliciously, just naturally: they picked cases that represent the process, which means readable ones, complete ones, recent ones.
Production supplies its own inputs, and its distribution includes everything the assembler skipped. Scanned documents at an angle. Duplicates. A different language. An email with the actual request in the third forwarded message. A form where somebody typed the answer into the wrong field for two years and everyone downstream knows to check.
Gap 2 · Nobody set an error budget
Ask a room what failure rate is acceptable for this process today and you will get silence, then a guess. Nobody knows, because nobody measured what humans do on the same task.
That silence has a consequence. Without an agreed number, the implicit standard becomes perfection, and the first visible mistake in production becomes evidence that the system does not work. Meanwhile the human process it replaced has its own error rate, unmeasured and forgiven.
Gap 3 · Quality is an anecdote
This is the gap that blocks everything downstream. Without a scored eval set, “is it better than last week” has no answer, so no change can be approved, so the system freezes at pilot quality.
It also makes model upgrades terrifying. A provider ships a new version, and you have no way to know whether your system improved or regressed, so you either pin to a deprecated model or upgrade and hope.
Gap 4 · The pilot ran on a laptop’s permissions
The demo used a data export, someone’s personal API key and read access to everything. Production needs scoped service credentials, permissions that match the requesting user, rate limits, secret rotation, network boundaries and an audit trail per action.
Two things bite here. The first is time: access provisioning in a regulated organization is measured in weeks, not days, and it is never on the project plan. The second is correctness. A retrieval system that ignored permissions during the pilot will happily quote a document the asker should not see, and that is the kind of defect that ends a program rather than delaying it.
Gap 5 · There was no exception path
In the pilot, cases the system could not handle went back to the person running the pilot, who dealt with them and moved on. Nobody counted them, timed them or noticed they had become a queue.
In production those cases need a named owner, a route, a response time and enough attached context that the human starts from a prepared case rather than a raw one. The exception path is not a fallback. On most processes it is where the remaining human cost lives, and it deserves as much design as the happy path.
Gap 6 · Load, latency and the real cost shape
Pilots run at one request at a time, on a weekday, with a patient person watching. Production runs at concurrency, in bursts, at month-end, against providers that rate-limit.
Three things change. Latency at the tail, which is what users actually feel, is far worse than the average you measured. Retries multiply your inference bill and are usually absent from the pilot’s cost figure. And the cost per task at real volume includes the review minutes the pilot’s enthusiastic human donated for free.
Gap 7 · Nobody’s job
The pilot had a champion. Production needs an owner, which is a different thing.
Systems drift. Corpora rot as documents are superseded. Providers deprecate models. Volumes shift, and the question distribution shifts with them. Somebody has to watch the eval scores, re-run the suites against new model versions, deprecate stale documents and decide when a boundary widens. If that is nobody’s job, the system degrades quietly and gets switched off eighteen months later with the explanation that AI did not work for us.
Why “let’s just do a quick POC” makes this worse
The instinct is reasonable: prove value cheaply before committing. The trouble is what gets cut to make it quick, which is always the same list. No eval set, no real credentials, no exception path, no owner.
So the quick proof of concept proves the one thing nobody doubted, which is that a modern model can do the task on a good day. It leaves every genuine risk untested, and it produces a demo that creates pressure to deploy something whose behavior on real inputs is unknown.
A better pilot costs slightly more and is designed to fail. Random inputs, real credentials, a scored set, an error budget, and the exception path built before the happy path. If it clears the bar under those conditions, the production build is largely mechanical. If it does not, you have learned that in three weeks for the price of three weeks.
The alternative framing is simpler still: run the assessment before the pilot. That is what the AI readiness audit is, and typically 3-10 days of it removes several of these gaps by rejecting the wrong first process outright.
What a useful pilot report says
Most pilot write-ups are a demo transcript with a recommendation attached. A report that can actually be acted on has six parts, and it fits on two pages.
The sample, described honestly: how many cases, drawn how, over what period, and what was excluded. If anything was excluded, say why, because that sentence is the one a skeptical reader will look for first.
The score by case type, not a single headline number. A system at 88% overall might be at 97% on the common category and 40% on the two that generate your complaints. One of those pictures leads to a good decision.
The failure taxonomy. Every miss, sorted by cause: missing knowledge, wrong retrieval, bad instruction, model limitation, broken input. This is what tells you whether the fix is a week of retrieval work or a different project altogether.
The cost per completed task at pilot volume, with the review minutes included and the assumptions shown.
The exceptions, counted and timed. How many cases the system refused or escalated, and how long a human spent on each. This is the number that predicts the operating cost after launch.
And the list of what remains unproven. Load, permissions, edge cases nobody sampled, and any behavior that has not been tested against a hostile input. A report that claims nothing is unproven has not looked.
AI pilot to production: the readiness checks
Seven checks, one per gap. If you can answer all seven, the build is a schedule rather than a gamble.
None of these are about the model, which is the point. The model is the one component you did not build and cannot fully control, so everything around it has to be built properly. That is also the honest reason some projects should not proceed: if a process has no ground truth and no owner, no amount of engineering makes it deployable, and we would rather say so in week one.
If the work turns out to be deterministic after all, the right answer may not be an agent at all. That comparison is in AI agents vs RPA, and the definition of what you are actually buying is in what is an AI agent.
| Gap | The check |
|---|---|
| Inputs | The evaluation ran on a random sample of real cases, including the ugly ones |
| Error budget | An agreed pass rate per case type, set before the demo, with a defined response when missed |
| Evaluation | A scored suite from historical cases, gating every change |
| Access | Scoped service credentials and permission-aware retrieval, from week one |
| Exceptions | A named owner, a response time, and full context attached to every escalation |
| Load | Tail latency and cost per completed task, measured at target volume |
| Ownership | A named owner and an agreed cadence for evals, drift and boundary reviews |
The short version
- Pilots succeed on a hidden subsidy: a human choosing inputs, absorbing errors and supplying judgment.
- Curated inputs are the original sin. Sample randomly, and keep the cases that make you wince.
- Without an agreed error budget, the implicit standard is perfection and the first mistake ends the project.
- Without a scored eval set, no change can be approved, so quality freezes at pilot level.
- Permissions, exception paths, tail latency and real cost per task are all discovered late by default.
- If running the system is nobody’s job, it degrades quietly and gets blamed on the technology.
Keep reading
- What is an AI agent? (And why it isn’t a chatbot) - The definition, the loop, and what an agent needs that a prompt does not.
- AI agents vs RPA: when rules stop being enough - Whether the process in front of you needs judgment at all.
- Questions, answered - Cost, safety, timelines and what stays human.
