MIT research from 2026 found that roughly 95 percent of enterprise GenAI pilots produce no measurable return in the income statement, and about 88 percent never reach production at all. The cause is almost never model quality. Around 80 percent of the work between a successful pilot and a working system is data engineering, governance, workflow integration and measurement — precisely the parts a typical pilot skips. Organisations that do get through report about 3.70 dollars back for every dollar spent, starting near 41 percent in year one and reaching about 124 percent by year three.
Why almost no pilot reaches the income statement
The 95 percent figure is misleading if you read it as “the models did not work”. In most cases the model worked fine. What failed is that the pilot proved one thing while the organisation needed to decide something else.
A typical pilot tests whether the model can perform a task. The decision sitting behind it is whether the business process will change, who will stop doing what, how it gets measured every month, and who is accountable when the system is wrong. Those are two entirely different questions, and the second one is the one that shows up in the accounts.
This is also where the well-known 80/20 split comes from. The part that looks like the project — choosing a model, writing prompts, running a demo — is about a fifth of the work. The remaining four fifths are cleaning and exposing the data, permissions and information classification, connecting to the tools the team already works in, and building the measurement. A pilot that touched none of those four did not reduce risk. It only pushed the risk forward.
A test to run before you start. Write in one sentence the decision that will be made at the end of the pilot, and the number that will settle it. If you cannot write that sentence, what you are planning is a demonstration, not a pilot. Demonstrations are legitimate — just do not budget one as a pilot and do not expect a return from it.
What pilots measure versus what they should measure
This gap repeats in almost every organisation, and it is usually the reason the pilot succeeded and the project died.
| What actually gets measured | Why it is not enough | What to measure instead |
|---|---|---|
| Model accuracy on a test set | The test set was built from cases you thought of, not from real traffic | Success rate on a random sample of last month of real traffic, including the strange cases |
| Time saved, as users estimate it | Self-reported estimates inflate consistently and appear in no budget line | Measured end-to-end cycle time, before and after, on the same class of task |
| Satisfaction in a survey | Measures enthusiasm for novelty, which decays after about six weeks | Week-six return rate among users who used it in week one |
| Token cost during the pilot | Reflects small volume and convenient cases | Cost per unit of work at full volume, including retries, retrieval and human escalation |
| “The team loves it” | Does not separate a nice-to-have from something they cannot work without | What happens when you switch it off for a week. It is the only test that cannot be faked |
The last row sounds flippant but it is the strongest one on the list. A tool you can turn off for a week without a single complaint will not survive the annual budget, and there is no point investing another quarter in it.
Four layers of ROI, and only one of them counts
When a chief executive asks what the return is, there are four possible answers and they are not equivalent. Telling them apart is what separates a pilot that gets refunded from one that gets closed.
| Layer | Example | Appears in the accounts | When it is legitimate |
|---|---|---|---|
| Hours saved | Each agent saves 40 minutes a day | No. The hours did not disappear, they moved to another task | Only when you can show where the hours went and what came out of them |
| Capacity released | The same team handles 30 percent more volume | Indirectly, as growth that required no hiring | When there is real demand for that capacity |
| Direct cost removed | Fewer contractor hours, one fewer external tool, less overtime | Yes, immediately | Always. This is the easiest layer to defend |
| Revenue added | More qualified leads, shorter sales cycle, lower churn | Yes, with a one to two quarter lag | When there is a control group, or at least a clean historical baseline |
Most pilots report the first layer and hope it will be read as the third. Do the opposite: build the pilot around the third or fourth layer even if the scope is far smaller. A proven 40,000 dollars of annual saving beats a theoretical million.
A six-week pilot you can actually decide on
The structure that works is shorter than most organisations plan and demands more up front.
Week 1 — define the decision and the baseline. The decision sentence, the number that settles it, and a measurement of the current state on exactly that metric. Without a measured baseline there is nothing to compare against at the end, and this is, surprisingly, the step most often skipped.
Weeks 2 to 3 — the cheapest version that answers the question. Buy before you build. Market data shows that starting the proof phase with an off-the-shelf component produces 3.2 times higher ROI, and that building before validating wastes about 14 months on average. We covered this in build or buy AI capability.
Weeks 4 to 5 — real traffic, real users. Not a lab. A defined group working with the tool in their daily job, including the ugly cases and including the escalations.
Week 6 — decide. Expand, narrow or close. All three outcomes have to be equally legitimate on day one, otherwise the result is known in advance.
The measurement budget is not optional. Allocate at least 15 percent of the pilot budget to measurement itself: collecting the baseline, weekly manual quality sampling, and tracking cost per unit of work. A pilot without a measurement budget ends in an argument about impressions, and an argument about impressions always resolves in favour of the status quo.
When to kill a pilot, and why that saves money
Closing early is not failure, it is a return mechanism in its own right. Four signals justify stopping before week six.
- Success rate on real traffic is materially below the internal test set, and the gap does not close after two rounds of fixes. This is almost always a sign that the data is not in a state that supports the task.
- Cost per unit of work at full volume exceeds the human alternative. Pricing, caching and routing can change that, but if the gap is a multiple rather than tens of percent, the architecture is wrong, not the pricing.
- Users drift back to the old process without being asked to. This is the earliest and most honest signal available.
- Nobody will own the process after the pilot. With no owner, nothing reaches production, however good the numbers look.
An organisation that closes two pilots a quarter after six weeks instead of dragging them for a year frees budget for the third attempt — and that is precisely the mechanism separating the organisations reporting 3.70 dollars per dollar from everyone else.
Where the gap between 41 percent and 124 percent comes from
Those two numbers describe the same organisations at different points in time, and the difference between them is not a better model. It is three boring things.
The first is cost. The identical capability becomes materially cheaper with caching, batching and routing between models — stacking all three cuts an inference bill by 60 to 80 percent without changing the user experience at all. In year one you pay full price because you are measuring rather than optimising.
The second is scope. The first use case solves one problem. By year three the same infrastructure, the same governance and the same information classification serve five use cases, and the fixed cost divides. This is why investment in the data layer looks far too expensive in year one and looks obvious by year three.
The third is the process itself. In year one the tool is added to an existing process, so it adds cost before it removes any. Only when the process is rewritten around it — steps disappear, approvals merge, roles change — does the large return appear. Organisations that never reach that stage stay near 41 percent and conclude that AI did not pay off.
All three rest on the same foundation: a full cost model for the feature, which we set out in what an AI feature really costs, and pricing connected to that cost structure, which we covered in pricing a model-backed product. If you are scoping a pilot right now, define the acceptance criteria in the same meeting where you define the decision.
Building the measurement and decision framework around a pilot is part of the financial and growth planning work we do with clients.
Frequently asked questions
Why do 95 percent of AI pilots fail?
MIT research from 2026 found that about 95 percent of enterprise GenAI pilots produce no measurable return in the income statement and roughly 88 percent never reach production. The main cause is not model quality but that the pilot tested technical capability instead of testing a change to the business process. Around 80 percent of the work between pilot and production is data engineering, governance, integration into existing tools, and measurement — the parts most pilots skip entirely.
How long should an AI pilot run?
Six weeks is enough to decide and is usually better than six months. One week to define the decision and measure the baseline, two weeks for the cheapest version that answers the question, two weeks on real traffic with real users, and one week to decide. A longer pilot does not produce more certainty. It only makes closing more expensive and politically harder.
How do you measure ROI on an AI project?
Separate four layers: hours saved, capacity released, direct cost removed, and revenue added. Only the last two appear in the income statement. Measure the current state on exactly the same metric before you start, compare cost per unit of work at full volume including retries and human escalation, and allocate at least 15 percent of the pilot budget to measurement itself.
When should you kill a pilot early?
When any of four signals appear: success rate on real traffic is materially below the internal test set and does not close after two rounds of fixes, cost per unit of work is a multiple of the human alternative, users return to the old process on their own initiative, or no one will own the process after the pilot ends. Closing quickly frees budget for the next attempt, which is the real return mechanism.
Why does return rise from 41 percent in year one to 124 percent by year three?
Three reasons, none of them about the model. Inference cost falls 60 to 80 percent once caching, batching and model routing are stacked. Fixed infrastructure cost divides across more use cases running on the same governance and the same data. And most importantly, the business process is rewritten around the tool instead of the tool being bolted onto an existing process — until that happens, the tool adds cost before it removes any.
Continue reading
Why 88% of AI Pilots Never Ship – And a Five-Dimension Readiness Scorecard – ROI assumes the pilot reaches production. Score the five dimensions that decide whether it will.