Build versus buy is not a binary question. There are five routes: a frontier model API, an open-weight model on a managed endpoint, a fine-tuned open-weight model, a model trained from scratch, and the option most teams forget — not solving it with an LLM at all. Only three lenses decide between them: cost at projected volume, the capability ceiling the task demands, and regulatory exposure. For most organisations the practical answer is to start by buying, build an abstraction layer from day one, and revisit building only once real usage data exists.
Five routes, not two
The usual framing of the debate produces poor decisions because it hides three intermediate options, which is where most of the value sits.
| Route | What it means in practice | When it is right |
|---|---|---|
| Frontier model API | Calls to a commercial provider, no infrastructure of your own | Broad task, moderate volume, team without ML depth |
| Managed open-weight model | Open weights on an infrastructure provider’s endpoint | Need for version control and predictable cost |
| Fine-tuned open model | Further training on your data for a narrow task | Narrow repetitive task, well past ten thousand labelled examples |
| Model from scratch | Full training run, dedicated research team | Almost never. Only if the model is the product itself |
| No LLM at all | Rules, search, a classical statistical model | When the task is deterministic and fully specifiable |
The three lenses that decide
Cost at projected volume. Not today’s volume, but the volume you are planning for a year out. The quick calculation: units of work per month multiplied by cost per unit on the buy route, against the fixed engineering and infrastructure cost of the build route. We set out the method in our article on the total cost of ownership of an LLM feature.
Capability ceiling. If the task requires multi-step reasoning, a rare language or broad world knowledge, a small fine-tuned model will not get there no matter how much you train it. If the task is classification into seven categories, a frontier model is waste.
Regulatory exposure. When data must stay inside a jurisdiction or a closed network, buy routes narrow immediately. This is the only lens that can eliminate a route outright, which is why it should be checked first rather than last. The relationship between data types and permissions is set out in our article on classifying enterprise data before it reaches an LLM.
When fine-tuning genuinely pays
Fine-tuning is the route most teams choose too early. It becomes worthwhile when all of the following hold together, not some of them:
- The task is narrow, repetitive and well defined, and does not change every quarter.
- You hold many thousands of good-quality labelled examples, from your own production rather than a generic corpus.
- API cost or latency has actually become the binding constraint, not a theoretical concern.
- Someone on the team will maintain the model, measure drift and retrain when the distribution shifts.
If any condition is missing, fine-tuning adds a permanent maintenance cost without returning it. The pattern that works is the opposite: stay on an API, collect production data systematically, and return to the question after roughly ninety days of real usage, when you have numbers instead of estimates.
The abstraction layer: the one decision to make immediately
You do not need to choose a route on day one, but you do need to decide not to commit. A thin abstraction layer between your code and the model provider is the cheapest insurance in the project:
| What the layer holds | Why it matters |
|---|---|
| A single interface for model calls | Switching provider is a one-file change, not a rewrite |
| Routing by task type | Lets you send cheap tasks to a cheap model without touching logic |
| Uniform logging of input, output and cost | Without it there is no data to decide on three months from now |
| Output validation | Lets you swap models without changing the output contract |
The costs that are not in the spreadsheet
A build-versus-buy comparison done in a spreadsheet tends to weigh licence price against compute cost and miss three costs that actually decide the outcome: time to market, the opportunity cost of a team building instead of selling, and maintenance over years. Publications through 2026 report that organisations which built custom AI capability without first validating it against an off-the-shelf option lost substantial timeline before discovering the direction was wrong.
From the other side, buying forever creates single-vendor dependency and removes the organisation’s ability to improve what makes it distinctive. The practical balance is almost always hybrid: buy the foundation, build the layer above it where the competitive advantage lives — the data, the retrieval, the business rules and the interface — and not the model itself.
The recommended sequence
- Define the unit of work and the success criterion before choosing any technology.
- Check the regulatory lens. If it eliminates routes, everything else is moot.
- Run a proof of concept on an off-the-shelf option even if you are convinced you will build. It is cheap and it creates a baseline.
- Build the abstraction layer and the logging on day one.
- Return to the decision after ninety days with real cost per unit, success rate and task distribution.
That accompaniment, from defining the unit of work through to choosing the route, is exactly what sits within our areas of expertise.
Frequently asked questions
What is the practical difference between fine-tuning and retrieval from a knowledge base?
Retrieval adds current information into the prompt at run time; fine-tuning changes the behaviour of the model itself. For organisational knowledge that changes, retrieval is almost always the correct choice. Fine-tuning suits changes of style, format or a narrow task, not the injection of facts.
How many labelled examples does fine-tuning actually require?
The accepted order of magnitude is many thousands of quality examples for a narrow task. The determining factor is not quantity but quality and representativeness: a thousand clean examples from your own production are worth more than tens of thousands of synthetic ones.
Does an abstraction layer not add needless complexity?
A thin layer that only unifies the call, the logging and the validation costs a few days of development. The cost of not having one appears on the day you need to switch provider, route cheap tasks to a cheap model, or explain where an invoice came from.
When is training a model from scratch ever right?
Almost never, unless the model itself is the product you sell, or you hold uniquely large proprietary data in a domain general models do not cover. For most organisations the cost exceeds the benefit by orders of magnitude.
How do you know it is time to move from buying to building?
When three conditions hold together: the monthly cost of buying has passed the annual engineering and maintenance cost of the alternative, the task has stabilised and stopped changing, and you have production data sufficient to train and measure. Two out of three is not enough.