A business AI agent is not a language model with a long prompt. It is a closed loop: the model receives a goal, chooses an action, calls an external tool, reads the result and decides on the next step — until the goal is met, until it gets stuck, or until the step budget runs out. Under the hood there are only four components: model, tools, memory and control. In McKinsey’s November 2025 survey (1,993 respondents across roughly 105 countries) about 62% of organisations were experimenting with agents but only about 23% had scaled them, and Gartner forecasts that over 40% of agentic AI projects will be cancelled by the end of 2027. That gap almost never sits in the model.
What actually separates an agent from a chatbot
The difference is not answer quality. It is who decides the order of operations. In a chatbot the human decides: they ask, they get text back, and they decide what to do with it. In classic automation the developer decides in advance: the process is written as a flowchart and every branch was defined at build time. In an agent that decision moves to run time — the model picks which tool to call, in what order, and when to stop.
This is the only difference that matters, and it is also the source of every risk. Once the order of operations is not fixed in advance, you cannot test the system the way you test ordinary software, and you cannot rely on yesterday’s successful run following the same path today.
| Property | Chatbot | Classic automation | Agent |
|---|---|---|---|
| Who sets the order of operations | The user | The developer, at build time | The model, at run time |
| System access | Usually none | Fixed and predefined | Dynamic, through a tool layer |
| Behaviour on failure | Returns a wrong answer | Stops and reports an error | Keeps going and compounds the error |
| What you test before launch | Phrasing quality | Known paths | Outcome distribution across runs |
The key line: automation that breaks stops. An agent that breaks keeps working. That is why control is a component of the system, not something you bolt on at the end.
The execution loop: plan, act, observe
Regardless of vendor, language or framework, every agent runs the same loop. The model receives the goal and the current state of the world, produces a decision about the next step, issues a call to an external tool, gets a result back — and then runs itself again with that result as part of its context. The loop ends when the model declares it is done, when a tool returns a blocking error, or when an external mechanism cuts it off.
The important word here is observe. The agent knows nothing about the world except what tools hand back to it as text. If the CRM returns an empty field, then as far as the agent is concerned the customer genuinely has no phone number. Observation quality is therefore a first-order engineering variable, not a technical detail.
| Loop stage | What actually happens | Typical failure mode |
|---|---|---|
| Goal intake | The goal becomes initial context alongside system instructions | A vague goal that produces a different path on every run |
| Step planning | The model selects a tool and its arguments | Right tool, wrong argument — the single most common failure |
| Execution | The tool layer runs the call against the real system | An over-broad permission that allows a destructive action |
| Observation | The result returns into context | A long payload that pushes earlier critical information out |
| Decision | Continue or stop | An endless loop that burns budget up to the ceiling |
The four components every agent is built from
Break an agent apart and you get four components. Three of them are not the model, and that is where most of the engineering work and nearly all of the failures live.
| Component | Its job | What breaks when it is weak |
|---|---|---|
| Model | Produces the decision about the next step | Elegant phrasing with faulty reasoning; upgrading the model does not fix the other components |
| Tools | The agent’s only interface to reality | An imprecise tool description leads to misuse even with an excellent model |
| Memory and context | What the agent knows at the moment of decision | Critical information pushed out, or polluted context that drags the agent down the wrong path |
| Control | Limits, permissions, step budget and approval points | No way to stop a mistake before it touches a customer or a system |
In the projects we see, the first reaction when something does not work is almost always to swap the model. In most cases that is not the problem. Precise tool descriptions, structured output instead of free text, and a clear stopping condition improve success rate more than any model upgrade.
The tool layer: what MCP actually changed
Until recently every connection between a model and an enterprise system was written from scratch: a bespoke integration for each combination of model and system. Model Context Protocol replaced that with a uniform layer — a server that declares tools, and a client that calls them.
The stable revision of the specification, dated 28 July 2026, is the largest revision of the protocol since it launched. Protocol-level sessions and the Mcp-Session-Id header were removed from the Streamable HTTP transport; the initialize handshake was dropped; and tool, resource and prompt lists no longer vary per connection, which makes them cacheable. Alongside that came multi round-trip requests, header-based routing, a formal extensions framework and hardened authorization. A server that needs state across calls now receives it through an explicit server-minted handle passed as an ordinary tool argument.
Why this matters to people who do not write code: a stateless protocol can be deployed behind a standard load balancer, scaled horizontally and monitored with ordinary tooling. In other words, the tool layer stopped being a research project and became infrastructure you can budget, operate and audit. If you plan to run that layer on your own infrastructure, the full cost model is in our guide to running agents on your own hardware.
Context is a budget, not a storage bin
The most common design mistake is treating the context window as storage. It is not. It is a budget consumed on every step, and you pay for it again on every call. An agent that runs twenty steps pays for its own history twenty times.
Three design decisions follow. Return compact structured output from tools rather than whole documents. Compress history into a state summary after a fixed number of steps. Retrieve enterprise knowledge at run time instead of preloading it. The financial side of those decisions is covered in our guide to reducing inference cost.
Where this breaks in practice
Three causes explain most of the failures we encounter.
Compounding errors. In a loop, a mistake at step three does not stay at step three. It enters the context and from there feeds every decision that follows. The visible failure therefore appears far from where it was created, which is why debugging an agent requires tracing a full run rather than inspecting a single answer.
Optimistic measurement. The common success metric asks whether the agent succeeded once under favourable conditions. In production what counts is stability: same task, one hundred runs, how many finished correctly. The gap between those two numbers is the gap between an impressive demo and a system you can rely on. Closing it is the subject of our guide to agent evals.
Agent washing. Gartner estimates that out of thousands of vendors marketing agentic capability, only around 130 offer the real thing; the rest are rebranding existing assistants, RPA and chatbots. The question that filters fastest in a sales call: which tools can this agent invoke, and who decides when it invokes them.
The same Gartner note contains the other side of the story: by 2028 roughly 15% of day-to-day work decisions will be made autonomously and 33% of enterprise software applications will include agentic capability, up from under 1% in 2024. The direction is clear; what breaks is the timeline and the expectations.
A checklist before your first agent
- Define one task that ends. A task with an unambiguous completion condition, not an open-ended area of responsibility.
- Count the tools. If a first agent has more than ten tools, you have probably mapped a job title rather than a task.
- Write tool descriptions like contracts. What goes in, what comes out, what happens on failure, and when the tool must not be used.
- Set a step budget and a token budget. A hard ceiling that stops the loop even when the model is convinced it is on the right track.
- Mark the irreversible actions. Sending email, charging, deleting, updating a customer-facing status — all of these need a human approval point.
- Define permissions before you write a prompt. The permission model is what limits damage, not the wording of instructions. The full treatment is in our guide to agent boundaries.
- Record every run. Without a full log of calls and results there is no debugging and no improvement.
If you would rather start from a structured readiness assessment, begin with our digital strategy diagnostic. If you already know what you want to run, the implementation stage lives on the deployment page.
Frequently asked questions
What is the difference between an AI agent and ordinary automation?
In automation the order of operations is fixed in advance by a developer. In an agent the model chooses its actions at run time based on the goal and on what it observes. Automation is therefore tested along known paths, while an agent is tested by the distribution of outcomes across repeated runs.
Do I need an especially strong model to build an agent?
In most cases no. Success rate is determined more by the quality of tool descriptions, context management and stopping conditions than by model choice. A stronger model hides design problems for a while, but it also makes every step in the loop more expensive.
What is MCP and why does everyone talk about it?
Model Context Protocol is a uniform protocol for connecting models to tools and data sources. The stable revision dated 28 July 2026 made it stateless: no more protocol-level sessions, cacheable tool lists, and hardened authorization. In practice this lets you deploy a tool layer as ordinary infrastructure instead of as a one-off project.
How long does it take to get a first agent into production?
A single task with three to five tools and a human approval point usually reaches production in four to eight weeks. Most of that time does not go into building the loop; it goes into clean system integration, permission design and building the test suite.
What is the sign that an agent is the wrong fit for a task?
If you can write the process as a complete flowchart with no ambiguous decisions, you do not need an agent — automation will be cheaper, faster and testable. An agent pays off when the space of possibilities is too wide to write down in advance.