The Zero-Shot Challenge — can an AI agent learn a protocol in one conversation?

The claim: an AI agent that has never heard of AX can discover, learn and speak this site’s machine language in a single conversation — no SDK, no docs, no API key.

Don’t take our word for it. Paste the prompt below into any agent that can fetch URLs — ChatGPT, Claude, Gemini, or your own — and watch.

The prompt

paste this into your agent
Fetch https://rainz.ai/llms.txt and read the "For AI agents" section at the end.
Then execute what it teaches, zero-shot:
1. GET https://rainz.ai/wp-json/ax/v1/x?s=ax1+q.svc.pr+n%3D3 and show the raw reply.
2. Decode the reply using only the protocol's own hints.
3. Follow the nxu URL from the reply (GET, show raw). If your browsing tool
   refuses a URL it found inside a page, print that URL and ask me to paste it
   back to you — a URL in my message will pass.
4. If an nxx referral to another site appears, do the same with it.
GET requests only — do not POST anything.
Finally report: every URL you fetched, each raw response, and anything that
confused you.

What success looks like

The agent receives a plain-text reply like this — offers, availability and next actions in one GET, ~80% fewer tokens than the equivalent JSON crawl:

live reply from rainz.ai
ax1 ok op=o n=3 cov=1 cb=2 nx="a.quote i=<sku>; a.lead @ POST …/intent" nxu=… nxx=…
=i:of-1|t:"AI implementation for business"|pr:quote|av:available|u:https://rainz.ai/
…

It then decodes every field correctly — because the protocol teaches its own syntax: a malformed query returns a worked example, the reply header reports cov= (how much of your query was understood), and each answer carries the next step as a literal URL.

Why step 3 sometimes needs you

⚠️

Some hosted agents (ChatGPT’s browser among them) refuse to open any URL they discovered inside a fetched page — a provenance safety rule, regardless of the URL itself. The protocol degrades gracefully: the agent shows you the next-step URL, you paste it back, the chain continues. Agents with real HTTP access (Claude, API-based agents, crawlers, MCP clients) walk the whole chain unaided.

For builders

Did your agent pass? Tell us — POST a lead through the protocol itself: it’s the a.lead action your agent just learned.

What the challenge actually measures

The zero-shot challenge is not a test of the agent. It is a test of the protocol and of the site implementing it. An agent that has never encountered AX before is given a task and nothing else, and either the specification is self-describing enough for it to succeed or it is not. That framing matters, because it means a failure is information about the design rather than a verdict on the model.

Three things are under observation. Can the agent find the entry point without being told where it is? Once it has the manifest, can it work out the structure without external documentation? And can it complete the task end to end without a human clarifying anything mid-way? A protocol that requires a tutorial to be usable has failed at exactly the thing it exists to do.

Why zero-shot is the right bar

Standards are usually evaluated by whether a competent engineer can implement them from the document. That test is far too generous for a protocol whose consumers are agents. An agent arriving at your site has not read the specification, has no prior familiarity with your conventions, and will not ask you a clarifying question. It has one pass at the material, and if the material is ambiguous it will guess.

Zero-shot performance is therefore the closest available proxy for real conditions. It also has a useful side effect: because the bar is high, it exposes exactly the kind of implicit knowledge that specification authors do not notice they are relying on. Every place an agent stumbles is a place where the document assumed something it never stated.

Running the test against your own site

You can reproduce this on your own site in a few minutes. Open a fresh conversation with any capable model, give it nothing but your domain and a task a real customer might have, and watch where it goes. Do not help. Do not correct it. The value is entirely in observing the first point at which it gets something wrong.

Use a new conversation each time, because a model that has already been corrected once is no longer running zero-shot. And vary the task: an agent asked to summarise what you do will exercise completely different parts of your site than one asked to find your pricing or check whether you serve a particular market.

How to read the results

The common mistake is to treat the score as a grade. The number matters far less than the question of where exactly the agent stopped. An agent that discovers the manifest but misreads the service structure is telling you something entirely different from one that never found the entry point at all. The first is a wording problem, the second a discovery problem, and the fixes have nothing in common.

Bear in mind too that the result is influenced by which model runs the test. Different models arrive with different prior knowledge, and an agent meeting the protocol for the first time behaves differently from one that has seen similar structures before. The meaningful comparison is therefore not between your site and someone else’s, but between your site today and your site last month, on the same model and the same prompt.

From test to improvement

The exercise earns its time only if it produces a list of actions. Record the precise point at which the agent went wrong, change one thing, and run it again. Changing several things at once may improve the score but will not tell you what caused the improvement. Three or four iterations are usually enough to move a site from one where agents get confused to one where they finish the task on the first attempt — which is, in the end, the only result that matters commercially.

What a failed run tells you

A run that ends in failure is more useful than one that succeeds, provided you record where it stopped. Failures cluster into four kinds. Discovery failures mean the agent never found an entry point — the manifest is missing, misplaced, or blocked. Comprehension failures mean it found the manifest but could not work out what the fields meant, which points at naming rather than structure. Navigation failures mean it understood the structure but could not reach the content, usually because a linked page redirects or requires script execution. Completion failures mean it got everything and still produced the wrong answer, which is almost always a content problem rather than a protocol one.

Classify the failure before you change anything. The instinct to fix the last thing you saw go wrong is strong and usually misdirected: an agent that gives a wrong answer at step four often did so because of an ambiguity introduced at step one.


© 2026 Rainz.ai · AX Protocol
OverviewDocsSupportTerms of ServicePrivacy PolicyRefund PolicyRainz.ai