Every second digital agency today says the same sentence: "we do AI". Some of them really do. Others added the word to their homepage and hoped nobody would ask follow-up questions. The problem is that the gap between the two only becomes visible months later – after the budget is spent and the system still does not do what you needed it to do.
The good news is that you do not need to be a technical expert to tell them apart. You need five questions. Ask them in the first meeting, listen carefully to how the answers are phrased, and most of the risk disappears before you sign anything.
Question 1: What results have you achieved for clients like me?
This is the opening question, and it is the one that filters fastest. A serious agency answers with numbers. A vague agency answers with adjectives.
Look for concrete figures. "We saved a client 15 working hours a week" is an answer. "We improved their efficiency" is not an answer – it is a way of avoiding one.
Ask for a case study that actually resembles your business. An agency that automated workflows for a logistics company has not necessarily solved anything close to your problem. Relevance matters more than the length of the client list.
Ask about real timelines. How long did that project take from kick-off to a working system? If every answer is "a few weeks", ask which weeks, and what happened in them.
A real example of what happens when this question is skipped: an e-commerce business paid roughly 20,000 shekels and waited two months for a chatbot that, when it finally went live, could not answer the single most common question its customers asked – "when will my order arrive?". The agency was competent. It simply had no e-commerce experience, and nobody had asked.
Question 2: What exactly do I get, and what costs extra?
Most unpleasant surprises in AI projects are not technical. They are financial, and they arrive in month three. Ask for the full cost picture up front, item by item.
| Cost component | Why it matters |
|---|---|
| Monthly licence fees | The underlying AI tools and platforms are usually billed separately from the agency fee, and the amount often scales with usage. |
| Implementation and onboarding | A one-off cost that can be larger than the build itself, particularly if your existing systems need to be connected. |
| Ongoing maintenance | AI systems are not "set and forget". Someone has to monitor, correct and retrain them – and someone has to pay for that. |
| Updates and upgrades | Models change constantly. Find out whether moving to a newer model is included or quoted again as a new project. |
The practical tip: ask for a written, itemised quote covering all four rows. Not a total. An itemised breakdown.
The warning sign: if an agency is evasive about costs at this stage, that is not a negotiating style. That is a preview of how the rest of the engagement will feel.
Question 3: Who supports me after the system goes live?
Launch day is the beginning of the relationship, not the end of it. Ask four specific things.
Do I get a dedicated account manager? Or does every request go into a shared inbox and wait its turn?
What are the response times when something breaks? Ask for the number of hours, in writing, and ask what happens outside business hours.
Do you train my team? A system nobody knows how to use is an expense, not an asset.
What happens when I need a change? Small adjustments should not require a new contract every time.
Consider a law firm that commissioned an AI system for contract analysis. The system worked well – until the relevant legislation changed and the outputs stopped being reliable. The agency, by then, was unreachable. A competing firm had signed a contract that included ongoing updates, and got its system adapted to the new legislation within a week. Same technology, very different outcome, and the difference was written into the contract on day one.
Question 4: How do you measure success?
If nobody defines success in advance, everybody defines it afterwards – and it always turns out to have been achieved. Agree on the metrics before the work starts.
Good metrics look like this: hours saved per week; an increase in conversion rate expressed as an exact percentage; a measurable reduction in customer response times; how much more your team can handle without adding headcount; and return on investment within a defined period.
Metrics that should worry you: "improving the user experience", "streamlining processes", "embedding innovation". These are not measurements. They are sentences that can never be proved wrong, which is precisely why they get used.
Question 5: What happens if it does not work?
Not every project succeeds, and an honest agency will say so. What matters is what has been agreed for that scenario.
Is there a trial or pilot period? A limited, well-defined first phase lets both sides find out whether this is going to work before the full budget is committed.
What are the exit terms in the contract? Read them before signing, not while trying to leave.
Do the data and the systems stay with you? If the engagement ends, you should keep what you paid to build.
Is the agency accountable if the results do not arrive? Ask what "accountable" means in practice – a partial refund, additional work at no cost, or nothing at all.
A good agency welcomes this question rather than deflecting it, and will usually respond by proposing a focused pilot, clear KPIs and fair terms. An agency that demands a long commitment with no realistic way out is telling you something about its own confidence in the outcome.
The bottom line
Five questions, one meeting. The agencies worth working with will answer all five with specifics and will not be offended that you asked. The rest will produce adjectives, and you will have saved yourself several months and a significant budget.
At Rainz.ai we prefer to answer these questions before the engagement starts rather than after – with concrete numbers, an itemised quote and agreed metrics. If you are weighing up an AI project for your business, talk to us and ask us all five.