How to measure ROI on an AI project: baseline, pilot and beyond

“Is it worth it?” is the first question every owner and finance director asks about AI, and the one most projects cannot answer. Not because the value is missing, but because nobody measured how things worked before the pilot started. This guide gives you a simple way to measure ROI on an AI project in three phases: before, during and after the pilot.

Start with a simple formula

You do not need a complex model. For most SME projects this is enough:

ROI = (value gained − total cost) ÷ total cost

• Value gained — hours saved × the loaded hourly cost of the people involved, plus extra revenue, plus costs avoided (errors, rework, outsourcing, overtime).

• Total cost — licences or usage fees, setup and integration, internal time spent on the project, training, and ongoing maintenance.

Be precise about what “return” means. For example, Cabana Life’s AI sales assistant generated $4,713.70 in sales over 30 days on $647 in platform cost, which the vendor reports as a roughly 7.3x return. That is a useful signal, but it compares revenue with software cost. It does not subtract the cost of the goods sold or the team’s time. When you build your own business case, decide up front whether you are measuring revenue, gross margin or time, and stick to it.

Phase 1: Before the pilot — set the baseline

Measure the current process for two to four weeks before you change anything. Pick one primary metric and two or three guardrail metrics.

• Volume — how many emails, invoices, quotes or reports per week.

• Time — minutes per item, from start to finish.

• Speed — how long the customer or colleague waits for the result.

• Quality — error rate, rework rate or complaints.

• Cost — people time, outsourcing and tools currently used.

• Outcome — the business result you care about, such as conversion rate or days to collect payment.

A one-line baseline template

For each metric, write: what it is, how you measure it, today’s value, the target, and who owns it. For example: “Average first-response time on customer emails, measured from the help desk, currently [X] hours, target [Y] hours, owner: customer service lead.”

Phase 2: During the pilot — measure what really happens

A pilot is an experiment, so measure it like one. Track these every week:

• Adoption — what share of the work actually goes through the AI tool.

• Time per item with AI — including the time people spend reviewing and correcting the output.

• Quality — review a sample every week and count errors against your baseline.

• Edit or override rate — how often people rewrite or reject what the AI produced.

• Cost per item — usage fees and licences divided by volume.

• User feedback — a two-question pulse: is it saving you time, and do you trust the output?

If you can, compare against a control: half the team with the tool and half without, or alternating weeks. It is the easiest way to separate the effect of AI from a quiet month.

Hidden costs to include

• Review time. If every output needs ten minutes of checking, that time belongs in the calculation.

• Prompt and knowledge upkeep. Someone has to keep instructions, product information and policies current.

• Integration and IT time, even for “no-code” tools.

• Training and the temporary slowdown while people learn.

Phase 3: After the pilot — decide, then keep measuring

At the end of the pilot, compare against your baseline and make a clear call: scale, adjust or stop. If you scale, keep a light monthly review:

• ROI using the same formula and the same definitions as the pilot.

• Quality drift — results can slip as products, prices or policies change.

• Adoption — usage often falls once the novelty fades.

• Cost per item — this can move a lot, in both directions.

Be honest about time savings. Hours saved only become money if the time goes to something valuable: more sales calls, faster invoicing, less overtime or growth without new hires. Write down where the freed time will go before you count it.

The cost side keeps moving

AI model prices are falling quickly, which changes the maths for many projects. Anthropic says its Claude Haiku 5.5 costs around 75% less to run on average than the previous Haiku model. OpenAI prices GPT-6 Luna at $0.10 per million input tokens and $0.50 per million output tokens.

How you build matters as much as which model you use. In an OpenAI customer story, Asana reports that it made its browser agent about 76x cheaper per run in tests by optimising the workflow (mainly better caching and history management) and moving to a newer model. On the original model, the workflow changes alone cut the cost 29x.

Three practical takeaways:

• Measure cost per task, not just the monthly bill.

• Revisit use cases you rejected as too expensive. The numbers may now work.

• Re-check prices and model options every quarter, and avoid business cases that only work at today’s price.

Common mistakes

• Starting the pilot before measuring the baseline.

• Counting time saved but ignoring time spent reviewing.

• Mixing revenue, margin and time in one “ROI” figure.

• Measuring only during the exciting first month.

• Judging success by how impressive the demo looked rather than by one agreed number.

Frequently asked questions

How long should we measure before deciding?

For most SME pilots, two to four weeks of baseline and six to twelve weeks of pilot is enough to see a real trend. Our guide on running an AI pilot in 90 days shows a week-by-week plan.

What if the benefit is hard to put in euros?

Measure it anyway, in time, speed or quality, and agree in advance what improvement would justify the cost. A clear non-financial target is better than a vague financial one.

Should we include soft benefits like employee satisfaction?

Track them as guardrails, but do not use them to rescue a case that fails on its primary metric.

Want help setting a baseline and running a pilot you can actually measure? See how we support AI implementation, or start with our AI roadmap workshop to choose the right first project.