Somewhere a vendor is telling you their agent "predicts demand within a few per cent," or "achieves 95% accuracy on invoices." These numbers are almost always computed on the vendor's own test data, against no agreed baseline, over no stated window. They tell you the model is good at a quiz it wrote itself. They tell you nothing about whether your firm shipped faster, or whether the work that left the building was any good.
The claim that matters for a craft or professional-services business is narrower and harder to fake. How many minutes of expert time actually left the building, and did the quality hold after they left?
That is two numbers, not one. Autonomy share, and spot-check acceptance. Everything else is decoration.
Autonomy share is simple arithmetic. Automated minutes divided by a frozen manual-time baseline, over runs closed in a window.
The word that does the work is frozen. The baseline is captured when a service request closes, from the workflow's templates. Each step's execution minutes and orchestration minutes, stored as a snapshot at that moment. The point of freezing it is precisely that it must not drift upward as you optimise. If your baseline moved every time the agents got faster, autonomy share would quietly trend toward 100% while telling you nothing. A frozen baseline is the difference between a measurement and a moving target.
The honest part is in the credit rules. Not every minute counts as automated. The scorer walks each baseline slot and credits autonomy only where it actually happened.
Automatic retries after a transient error are not charged. A provider rate limit cost no human time. The metric is trying, with unusual care, to measure displacement rather than activity. It wants to count the minutes your experts actually got back, not the minutes the machine was busy.
Displacement alone is a dangerous number, because it rewards a machine for doing lots of work badly. So the second number is spot-check acceptance. A human re-reviews a sample of outputs after they have shipped, and the share they accept is tracked.
The sampling is deliberately not random in a way anyone can game. When a fully-automated step completes, the system hashes the job's identifier and checks the result against a per-template sampling rate. Because the hash is reproducible, retrying the same job cannot dodge the review. And the review is non-blocking. The verdict is recorded, but it never changes the job or the run. The work has already left the building. The spot check exists to find out whether it should have.
Two details matter. First, a rejection requires a reason. The system refuses a bare "no," because the rejection feeds back into future checks and the training corpus. Second, the two numbers are only meaningful read together. This is where the implementation is candid in a way almost no marketing page is.
"A rising share with falling spot-check acceptance means automation is getting ahead of quality."
That single sentence is the whole discipline. Displacement up, acceptance down: you are not saving time, you are exporting rework. The machine is signing off work your experts would have rejected.
Here is the uncomfortable truth the metric's own documentation states plainly. The minutes are template estimates, not stopwatches. Operator time-logging was dropped as unreliable, so there is no reconciliation against actuals. The L2 credit is a judgement call, not a measurement. The metric knows what it is and says so.
That candour is exactly what you should demand from any vendor, and exactly what a vendor should demand from you. The reason most firms cannot tell you their autonomy share is not that the maths is hard. It is that they have never written down how long the manual process actually took. That missing number is the baseline.
So the first, and best, question to ask any AI vendor is not "what is your accuracy?" It is: walk me through your baseline. How was the manual time measured? When was it frozen? What counts as automated, and what gets charged back when a human has to step in? And then: show me the spot-check acceptance next to the autonomy share, on the same chart, over the same window.
A vendor who can answer that already knows the number cannot be trusted on its own. A vendor who cannot has never actually measured the thing they are selling. That question, "where is your baseline?", is the discovery exercise that turns a demo into a paid pilot, because answering it requires looking at your actual process, your actual minutes, your actual rework.
The metric worth reporting is not "the model is X% accurate." It is two numbers read together.
Report them on the same chart, over the same window, with the baseline's limitations printed underneath. If autonomy share is rising and acceptance is holding, you have displaced real work. If the two diverge, you have found automation running ahead of quality, and you have found it before a client did.
That is the entire job of AI governance in a professional-services firm. Make the displacement real, and make the quality check late enough to be honest. Everything a model "predicts within X%" is a claim about the model. These two numbers are a claim about your business.
AI can do the legwork. It can't do your judgment.
Gathering and summarising information is now a commodity. The judgement layer is the real moat.
How a step earns the right to run unattended.
How a workflow step earns promotion to unattended operation, and what drift is designed to pull back to human approval.