Back to blog Playbooks

How to Choose the First Process for an AI Agent Pilot

Phinite teamNov 5, 20264 min read

The most common reason an agent pilot never reaches production is decided before anyone writes a prompt. The process was chosen because it was interesting, visible or politically safe, not because it could be measured. Six weeks later the demo works, the agent is "accurate", and nobody can say whether the business got anything.

This guide is the filter we use before a pilot starts. It is short on purpose. If a candidate process fails any of the four tests, pick another one.

Test 1: Is there volume, and does it repeat?

An agent earns its keep on work that arrives every day in roughly the same shape. Ticket triage, invoice matching, claims intake, account opening, servicing requests. Hundreds of cases a week is enough to measure. A dozen a month is not: the statistics will never settle, and every single miss will look like a trend.

Volume also tells you something about the data. A process that runs at scale already has a queue, a system of record and a history. Those are the three things the agent will need to read, write and be measured against.

Test 2: Are the rules written down, and does someone own exceptions?

Agents follow rules well and judgment poorly. A good first process has a policy a person could explain in an afternoon: what makes a case in scope, which fields must be checked, what gets escalated. It also has a named person who decides the exceptions today, because that person becomes the approver for anything the agent holds.

If nobody can describe the rules, the pilot will spend its first month discovering them. That is useful work, but it is not an agent pilot.

Test 3: Is there a cost line the finance team recognizes?

The pilot will produce a number. For that number to mean anything, it has to connect to something Finance already measures: minutes per case, cost per processed invoice, headcount on a queue, overtime, an outsourced contract. If the only available baseline is "it feels slow", you cannot show improvement and you cannot price the outcome.

This is also the moment to agree who confirms the figure. Phinite calculates and publishes; the client's Finance team validates. Deciding that before the pilot avoids an argument after it.

Test 4: Can the owner name one number?

The last test is the hardest and the most important. Ask the process owner: if this works, which number moves? Not three numbers. One. We call it the bridge metric, because it connects what the agent did to what the business got.

For triage, it might be the share of tickets handled correctly with no person involved. For invoices, the cost per correctly processed invoice. For onboarding, days to open an account. The metric must be one the owner already looks at, built from data the agent can produce on every run.

If the team cannot name the metric, that is not a delay. That is the finding. It means the process is not ready to be measured, and an agent will not fix that.

What a good candidate looks like

Put the four tests together and the shape of a strong first pilot is clear:

  • A queue with hundreds of cases a week and a system of record.
  • A written policy and a named exception owner.
  • A baseline Finance recognizes, with an agreed way to validate results.
  • One metric the owner tracks today, with guardrails beside it: critical misses, wrong actions and SLA breaches.

Service desk triage, freight invoice audit, claims intake and loan servicing requests all pass. "An assistant for the whole sales team" does not, however appealing it sounds.

What happens next

Once the process is chosen, the pilot has a predictable arc. Weeks one and two scope the account and the workflow. Weeks three to six agree the number, the baseline and what a person approves. Weeks seven to twelve build and run against the live baseline, with success criteria fixed up front. By the end of the quarter you have a running agent and the evidence needed to compute its number.

Start with one process, one agreed metric and the evidence to compute it. Everything else in an agent program is easier once that is true.

Written byPhinite teamWrites on building, running and governing AI agents in production.

Prove what your agents changed in the business.