An ops automation system that absorbed 70% of a manual review queue.

01 — Client context

A payments scaleup with a manual review team triaging thousands of edge-case transactions a week. Headcount was growing faster than volume, and onboarding new reviewers took six weeks before they were trusted with high-value cases.

02 — Operational problem

The review queue was the operational bottleneck behind every customer SLA. The team had tried two off-the-shelf tools and one in-house attempt; none survived contact with the messy reality of bank-feed data, partner exceptions, and regulator-driven policy churn.

03 — System designed

A two-stage agentic pipeline: a deterministic rules layer for the 60% of cases that were genuinely mechanical, and a small set of specialised LLM agents — each with a narrow remit (entity resolution, policy lookup, draft decision) — handing off to a single human reviewer with an evidence panel attached.

04 — Implementation shape

• Two-week diagnostic sprint sitting with senior reviewers
• Eval harness built before any agent was written, scored against 1,400 historical cases
• Production rollout in shadow mode for three weeks, then staged exposure by case-value band
• Reviewer UI rebuilt around agent evidence, not raw model output

05 — Guardrails, evals & governance

• Hard caps on autonomous decisioning by case value and customer segment
• Per-policy evals re-run on every prompt or model change, gated in CI
• Full decision audit trail wired into the existing compliance warehouse

06 — Measurable result

• −68% average handling time across the queue
• 70% of cases resolved without a reviewer touching them
• Onboarding for new reviewers reduced from 6 weeks to 9 days