ClearPath: a LangGraph agent that knows when to escalate

01 / The problem

Automate the easy part, never the risky one.

Most prior-authorization requests are routine: the documentation either meets the written clinical criteria or it doesn't. Reviewers spend their time on the routine ones. But a wrong automated denial harms a patient, and a decision nobody can explain can't be defended later.

So the design goal wasn't "maximum automation". It was: resolve what is clearly safe, escalate everything else, and leave an audit trail for every decision.

02 / How it works

Propose, criticize, then route.

The LangGraph pipeline; the routing decision is code, not the model

Retrieve: RAG over the written clinical criteria for the requested procedure, so every decision is grounded in a cited rule.
Proposer and critic: one step drafts a decision with a rationale; a second step challenges it against the criteria before anything is final.
Route in Python: only confident approvals are auto-resolved. Every denial and every low-confidence case goes to the human review queue. The model can't change that rule.
Human in the loop: a reviewer can approve, deny or send back; a send-back re-runs the graph with the reviewer's notes (capped at 3 rounds).
Traced: every node is traced in Langfuse (inputs, outputs, cost, latency) and shown as an in-app audit trail.

Shipping it

Containerized with Docker and deployed on Google Cloud Run with scale-to-zero, an Artifact Registry cleanup policy and a budget alert.
Exposed as an MCP server (official MCP Python SDK, stateless streamable HTTP, DNS-rebinding protection) with four tools: list sample cases, search criteria, review a case, and read the eval metrics.
Later hardened with a full OWASP security review: see the security case study.

03 / Results

Measured, not estimated.

20 synthetic cases, GPT-4o	Result
Auto-resolved	11 of 20 (55%)
Escalated to a human	9 of 20 (45%): every denial
Average cost per decision	about $0.005
Average decision time	about 3.6 seconds

The trade-off

The critic step costs most of the per-decision spend, and it would be cheaper to skip it. I kept it because a second, adversarial pass is what makes the auto-approvals trustworthy, and at half a cent per decision the cost is small next to a reviewer's time. The data is synthetic; a real deployment would need a clinical evaluation set.

04 / What I'd do next

Where it goes from here.

Evaluate on a larger, clinician-labelled case set and report precision on auto-approvals specifically.
Calibrate the confidence threshold against reviewer outcomes instead of a fixed cut-off.
Track reviewer send-back reasons to find which criteria the agent misreads most.

Try it Browse the code

More case studies

An agent that knows when to escalate.

Automate the easy part, never the risky one.

Propose, criticize, then route.

Shipping it

Measured, not estimated.

Where it goes from here.

Securing an AI agent

Testing AI phone agents

An agent that knows when to escalate

Need an AI system that holds up in production?