Where AI stops: clinical escalation and the human in the loop
Agents are already collecting histories, answering 'where is my order,' and drafting check-ins. The question every program has to answer precisely is where the agent hands off — because the line is legal, not just a product decision.
Lithos Staff
Photograph via UnsplashAn AI agent can run the non-clinical surface of a telehealth program: structured intake, patient education, order and encounter status, scheduling, refill requests, and drafting messages for clinician review. What it cannot do is make the clinical decision: diagnose, prescribe, change a dose, or decline care. Those belong to a licensed clinician in the patient's state, and prescribing law makes that explicit. The design that works is a rules layer that classifies every agent interaction as inform, collect, route, or decide — with 'decide' always escalating to a clinician, red-flag symptoms escalating immediately, and every handoff logged.
Four kinds of interaction
Every message between a patient and a program does one of four things. Sorting interactions this way is what makes the AI boundary tractable:
| Type | Examples | Who acts |
|---|---|---|
| Inform | How the program works, what to expect from a medication, where the order is, when the check-in is due | Agent |
| Collect | History, symptoms, weight, side effects, photos, consent, identity | Agent, into a structured record |
| Route | Which queue, which modality, which clinician, how urgent | Rules layer, with agent input |
| Decide | Diagnose, prescribe, change dose, order labs, decline, refer | Licensed clinician — always |
The first three are where agents create most of the value in a telehealth program. The fourth is where the law lives: a valid prescriber-patient relationship, a licensed clinician in the patient’s state, and an e-prescription under that clinician’s NPI. See human in the loop.
Escalation design
- Red flags escalate instantly. Protocol-defined answers (chest pain, severe abdominal pain on a GLP-1, suicidal ideation, allergic reaction) bypass the queue: the agent stops, delivers emergency guidance where appropriate, and a clinician is paged. This is configured, not left to the model’s judgment.
- Uncertainty escalates. When an agent cannot classify an interaction confidently, it routes to a human rather than guessing. A wrong “inform” is a support problem; a wrong “decide” is a licensing problem.
- Clinical questions escalate. “Can I take this with my blood pressure medication?” is a question for the clinician, even when the agent could plausibly answer. The agent collects the detail and hands off.
- Drafts, not decisions. Agents can draft a check-in summary, a proposed dose continuation, or a patient message. A clinician approves, edits, or rejects; the record shows who decided.
- Every handoff is logged. What the agent saw, what it did, when it escalated, and what the clinician decided. This is what a board, an auditor, or an acquirer will ask for.
What this looks like in a GLP-1 check-in
The agent prompts the monthly check-in, collects weight, side effects, adherence, and new medications, and drafts a summary with the protocol’s suggested next step. Nausea at the expected level: the draft says “continue, step dose per protocol.” Severe abdominal pain: the agent has already escalated before the summary exists. In both cases a clinician makes the call and the prescription carries their name. The patient experiences a fast, attentive program; the program has a clinician on every decision.
Agents as the customer
The same boundary applies when the agent is not yours. Consumer assistants are beginning to act on behalf of their users — “refill my finasteride” — and the infrastructure they call has to enforce the line: the assistant can register the patient, submit the request, and report status; a licensed clinician reviews and signs. Infrastructure that bakes the human loop in is what makes it safe to let agents deliver care at all.
Frequently asked questions
Can an AI prescribe medication?
No. Prescribing requires a licensed clinician with a valid relationship to the patient under state law; electronic prescribing systems identify the prescriber by NPI and, for controlled substances, by identity-proofed credentials. An agent can prepare everything up to the decision and can carry out what the clinician decides, but it cannot decide.
What can an AI agent safely do in a telehealth program?
Collect a structured history, answer questions about the program and logistics, report encounter and order status, help schedule, submit refill requests for clinician review, and draft check-in summaries or messages that a clinician approves. In short: inform, collect, and route.
How should red-flag symptoms be handled?
Immediately and outside the normal queue. Certain answers — chest pain, suicidal ideation, signs of pancreatitis on a GLP-1, and others defined by protocol — should trigger an instant escalation to a clinician or emergency guidance, with the agent stepping out of the conversation rather than continuing to triage.
Does human-in-the-loop slow everything down?
Not if the loop is designed. Clinicians reviewing well-structured intakes and agent-drafted summaries move faster than clinicians typing from scratch. The clinician’s time goes to the decision, which is the part that matters.
From first call to first patient, in weeks.
A 15-minute intro call, sandbox credentials the same day, go-live in 3–4 weeks — new launches and existing patient bases alike.