The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
06:00 · June 25, 2026 · arXiv cs.AI RSS
Autonomous AI systems are transitioning from advisory to autonomous roles for medication prescriptions. Recent United States bill H.R. 238 and Utah's prescription-renewal pilot both authorize AI to prescribe medications in an agentic capacity. While some regulatory guidelines suggest aggregate model performance metrics for clearance, they do not require i) calibrated per-prediction confidence for action-gated thresholds, ii) differentiated communication of uncertainty arising from model ignorance (epistemic) versus genuine clinical ambiguity (aleatoric), and iii) inferential transparency at the moment of decision that allows for liability allocation. Here, we present a regulatory and technical argument (tested with a survey of 136 U.S. prescribing clinicians) positioning these as minimum architectural requirements for safe autonomous prescribing. Our results suggest prescribing clinicians i) would not permit autonomous prescribing without a calibrated confidence-based escalation mechanism, ii) preferred a competing-options summary when uncertainty was aleatoric but shifted to abstention when uncertainty was epistemic, and iii) were only willing to accept additional liability when inferential transparency enabled a substantive judgment under acknowledged uncertainty. These findings indicate our recommended architectural features would encourage higher rates of clinician adoption, largely through collapsing much of what "autonomy" conventionally means. A system meeting these requirements would function less as an autonomous agent and more as a heavily supervised decision-support tool. As legislation and state pilots proceed, our technical argument backed by clinician perspectives provides opportunities for regulation to constrain the degree of autonomy ethically granted to AI in prescribing while aligning liability with the institutional actors who control system design and deployment.
Summary
The paper examines the shift of AI systems from advisory tools to autonomous agents capable of issuing medication prescriptions without direct clinician oversight. Recent U.S. legislative moves, including bill H.R. 238 and Utah’s pilot program for renewing prescriptions of 192 chronic-condition drugs, have begun to authorize such agentic systems, yet current regulatory drafts rely primarily on aggregate performance metrics rather than per-decision safeguards.
Drawing on a survey of 136 U.S. prescribing clinicians, the authors identify three minimum architectural requirements for safe deployment. First, models must produce calibrated scores that trigger automatic escalation when falls below a predefined threshold. Second, systems must distinguish epistemic uncertainty, which stems from gaps or biases in training data, from aleatoric uncertainty, which reflects irreducible clinical ambiguity such as competing therapeutic options. Third, every prediction must include inferential transparency that records data provenance and decision logic at the point of action, enabling post-hoc liability assessment.
Survey responses indicate that clinicians would refuse autonomous prescribing in the absence of calibrated escalation. When uncertainty is aleatoric they favor a summary of competing options; when it is epistemic they prefer outright abstention. Clinicians also assign greater responsibility to the organizations that design and deploy such systems than to individual practitioners, but they accept more personal liability once an escalated case supplies sufficient transparency for them to exercise judgment.
Taken together, the three requirements substantially narrow the scope of autonomy. A system that abstains on low-confidence cases, routes differentiated uncertainty signals to human review, and maintains auditable decision trails operates more as a closely supervised decision-support instrument than as an independent prescribing agent. The authors argue that these constraints align technical design with both clinician trust and existing liability structures while legislation continues to advance.
Why it matters
The article provides actionable architectural requirements for high-risk AI systems in healthcare, directly aligning with the Dutch and EU focus on ethical, transparent, and human-centric AI. Researchers can apply these insights into uncertainty communication and liability allocation to design compliant AI models for the European market.
