From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
06:00 · July 11, 2026 · arXiv cs.AI RSS
Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
Summary
The paper addresses the common pattern in which enterprise LLM applications begin as prompt-dominant prototypes whose core behavior—source selection, entity routing, claim eligibility, and output constraints—resides chiefly in natural-language instructions and retrieval context. Productization exposes the limits of this approach: prompts cannot reliably enforce traceability to bounded sources, versioned contracts, or reproducible traces once answers reach users. The authors therefore reconstruct the prototype as a harness-engineered architecture in which deterministic control moves into code, manifests, schemas, and validation artifacts that surround a replaceable composition boundary. Within this boundary the language model handles only phrasing; source-backed claims, routing metadata, answer contracts, and trace generation remain authoritative and auditable.
The method is instantiated on a public-data slice covering five Korean corporate groups and 25 listed companies, yielding 113 source-backed runtime claims. A source-to-claim pipeline separates raw documents, evidence records, runtime-eligible claims, maintained wiki context, and reader-facing answers, while manifests define permissible sources and routing rules bind questions to entities. Validation checks source grounding, entity routing, trace completeness, output hygiene, and recommendation-language constraints. Three research questions guide the evaluation. The first confirms that the harness preserves its contracts across fixed validation scenarios, with fault injection demonstrating that validators correctly flag deliberate violations. The second shows that the same guarantees hold under model substitution: across three hosted models and 270 composition-boundary runs, all harness-enforced checks passed while model-side failures were isolated and recorded.
The third question tests whether code-owned enforcement is load-bearing. With the model fixed, prompt instructions alone permitted recommendation-language and internal-trace violations to reach the reader; the harness blocked every such case. A bolt-on external guardrail also prevented violations yet reduced utility through over-refusal, whereas the harness maintained full utility. The resulting pattern therefore supplies versioned source, control, and validation artifacts that turn exploratory prototypes into auditable enterprise agents without tying guarantees to any single model or prompt formulation.
Why it matters
Directly actionable for Dutch/EU teams building compliant LLM agents; aligns with Netherlands emphasis on ethical, transparent AI and EU regulatory needs for auditability. Offers novel, technically rigorous methodology with high reproducibility for researchers and advanced practitioners.
