Evidence-Ledger Adjudication for Claim-Evidence Traceability
06:00 · July 30, 2026 · arXiv cs.AI RSS
AI agents can draft claims faster than authors can check whether the cited or retrieved evidence supports them. We study evidence-ledger adjudication: a claim-evidence traceability workflow that pairs each claim with an evidence packet, assigns a support relation, and routes unsupported, contradicted, or mixed-evidence claims back to the author. The empirical core is a 2,335-row blind benchmark built from independent external labels in AVeriTeC, CLIMATE-FEVER, and SciFact. Gold relations and source evidence labels are hidden during prediction and joined only for scoring. On this benchmark, the agent evidence-ledger condition achieves 0.676 relation accuracy and 0.601 macro-F1, compared with 0.383 accuracy and 0.303 macro-F1 for the best non-agent baseline. It also routes 1270/1435 claims whose gold labels indicate contradiction, missing evidence, or mixed evidence, while routing 295/900 supported claims. These results show that evidence-ledger adjudication can turn heterogeneous evidence packets into an auditable traceability layer for AI-assisted writing.
Summary
Evidence-ledger adjudication addresses a practical bottleneck in AI-assisted writing: language models generate claims and citations faster than authors can verify whether the accompanying evidence actually supports them. The workflow records each claim together with its evidence packet, assigns one of four normalized support relations—supports, contradicts, missing evidence, or mixed evidence—and issues an author-review route flag when the relation is anything other than supports. Supported claims remain in the draft; all others are returned for revision, additional retrieval, or removal. The output also includes a confidence score and a concise rationale, producing an auditable intermediate artifact rather than a final verdict.
The empirical evaluation uses a 2,335-row blind test set assembled from three externally labeled sources whose annotations were created independently of the workflow. AVeriTeC contributes 500 rows with question-answer evidence packets, CLIMATE-FEVER supplies 1,535 climate-related claims paired with annotated evidence sentences, and SciFact adds 300 rows drawn from scientific abstracts. Gold relations and source evidence labels are withheld during prediction and joined only at scoring time, ensuring the adjudicator operates without access to the original dataset metadata.
On this benchmark an agent-based evidence-ledger system reaches 0.676 relation accuracy and 0.601 macro-F1. It correctly routes 1,270 of the 1,435 claims whose gold labels indicate contradiction, missing evidence, or mixed evidence, while routing only 295 of the 900 supported claims. The strongest non-agent baseline, a TF-IDF logistic classifier tuned on 3,877 additional rows, achieves 0.383 accuracy and 0.303 macro-F1 and routes substantially fewer problematic claims. These figures indicate that the agent condition converts heterogeneous evidence packets into a usable traceability layer without relying on shallow lexical overlap or supervised text classification alone.
Why it matters
Strong technical depth and novelty in claim-evidence verification directly support Dutch/EU priorities on transparent and ethical AI. Researchers can adapt the blind-benchmark protocol and routing rules for local compliance or tool-building. High audience fit for advanced readers studying agent evaluation and fact-checking.
