Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents
06:00 · July 13, 2026 · arXiv cs.AI RSS
Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.
Summary
The Hypothesis Evolution Protocol (HEP) addresses a core limitation in current LLM agents for scientific discovery: their cycles of hypothesis generation, testing, and belief revision remain implicit in unstructured logs or internal states, making the process difficult for either the agent or a human researcher to inspect or verify. HEP supplies an explicit harness that externalizes this cycle as a sequence of auditable operations. Each hypothesis is stored as a persistent registry object carrying a natural-language statement, a belief probability that represents the agent’s current assessment of its truth, an append-only event log, and a lifecycle state.
The registry is accessed through three tool groups. Propose & Evolve tools allow hypotheses to be introduced de novo, derived from existing ones, refined, or merged, thereby creating traceable lineages rather than flat lists. Test & Judge tools attach evidence only after validation and enforce threshold-based state transitions: a hypothesis moves to supported only when its belief reaches or exceeds 0.8 and to refuted only when it falls to or below 0.2. Hypotheses that cannot be tested further are placed in a dormant state with an explicit rationale. A Read interface lets the agent query the current population of hypotheses and their histories at any time.
Evaluations were performed on three open-ended materials-science questions that require explanatory understanding rather than optimization toward a preset target. The tasks concerned polymorph selection in AO₂ dioxides, B-site cation ordering in A₂BB′O₆ double perovskites, and prototype preference among rocksalt, NiAs, zincblende, and wurtzite structures in MX compounds. In each case an HEP-equipped agent operated the full hypothesis–test–evidence–belief loop, generalized the same protocol across chemically distinct families, and made fuller use of the protocol’s mechanisms as the capability of the underlying LLM increased. The resulting execution traces therefore constitute an inspectable record of how evidence altered beliefs and how hypotheses were retained, revised, or retired.
Why it matters
This research is highly relevant to the Dutch AI market's strong emphasis on transparent, ethical, and auditable AI systems. It provides researchers with a concrete methodology to build explainable AI scientists, aligning with EU regulatory standards for AI traceability and accountability.
