LLM Agents Perform Controlled Experiments Using Simulation Models
06:00 · August 26, 2026 · arXiv cs.AI RSS
Large language models (LLMs) have shown strong capabilities in reasoning, planning, and tool use, but many scientific and engineering tasks require more than plausible text and code generation. They require understanding how a system responds to intervention, which in practice depends on controlled experimentation. In this work, we propose a multi-agent framework that enables LLM agents to conduct controlled experiments with scientific simulation models for pharmaceutical process design. Given a user query and a baseline configuration, the system constructs a structured task representation, designs experiments, executes comparative simulation, interprets the resulting outcomes, and synthesizes evidence-based recommendations for process parameter optimization. By coupling language models with high-fidelity simulation models in an interactive agent framework, the proposed system supports reasoning through intervention, comparison, and observation. As a result, it produces more specific and actionable outputs than language-only reasoning. In an industrial application setting, this advantage is reflected in higher output specificity as well as improved user-rated correctness and helpfulness. Ablation studies and visualized case analyses further demonstrate the effectiveness and practical utility of simulation-integrated experimental reasoning.
Summary
Large language models excel at multi-step reasoning and tool use, yet many engineering problems demand more than text generation: they require determining how a system behaves under deliberate changes. The authors address this gap by coupling LLMs with high-fidelity simulation models that capture physical and chemical dynamics, allowing agents to test hypotheses through controlled intervention rather than relying on language alone.
The resulting multi-agent framework is applied to pharmaceutical process design. Given a user query and a baseline configuration, the system first produces a structured task description. A planner then designs comparative experiments that vary selected parameters while holding others fixed. These experiments are executed on the simulation model, outcomes are interpreted by dedicated agents, and the findings are synthesized into parameter recommendations that rest on observed differences rather than plausible inference.
In an industrial evaluation the simulation-integrated approach yielded outputs rated higher in specificity, correctness, and helpfulness than those produced by language-only reasoning. Ablation studies confirm that the explicit cycle of hypothesis, controlled comparison, and evidence synthesis accounts for the measured gains, while case visualizations illustrate how intermediate artifacts remain traceable for engineering review.
Why it matters
This research is highly relevant for Dutch AI researchers and industrial R&D teams, particularly in the strong local chemical, pharmaceutical, and high-tech manufacturing sectors. It provides a novel, actionable framework for grounding LLM reasoning in scientific simulations, addressing the critical need for reliable and evidence-based AI decision support in enterprise environments.
