SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition
06:00 · August 3, 2026 · arXiv cs.AI RSS
Large language model (LLM) agents have been increasingly adopted in scientific research for organizing and invoking specialized computational tools. However, their reliance on predefined tool spaces with static semantics limits their applicability to open-world scientific workflows, where tool requirements, capabilities, and boundaries evolve dynamically. To this end, we propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition. Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation, whereas during inference, it formulates active requests and utilizes a LinUCB-based bandit gate to dynamically balance exploration and exploitation. Once a novel tool is acquired, its scientific ontology is completed online for seamless integration into the known graph. Moreover, we introduce OpenSciToolBench, a benchmark containing 900 realistic tasks across four difficulty levels. Extensive evaluations show that SciToolAgent-Evo achieves state-of-the-art performance, validating its robustness and generalization.
Summary
SciToolAgent-Evo addresses a core limitation of current LLM agents in scientific domains: their dependence on fixed tool libraries whose semantics and boundaries remain static. In open-world research settings, required capabilities such as descriptor calculators or format converters often lie outside the agent’s initial inventory, and conventional systems either fail or over-exploit the tools they already know. The proposed agent counters this by maintaining an evolving memory that combines a skill library, an experience pool, and an ontologized tool graph. During accumulation phases it extracts compact, reusable knowledge from pairs of successful and unsuccessful trajectories; at inference time it issues explicit requests for missing tools and routes decisions through a LinUCB bandit that continuously trades off exploration of unknown tools against exploitation of known ones.
When a new tool is obtained, its scientific ontology—covering inputs, outputs, domain relations, and usage constraints—is completed online and merged into the existing graph, enabling immediate reuse in subsequent tasks. This self-evolution loop removes the need for manual ontology updates and supports continual adaptation across biology, chemistry, and materials-science workflows.
To evaluate such capabilities the authors introduce OpenSciToolBench, a 900-task benchmark derived from SciToolKG. Tasks are stratified into four difficulty levels that progress from direct tool invocation (one or two tools) to open-ended research-strategy formulation (five to ten tools), and they appear in four formats: question answering, multiple choice, content completion, and true/false. Construction follows a two-stage verification pipeline that combines LLM judging with human review for tool-chain coherence and scientific validity. Experimental results across this benchmark and prior scientific tool-use suites indicate that SciToolAgent-Evo attains state-of-the-art performance, with ablation studies confirming the contribution of the evolving memory, active requests, and bandit-based exploration gate.
Why it matters
This research is highly relevant for Dutch AI researchers and institutions focusing on autonomous agents and AI-driven scientific discovery. The methodologies for open-world tool acquisition and the new benchmark provide actionable frameworks for advancing LLM applications in fields like chemistry and materials science.
