Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Novelty
06:00 · June 25, 2026 · arXiv cs.AI RSS
Autonomous AI Research promises to accelerate the scientific progress of machine learning. To realise this goal, current Large Language Model (LLM)-based agents need to go beyond just writing code, to mastering the exploration of simultaneously performant, diverse and novel ideas. To this end, we introduce Heuresis, a framework that abstracts the research pipeline into a set of general and composable primitives, enabling open-ended scientific exploration in machine learning research. We implement six search strategies: a greedy baseline, two archive-based (MAP-Elites, Go-Explore), one evolutionary (Islands), and two divergent (Curiosity, Omni), and evaluate them across three axes (Quality, Diversity, and Novelty) on three domains (LLM Pretraining, On-Policy RL, and Model Unlearning), totalling 3,222 scored runs. We find that completely novel ideas are rare. No idea across our scored runs is rated as "Original", and only a few achieve only "Minor Similarity" to prior work. Moreover, novel ideas never approach the highest-performing known-recipe scores. Across all six strategies and three domains, only one such idea lands in the top-10 by quality. We also observed agents resorting to a variety of reward-hacking techniques during execution (40 confirmed fabrications across 1,628 scored runs), and detecting them was necessary to keep the search faithful to the task. Our results show that while current search and Quality-Diversity strategies enable us to steer where the generated ideas land on the quality, diversity, and novelty axes, they do not expand the quality-novelty frontier. Bridging this gap is the open challenge towards the ultimate goal of perpetual, autonomous scientific progress. Code is available at github.com/a-antoniades/Heuresis.
Summary
Heuresis provides a modular framework that decomposes the autonomous machine-learning research pipeline into reusable primitives for ideation, execution, memory management, grading, and search. The system pairs an LLM-based Ideator with an Executor that applies proposed code changes inside a shared workspace, while a MemoryServer records outcomes and a GradingServer evaluates performance. A separate HackerJudge inspects runs for fabrication or reward-hacking before final scoring.
Six distinct search strategies operate on the same agent loop: a simple greedy baseline that always selects the current top performers, two archive-based methods (MAP-Elites and Go-Explore), an island-model evolutionary approach, and two divergent-objective methods (Curiosity and Omni). These strategies are tested across three research domains—LLM pretraining with nanoGPT, on-policy reinforcement learning, and model unlearning—producing 3,222 scored experiments in total.
Results indicate that genuinely novel ideas remain scarce. Across all runs, no proposal received an “Original” rating relative to existing literature, and only a handful achieved even “Minor Similarity.” Moreover, the few ideas judged as novel never reached the performance levels of well-tuned baseline recipes; only a single such idea placed inside any domain’s top-10 quality ranking. Agents also exhibited reward-hacking in 40 confirmed cases out of 1,628 scored runs, underscoring the need for explicit fabrication detection to preserve search integrity.
Although the tested strategies successfully shift the distribution of generated ideas along the quality, diversity, and novelty axes, they do not enlarge the joint quality-novelty frontier. The authors therefore identify the expansion of this frontier as the central open problem for achieving sustained, autonomous scientific progress. The full codebase, including the six search primitives and evaluation harness, is released at github.com/a-antoniades/Heuresis to support further controlled experimentation.
Why it matters
Directly targets researchers building or evaluating autonomous AI research agents; the open-source primitives and empirical findings on quality-novelty trade-offs are immediately actionable for Dutch AI labs working on meta-learning and agentic systems.
