$\varepsilon$-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution
06:00 · August 15, 2026 · arXiv cs.AI RSS
LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion. We introduce $\varepsilon$-MemEvo, a framework for cross-task knowledge transfer in LLM program evolution. $\varepsilon$-MemEvo stores prior experience as task-agnostic tactic memories: compact natural-language summaries of successful algorithmic strategies rather than raw code, enabling transfer across tasks with different APIs and evaluators. To avoid negative transfer from semantically mismatched memories, $\varepsilon$-MemEvo uses an adaptive injection gate that decides whether retrieved memories should be injected, and at what intensity. We evaluate $\varepsilon$-MemEvo on 8 diverse optimization benchmarks spanning mathematical optimization and systems engineering, using a content-level Leave-One-Out protocol that excludes target-task memory entries. On the primary GPT-5 backbone, $\varepsilon$-MemEvo improves AUCC over AdaEvolve on all 8 tasks, with a mean relative gain of +8.7%, and improves early-stage convergence by +9.4% on average. Ablations show that naive memory injection can fail catastrophically, while adaptive gating remains safe across all five ablation tasks. The data-updated posterior is interpretable in observed states: it favors skip during improving search and shifts from skip to hint across early and late plateaus. These gains incur less than 1% computational overhead.
Summary
ε-MemEvo addresses a core limitation in LLM-driven program evolution frameworks such as FunSearch and AlphaEvolve: each optimization task begins without access to strategies discovered on earlier problems, so experience is discarded after every run. The system stores prior outcomes as compact natural-language tactic summaries rather than raw code. These summaries capture reusable algorithmic patterns and remain independent of any single task’s API or evaluator, allowing transfer across domains that range from geometric packing to load balancing and SQL query tuning.
To prevent harmful transfer when retrieved tactics do not match the current search state, ε-MemEvo adds an adaptive injection gate. The gate employs Thompson sampling over Beta-distributed arms to choose among three actions—skip, hint, or guide—at each iteration. The policy is updated from delayed rewards produced by the underlying AdaEvolve search loop, so the probability of injection rises or falls according to observed success. A content-level Leave-One-Out protocol ensures that memory entries derived from the target task itself are excluded during evaluation, providing a strict test of cross-task generalization.
On eight optimization benchmarks the gated approach yields a mean relative improvement of 8.7 percent in area-under-convergence-curve (AUCC) over AdaEvolve when using a GPT-5 backbone, with an average 9.4 percent gain in early-stage AUC@20. The largest accelerations appear on tasks that benefit most from rapid cold-start progress. Ablation experiments show that always-inject and simple rule-based policies can cause complete search failure on multiple tasks, whereas the learned gate remains safe across all tested conditions and adds less than one percent per-iteration overhead. The posterior probabilities themselves are interpretable: the gate favors skipping while scores improve and shifts toward hinting once plateaus are detected.
Why it matters
Provides novel, reproducible methods for improving LLM evolutionary search that Dutch AI researchers and advanced labs can directly implement or extend for optimization tasks.
