Thinking of ACE? We Can Do It with Fewer Tokens

Summary

Two recent agentic memory systems, ACE and ALTK-Evolve, turn an agent's past execution traces into reusable lessons that are stored and retrieved at inference time, without any parameter updates. Both approaches operate on the same class of ReAct-style agents and address the same failure patterns on multi-step tasks such as API pagination errors or incorrect entity resolution. They also share a core design choice: they preserve detailed, itemized lessons rather than collapsing them into short summaries, because compression tends to discard the very distinctions that later prove useful.

The systems diverge in how those lessons reach the model. ACE maintains a single evolving playbook that is injected in full at every reasoning step. ALTK-Evolve instead consolidates near-duplicate lessons into individually addressable guidelines, each carrying a support count that records how many independent trajectories produced it. At inference the system supplies only a small fixed core plus a task-specific subset retrieved by embedding similarity or LLM scoring, or the full set when the model has sufficient context capacity. This selective delivery is the main source of the observed efficiency difference.

Evaluations on the AppWorld benchmark, using identical base ReAct agents and the same two models, show that the selective approach matches or exceeds ACE accuracy while cutting inference tokens substantially. With the stronger model the method improves both task-goal and scenario-goal completion at roughly 40 percent of ACE's token cost. With the weaker model the accuracy figures are statistically indistinguishable, yet token usage drops to about one-seventh. The advantage is most pronounced on harder tasks, where a smaller, curated prompt avoids overwhelming the model with irrelevant guidance.

The comparison underscores that the cost of agentic memory is driven less by how lessons are stored than by how they are served. Calibrating the amount of context to the model's capability and the immediate task yields comparable reliability at materially lower inference expense.

Why it matters

This article provides actionable insights for ML Engineers building LLM agents, offering a concrete method (ALTK-Evolve) to reduce inference costs and token usage without sacrificing accuracy. It directly addresses production challenges like context overload and compute efficiency, which are critical for Dutch enterprises scaling AI solutions.