Accurate and Efficient Long-Term Memory for LLM Agents
06:00 · July 21, 2026 · arXiv cs.AI RSS
LLM agents augmented with persistent memory can recall past interactions, but existing systems suffer from two limitations: flat, unstructured storage loses relational context needed for multi-hop and temporal reasoning, and reliance on expensive LLM-based classification makes them impractical for latency-sensitive deployment. Without mechanisms to validate new information against stored knowledge, these systems silently accumulate contradictions. We present MOSAIC (Memory-Organized Structured Agent for Information Collection), a structured, conflict-aware long-term memory framework for LLM agents that is substantially more accurate and efficient. MOSAIC introduces three key capabilities: (1) entity-typed graph storage with semantic classification preserving relational structure across events, personas, and relationships, enabling multi-hop and temporal reasoning over conversation history; (2) hash-accelerated dual-path retrieval replacing LLM-based classification with locality-sensitive hashing, achieving near-instantaneous lookup with negligible accuracy loss; and (3) active conflict detection at save time that cross-references new information against existing graph neighbors, triggering updates or deletions for contradictory entries. Evaluated on LoCoMo (long-conversation QA), HaluMem, and a novel clinical-guideline error compounding test, MOSAIC achieves 89.35% accuracy on LoCoMo (+27.21 pp over the best baseline), best HaluMem-Medium extraction F1(86.77%) and HaluMem-Long extraction F1 (85.84%), best QA correctness on both Medium and Long (73.10%, 70.75%), and detects 66% of injected factual conflicts-4.7 times higher than the best baseline (14%)-while hash-accelerated retrieval keeps average search latency at 0.58 s per question.
Summary
MOSAIC addresses persistent shortcomings in memory-augmented LLM agents by replacing flat, append-only storage with a typed entity graph that distinguishes events, personas, and relationships. This structure preserves the relational and temporal links required for multi-hop reasoning, while an ingestion-time conflict detector cross-references each new fact against its graph neighbors and resolves contradictions through updates or deletions. Retrieval is accelerated by locality-sensitive hashing that bypasses repeated LLM classification, keeping lookup latency near 0.58 seconds without measurable loss of accuracy.
The design rests on the principle of neighbor-conditioned stability: an entity’s priority or belief state is re-evaluated only when one of its immediate neighbors changes. This locality mirrors the dependency patterns found in clinical intake, insurance adjudication, and technical troubleshooting, where decisions hinge on small clusters of related facts rather than global state.
Evaluated on LoCoMo long-conversation QA, HaluMem extraction and QA tasks, and a clinical error-compounding benchmark, MOSAIC reaches 89.35 percent accuracy on LoCoMo, an absolute gain of 27.21 points over the strongest baseline. It records the highest extraction F1 scores on both medium- and long-context HaluMem splits, improves QA correctness to 73.10 percent and 70.75 percent respectively, and identifies 66 percent of injected factual conflicts—more than four times the rate of prior systems—while maintaining low search latency. The framework is positioned for safety-critical settings where undetected contradictions can propagate into harmful downstream actions.
Why it matters
Provides novel, reproducible graph-based memory methods directly applicable to reliable LLM agent development; aligns with Dutch/EU emphasis on ethical, transparent AI and supports SME adoption of robust agent systems.
