ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding
06:00 · August 3, 2026 · arXiv cs.AI RSS
Multimodal agents operating in long-horizon environments must build and continually update multimedia memories to support entity-consistent, temporally grounded reasoning. However, existing agentic memory approaches often discard fine-grained dentity cues under aggressive compression and segment-wise processing. They also rely heavily on vector similarity retrieval, which can surface semantically related yet identity-mismatched evidence, leading to entity confusion, error propagation, and hallucinated answers. We propose ViSAGE, a multimodal agentic memory framework that constructs self-correcting, entity-centric memories. Specifically, ViSAGE anchors entity identity via cross-modal binding over long temporal ranges. It then applies bidirectional memory refinement to propagate delayed identity evidence, retroactively unifying historical records and improving future reasoning. We also introduce multi-agent cross-verification to assess retrieved evidence under an identity-evidence alignment onstraint, enabling abstention instead of unsupported answers when evidence is missing. Extensive results demonstrate that ViSAGE consistently outperforms the strongest baseline, achieving 5.9% higher accuracy.
Summary
ViSAGE is a multimodal agentic memory framework developed to support consistent, temporally grounded reasoning over extended video sequences. It tackles two recurring problems in existing systems: loss of fine-grained identity cues during aggressive compression or chunk-wise processing, and reliance on vector-similarity retrieval that can return semantically related yet identity-mismatched evidence. These issues frequently produce entity confusion, error propagation, and unsupported answers.
The framework maintains a dual memory structure consisting of sequential Incident Logs that record narrative events and Object Cards that accumulate and revise entity attributes. Cross-modal entity binding first aligns visual tracklets with complementary audio and appearance signals across long temporal spans. Bidirectional memory refinement then propagates newly acquired identity evidence both forward and backward, retroactively unifying earlier records and stabilizing subsequent reasoning. During query handling, multi-agent cross-verification evaluates retrieved evidence against an identity-evidence alignment constraint, allowing the system to abstain rather than generate answers when supporting evidence is absent or inconsistent.
Evaluations on M3-Bench-robot, M3-Bench-web, and Video-MME-long show consistent gains over strong baselines, with reported accuracies of 45.5 percent, 58.4 percent, and 79.1 percent respectively. The largest improvements appear on identity-critical queries, and the verification stage demonstrably reduces hallucinations by enforcing verified refusals when evidence is insufficient.
Why it matters
This research is highly relevant for Dutch AI researchers working on multimodal models and embodied AI. Its emphasis on epistemic safety and reducing hallucinations through verified refusals strongly aligns with the Netherlands and EU regulatory focus on transparent, trustworthy, and reliable AI systems.
