Object-Centric Environment Modeling for Agentic Tasks
06:00 · July 7, 2026 · arXiv cs.AI RSS
Large language model (LLM) agents can improve through accumulated experience, but free-form textual memories become difficult to maintain, validate, and reuse as interactions grow. Recent symbolic approaches learn executable skills or programmatic world models, yet often store local procedures or assume simplified dynamics. We propose Object-Centric Environment Modeling (OCM), which organizes experience into an executable object-centric environment model. OCM maintains two connected code bases: object knowledge, which defines environment entities and mechanisms as Python classes, and procedure knowledge, which records reusable interaction patterns that must import and use the object model. OCM works in an online setting: after each episode, OCM reflects on the trajectory, updates both knowledge bases, and verifies that all procedures execute against the updated object model. During future interaction, the agent uses progressive knowledge disclosure to inspect compact code signatures first and read source code only when needed. Experiments show that OCM achieves the best average rank across benchmarks and reduces invalid actions, demonstrating that agents can benefit from building object-centric environment models.
Summary
Large language model agents accumulate experience across repeated interactions with partially observable environments, yet free-form textual memories grow difficult to maintain, validate, and reuse. Existing symbolic methods either store isolated, locally valid procedures without a shared account of objects and preconditions or attempt to learn monolithic programmatic world models that assume simplified dynamics. Object-Centric Environment Modeling (OCM) addresses this gap by representing accumulated knowledge as an executable, object-centric environment model expressed in two interdependent Python code bases.
Object knowledge encodes persistent environment entities and their interaction mechanisms as Python classes, with attributes and methods capturing properties, states, and affordances. Procedure knowledge stores reusable interaction patterns as separate Python files that must import and operate on those classes. After each task episode the agent reflects on the trajectory, proposes updates to both code bases, and verifies consistency by executing every procedure against the revised object model; only verified updates are committed. At inference time the agent applies progressive knowledge disclosure, first inspecting compact code signatures before retrieving full source when needed.
Evaluations on ScienceWorld, ALFWorld, and PlanCraft show that OCM attains the highest average rank among compared methods while markedly lowering the rate of invalid actions. The results indicate that grounding procedural experience in a shared, auditable object model enables more structured knowledge accumulation and more reliable action selection in complex interactive settings.
Why it matters
This research is highly relevant for Dutch AI researchers and developers working on autonomous LLM agents. It provides a structured, programmatic approach to agent memory and environment modeling, which can be directly applied by technical teams in the Netherlands to build more robust and reliable AI systems.
