NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning
06:00 · June 29, 2026 · arXiv cs.AI RSS
Multimodal large language models (MLLMs) are increasingly deployed as embodied planners in egocentric environments, where task success requires not only achieving instructed goals but also acting in socially appropriate ways. While explicit goals may render certain actions optimal, implicit social norms often impose hidden constraints. Existing evaluations typically focus on explicit goal achievement or direct norm knowledge, seldom assessing whether planners can infer and apply these hidden constraints within action sequences. We introduce NormAct, a benchmark for embodied social-norm interactions that evaluates plans on Goal Achievement, Norm Compliance, and overall Task Success. NormAct uniquely embeds hidden norms within ordinary tasks, testing whether models can realize them without explicit instruction. Experiments with state-of-the-art MLLMs (GPT-5.4, Claude Opus 4.7, Gemini 3 Pro) reveal a significant gap: models achieve explicit goals in 67.3\% of cases, but comply with hidden norms in only 26.4\%. Cue-condition experiments indicate that this gap stems not from a lack of general social knowledge, but from challenges in activating and grounding relevant norms in context. To address this, we propose NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms prior to planning, increasing Task Success from 24.2\% to 46.7\%. Our results underscore the importance of enabling embodied agents to proactively detect hidden norms, ground them in visual evidence, and integrate them as action-planning constraints. Our benchmark is publicly available at https://huggingface.co/datasets/Caleb196x/NormAct.
Summary
Multimodal large language models are increasingly used as embodied planners that must translate first-person visual observations and natural-language instructions into executable action sequences. In human environments, however, success depends on more than reaching an explicit goal; agents must also respect implicit social norms that are rarely stated in the task description. These hidden constraints, such as respecting ownership before taking an object or waiting in line before approaching a counter, can require the planner to insert additional steps, delays, or detours that are invisible to standard goal-completion metrics.
NormAct addresses this evaluation gap by embedding social norms inside ordinary embodied tasks and scoring generated plans along three separate axes: whether the instructed goal is achieved, whether the hidden norm is respected, and whether both conditions hold simultaneously for overall task success. Experiments with current frontier models show a clear divergence: explicit goals are reached in 67.3 percent of cases, yet hidden norms are observed in only 26.4 percent. Controlled cueing experiments indicate that the shortfall arises less from missing normative knowledge than from difficulty activating and visually grounding the relevant constraint within a given scene.
To mitigate this activation bottleneck, the authors introduce NormPerceptor, a context-conditioned cue generator that first infers scene-relevant social norms from the observation and instruction before planning begins. When inserted into the pipeline, the module raises task success from 24.2 percent to 46.7 percent, recovering a substantial fraction of the improvement previously obtained only with human-authored cues. The benchmark, which includes graded cue conditions to isolate failures along the chain of norm detection, grounding, and action translation, is publicly available for further research.
Why it matters
This research is highly relevant to the Dutch AI market's strong emphasis on ethical, transparent, and socially responsible AI. The benchmark provides Dutch researchers and enterprises with actionable tools to evaluate and improve the social compliance of embodied AI agents, aligning with EU regulatory frameworks for safe AI deployment.
