Human-Centric Reflective Architecture for Human-AI Collaborative Decision-Making
06:00 · July 7, 2026 · arXiv cs.AI RSS
The use of Large Language Models (LLMs) across diverse areas of human activity-ranging from everyday tasks to safety-critical applications-aims to enhance decision-making effectiveness with minimal human feedback. Concurrently, it seeks to align decisions with human expectations, preferences, and needs while mitigating risks associated with AI non-determinism. However, humans frequently over- or under-rely on AI recommendations, and current AI systems remain poorly calibrated to human expectations. To address these challenges, we introduce a human-AI collaborative decision-making framework designed to augment human capabilities and align AI agents with human preferences and expectations. Specifically, this paper (a) formulates the collaborative decision-making task as a stochastic game between an AI agent and a human player, and (b) proposes the Human-Centric Reflective Architecture (HCRA), which integrates human-calibrated models with reinforcement learning agents that leverage linguistic feedback in an iterative, reflective process. Evaluation results demonstrate that HCRA enhances decision-making effectiveness and delivers high-quality recommendations.
Summary
The paper presents the Human-Centric Reflective Architecture (HCRA) as a framework for human-AI collaborative decision-making. It models the interaction explicitly as a stochastic game between an AI agent and a human player, where the objective is to maximize a human-utility function defined over user expectations and preferences. This game-theoretic formulation supplies formal convergence and termination guarantees for the iterative process.
HCRA integrates human-calibrated models with reinforcement learning language agents that operate through linguistic feedback. The architecture follows an iterative reflective loop inspired by Reflexion: an actor produces candidate recommendations, an evaluator scores outcomes against task criteria, and a self-reflection component converts sparse reward signals into linguistic summaries stored in short- and long-term memory. Human behavior models, trained on empirical interaction data, adjust AI confidence scores so that they better match human perceptions of reliability, steering the loop toward recommendations that optimize human utility rather than raw task accuracy alone.
The design targets two persistent problems in LLM-assisted settings: human over- or under-reliance caused by miscalibrated expectations, and the non-determinism inherent in model outputs. Instead of depending exclusively on large-scale pretraining and aggregated alignment methods such as RLHF, HCRA performs test-time tuning that incorporates historical interactions, a recommendation calibration model, and an acceptance model. Evaluation in the tourism-recommendation domain shows that the resulting recommendations improve decision effectiveness while reducing the practical consequences of poorly calibrated trust.
Why it matters
This research is highly relevant to the Dutch AI market's focus on ethical, transparent, and human-centric AI. The proposed HCRA framework provides advanced methodologies for researchers to build AI systems that align with human preferences, directly supporting EU AI Act compliance regarding human oversight.
