Beyond expert users: agents should help users construct preferences, not just elicit them
06:00 · July 1, 2026 · arXiv cs.AI RSS
Agents typically assume an expert user -- one with well-formed preferences about what they want -- and default to clarifying questions whenever the task is underspecified. We argue this assumption is unrealistic. Users often lack the domain knowledge to have completely specified preferences; if asked about their preference on some feature, the user may be unable to answer without the agent helping the user to learn some domain knowledge needed to form a preference for that feature, e.g., via examples or explanations. To formalize these principles, we draw on the Search-Experience-Credence framework from Information Economics to introduce CoPref, a model of how users construct preferences based on agent dialog actions. We then study these ideas concretely in agentic recommender systems, proposing CoShop, an interactive benchmark. In CoShop, an agent converses with and makes recommendations for a CoPref user. The agent's performance depends on whether it can help the user gain the knowledge needed to specify the task well. Evaluating five frontier models, we find that no agent exceeds 56% accuracy on CoShop despite five turns of interaction. Failures stem not from agents' ability to find items, but from how little the interaction expands what users know about what they want.
Summary
Agents interacting with users commonly treat preferences as already formed and respond to underspecified requests by asking clarifying questions. This approach overlooks a frequent reality: many users lack the domain knowledge required to articulate stable preferences in the first place. When an agent inquires about a particular feature, the user may need examples or explanations before a preference can even be stated.
To address this gap, the work draws on the Search-Experience-Credence framework from information economics. It introduces CoPref, a model that treats preference construction as an outcome of specific dialog actions rather than a simple elicitation process. The model distinguishes situations in which users can evaluate attributes directly from those in which they must first acquire relevant knowledge through interaction.
The authors then instantiate these ideas in agentic recommender systems by releasing CoShop, an interactive benchmark. In CoShop an agent must converse with a simulated CoPref user and recommend items whose suitability depends on knowledge the user acquires during the exchange. Five frontier models were evaluated over five turns of interaction; none exceeded 56 percent accuracy. The shortfall arose not from an inability to retrieve candidate items, but from limited success in expanding the user’s understanding of what they actually want.
Why it matters
This research is highly relevant for AI researchers and developers focusing on user-centric and transparent AI, a key priority in the Dutch AI market. By providing a formal framework and benchmark for improving how agents assist non-expert users, it offers actionable insights for enhancing conversational AI and recommender systems in enterprise applications.
