Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation
06:00 · June 24, 2026 · arXiv cs.AI RSS
Recommender systems often induce filter bubbles and semantic homogenization by monolithically optimizing for immediate user engagement. Standard single-objective models, including traditional Deep Q-Networks, are ill-equipped to navigate the trade-offs between platform retention and critical societal values like information diversity and provider fairness. To address these limitations, we introduce a multi-objective reinforcement learning framework that formalizes recommendation as a semantic multi-objective Markov decision process. By integrating high-fidelity semantic embeddings with a Pareto-DQN agent, our architecture treats engagement, diversity, and fairness as distinct, non-aggregable reward signals, avoiding the pitfalls of static reward scalarization. Empirical evaluations on the MovieLens small dataset shows that our hypervolume based action selection disrupts the feedback loops responsible for semantic collapse. By sustaining high state-trajectory variance, the Pareto-DQN effectively maps the Pareto frontier, achieving gains in auxiliary societal objectives with only marginal impacts on engagement. This work provides a path toward intrinsically aligned, responsible recommender systems.
Summary
Recommender systems that optimize solely for immediate user engagement tend to narrow the semantic range of suggestions over successive interactions, producing filter bubbles and progressive homogenization of content. The paper formalizes this sequential task as a multi-objective Markov decision process in which engagement, diversity, and fairness remain distinct reward signals rather than being collapsed into a single scalar. A Pareto-DQN agent is trained to approximate the set of non-dominated policies, with action selection guided by hypervolume contribution so that the model continues to explore the Pareto frontier instead of converging on a fixed trade-off.
Semantic representations are obtained by encoding concatenated item metadata—titles, genres, and tags—through the all-MiniLM-L6-v2 Sentence Transformer, yielding 384-dimensional L2-normalized vectors. User states are formed as the centroid of previously liked item embeddings within the same space, allowing inner-product similarity to serve as a direct measure of relevance and supporting zero-shot handling of unseen items. Candidate actions at each step combine nearest-neighbor retrieval around the current state with a controlled injection of long-tail items, keeping the action space tractable while preserving exposure diversity.
Evaluations on the MovieLens small dataset show that the resulting policy sustains higher variance in successive user-state trajectories than conventional single-objective DQN baselines. This sustained variance corresponds to measurable gains in the auxiliary objectives of diversity and fairness, accompanied by only modest reductions in engagement. The architecture thereby offers a concrete route toward recommender systems whose internal optimization already incorporates societal constraints without requiring post-hoc reweighting or external constraints.
Why it matters
Offers novel technical methods for ethical recommender design that align with Dutch and EU priorities on transparent, fair AI. Provides actionable RL architecture for practitioners balancing business and societal objectives.
