Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning
06:00 · July 21, 2026 · arXiv cs.AI RSS
Context reranking and pruning have become essential for improving the efficiency of modern Retrieval-Augmented Generation (RAG) systems, yet an interpretable and unified framework remains underexplored. Previous work has primarily emphasized lexical retrieval, cross-encoder architectures, model distillation, and Low-Rank Adaptation (LoRA), mostly relying on heuristic loss functions and empirical attribution. This paper presents Shapley Context Pruning (SCP), a novel framework for context reranking that establishes a cooperative-game-theory perspective for importance attribution by modeling the context as a cooperative game. Balancing the trade-off between fine-grained and coarse-grained representations, we employ a Deep Sets architecture to approximate a permutation-invariant value function at the sentence level, utilizing pre-trained language models as sentence embedders and optimizing via a pairwise margin ranking loss. To ensure practical scalability without sacrificing mathematical rigor, we leverage Monte-Carlo sampling for efficient training and inference, providing formal theoretical error bounds and sample complexity guarantees for preserving Top-K subset rankings. Furthermore, we conduct comprehensive experiments-spanning supporting-sentence recall, Needle-in-the-Haystack (NIAH) evaluations, long-context QA, and multi-hop reasoning-alongside rigorous ablation studies on embedding quality and attribution strategies. The model achieves competitive downstream QA performance against robust baselines.
Summary
Shapley Context Pruning (SCP) reframes context reranking and pruning in retrieval-augmented generation systems as a cooperative game in which individual sentences act as players whose marginal contributions are quantified by Shapley values. Rather than relying on heuristic loss functions or empirical attribution scores, the framework learns a permutation-invariant value function that evaluates arbitrary subsets of sentences and then attributes importance according to their average contribution across coalitions. This approach directly addresses the gap between local relevance signals and holistic utility that arises when long, noisy contexts are passed to large language models.
The architecture separates concerns into an embedder, a lightweight value network, and a Shapley estimator. Pre-trained sentence embeddings feed a Deep Sets model whose output approximates the value of any coalition; training uses a pairwise margin-ranking loss that encourages higher values for subsets containing supporting evidence. Monte-Carlo sampling renders both training and inference tractable while preserving formal error bounds and sample-complexity guarantees for recovering the correct top-K ranking. The resulting value network contains only three million parameters and operates independently of the underlying embedder, allowing it to serve as a modular, coarse-grained filter before more expensive cross-encoder stages.
Empirical evaluation covers supporting-sentence recall, needle-in-a-haystack retrieval, long-context question answering, and multi-hop reasoning benchmarks such as MuSiQue, 2WikiMultihop, and HotpotQA. Ablation studies examine the effects of embedding quality, sampling budget, and alternative attribution schemes including leave-one-out baselines. The method achieves competitive downstream performance while remaining substantially lighter than approaches that fine-tune or distill full language models. Additional analyses explore robustness under semantic redundancy and illustrate how the learned value function yields an interpretable “landscape” of context contributions.
Beyond immediate pruning use cases, the work supplies a theoretical lens for context engineering that treats information aggregation as coalition formation rather than simple concatenation or attention weighting. It also surfaces open questions about hierarchical context structures and bridging sentences that future work may address with the same cooperative-game formulation.
Why it matters
Directly addresses RAG efficiency and interpretability, aligning with Dutch strengths in ethical AI and SME adoption of advanced retrieval techniques. Researchers can apply the permutation-invariant value function and Top-K pruning algorithms to local deployments.
