H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrases
06:00 · August 4, 2026 · arXiv cs.AI RSS
Terminology-intensive retrieval, especially in medical settings, depends on preserving multi-word entities, abbreviations, numerical constraints, and compositional concepts. However, existing representations lie at two extremes: single-vector retrievers often over-compress local relevance signals, while token-level late interaction retains every tokenizer subword at substantial indexing, storage, and scoring cost. This mismatch raises a natural question: can context-dependent phrases provide a useful retrieval unit between global vectors and tokens? We introduce H+ Embedding, a unified multi-granularity retriever that predicts variable-length phrase partitions, preserves uncovered tokens as singletons, and applies importance-guided unit selection with weighted MaxSim interaction. Across 16 scientific, medical, and bilingual tasks, its phrase retrieval branch exceeds the global retrieval branch by 6.91 macro nDCG@10. It also nearly matches Token while using 13.7% fewer document vectors and outperforms content-independent grouping rules under moderate vector budgets. Context-dependent phrase interaction therefore provides an intermediate quality-cost point between global compression and token-level interaction for practical retrieval systems.
Summary
H+ Embedding addresses a persistent granularity mismatch in terminology-intensive retrieval, particularly in medical and scientific domains where queries often hinge on multi-word entities, abbreviations, numerical constraints, and compositional phrases. Single-vector retrievers compress an entire passage into one embedding for efficient nearest-neighbor search, yet they frequently suppress the distinct local signals that matter for precise matching. Token-level late-interaction models preserve those signals through contextualized MaxSim scoring, but they retain every subword as an independent unit, driving up indexing, storage, and scoring costs in proportion to sequence length.
The model introduces context-dependent phrases as an intermediate retrieval unit. A shared bidirectional encoder, initialized from Qwen3-0.6B-Base, produces contextual token states that feed three views: a single global vector, variable-length phrase vectors, and auxiliary lexical weights. A linear-chain Conditional Random Field predicts BIO boundaries for each token sequence, grouping tokens that should act jointly while leaving uncovered tokens as singletons. This partition is then pruned by importance-guided selection so that only the most salient units are retained under a fixed vector budget. Phrase vectors are formed by mean-pooling within each unit and L2-normalized; retrieval uses a weighted MaxSim interaction that accounts for unit importance on both query and document sides.
Evaluated under candidate-constrained settings across 16 scientific, medical, and bilingual tasks, the phrase retrieval branch improves macro nDCG@10 by 6.91 over the global branch alone. It nearly matches the performance of full token-level interaction while storing 13.7 percent fewer document vectors and outperforms fixed grouping heuristics such as bigrams or random selection once moderate vector budgets are available. The design therefore supplies a practical operating point between the compression of global embeddings and the cost of exhaustive token interaction.
Why it matters
This research is highly relevant for Dutch AI researchers and engineers building Retrieval-Augmented Generation (RAG) systems, particularly in the healthcare and scientific sectors. It offers a mathematically rigorous, cost-effective methodology to improve domain-specific search without the massive storage overhead of traditional token-level models.
