SchemaRouter: Field-Aware Tool Routing for Efficient Heterogeneous Agentic RAG
06:00 · August 25, 2026 · arXiv cs.AI RSS
Heterogeneous agentic retrieval-augmented generation (RAG) systems increasingly orchestrate external APIs, internal databases, vector stores, and graph stores. Exposing all tool descriptions to an LLM agent, or selecting tools only by vector similarity, causes two costly failures: over-fetching, which increases payload size, token use, and latency, and under-fetching, which omits fields needed to answer the query. We present SchemaRouter, a lightweight routing layer that represents tools, endpoints, parameters, response fields, domain concepts, units, provenance, and license policies as a schema graph. Given a query, SchemaRouter emits an executable tool plan specifying which tools to call and which fields to retrieve. A small LLM extracts intent, concepts, and source constraints, while field selection is deterministic over the graph through intent-group projection and concept-field matching with an alias layer. On a materials-science benchmark of 110 queries, SchemaRouter achieves answer accuracy of 0.71, matching fetch-everything within overlapping confidence intervals and exceeding prompt-all's 0.66, though their intervals overlap. It uses 227 retrieved-context tokens versus 2,066 for fetch-everything and achieves 2.7x lower end-to-end latency than prompt-all. It also obtains the best tool-exact rate of 0.93 and parameter validity of 1.0. SchemaRouter grounds provenance and license information in 62 percent of answers, compared with approximately 0 percent for all baselines. We also find that minimizing selected-field count is counterproductive: it reduces answer accuracy to 0.56 with negligible token savings, while recall-preserving projection restores top accuracy. SchemaRouter improves efficiency, schema-size-independent scaling, and verifiable provenance/license-grounded answering at competitive accuracy.
Summary
SchemaRouter addresses a practical bottleneck in heterogeneous agentic RAG systems that must coordinate external APIs, internal databases, vector stores, and graph stores. Conventional approaches either expose every tool description to the LLM or rely on vector similarity for tool selection. Both produce over-fetching, which inflates payload size and latency, or under-fetching, which omits fields required for a complete answer.
The method models the entire tool inventory as a schema graph whose nodes represent tools, endpoints, parameters, response fields, domain concepts, units, source types, and license policies, with edges capturing relationships and aliases. A compact LLM first extracts query intent, relevant domain concepts, and any explicit source constraints. Field selection then proceeds deterministically through intent-group projection and concept-to-field matching, yielding an executable plan that specifies exactly which tools to invoke and which fields to retrieve.
Evaluated on a materials-science benchmark of 110 queries, SchemaRouter delivers answer accuracy of 0.71, statistically indistinguishable from a fetch-everything baseline while exceeding the 0.66 accuracy of a prompt-all router. It reduces retrieved-context tokens from 2,066 to 227 and cuts end-to-end latency by a factor of roughly 2.7 relative to prompt-all. The router also records the highest tool-exact rate (0.93) and perfect parameter validity, while surfacing verifiable provenance and license metadata in 62 percent of final answers—information absent from all baseline outputs.
An additional finding concerns field-selection strategy: aggressively minimizing the number of selected fields lowers downstream accuracy to 0.56 with negligible token savings. A recall-preserving projection recovers the top accuracy tier, indicating that token count and answer quality, rather than field-count parsimony, are the metrics that matter for production routing.
Why it matters
This research is highly relevant for Dutch AI practitioners building enterprise RAG systems, offering a scalable method to reduce token costs and latency. Furthermore, its strong emphasis on verifiable data provenance and license grounding aligns perfectly with EU AI regulations and ethical AI standards prioritized in the Netherlands.
