Controlling Tool Use with Heading-Specific Activation Steering
06:00 · July 8, 2026 · arXiv cs.AI RSS
Tool-augmented large language models extend their capabilities beyond parametric knowledge through external tools, but tend to invoke them unnecessarily. We investigate whether tool-use decisions have any stable internal representation that can be extracted and manipulated, a question that is non-trivial given that tools exist entirely in context at inference time and have no direct encoding in model weights. We show that steering vectors extracted from heading-anchors positions exert bidirectional causal control over tool-invocation behavior across five open-source models and three domains, suppressing unnecessary tool use most effectively in domains where parametric reasoning suffices. However, geometric analysis reveals that this causal effectiveness does not correspond to clean linear structure: tool-invocation steps exhibit diffuse, bimodal alignment with the suppression vector rather than the consistent negative alignment a linear encoding account would predict, and different tool types recruit largely distinct internal signatures with low cross-tool feature overlap. We hypothesize these geometric properties are indicative of the non-parametric nature of tools, and distinguish tool-use steering vectors from those extracted for parametrically grounded concepts. The relationship between this geometric irregularity and the observed causal effectiveness remains an open question.
Summary
Tool-augmented large language models frequently call external functions such as web search or code execution even when their internal parameters already contain the necessary information. The authors examine whether the decision to invoke a tool leaves a detectable trace in the model’s residual stream, despite the fact that tools appear only in the prompt and carry no fixed weights. They extract steering vectors by recording hidden states at the moment the model is about to emit a structured section heading such as “### Search” or “### Reasoning,” then apply these vectors through activation addition or orthogonalization at a chosen layer.
Across five open-source instruction-tuned models and three task domains, the resulting vectors produce reliable bidirectional effects. Subtracting the vector reduces unnecessary tool calls below the prompt-only baseline, while projecting activations orthogonal to the vector increases tool use above it. Suppression proves strongest in settings where the model can already solve the query from parametric knowledge and weakens when external retrieval or user clarification is genuinely required. Larger models exhibit greater resistance to the intervention in all domains.
Geometric inspection nevertheless shows that the causal efficacy does not rest on a simple linear encoding. Tool-invocation steps display a diffuse, bimodal distribution of cosine similarities with the suppression vector rather than the uniform negative alignment expected under a linear representation account. In addition, vectors derived from different tool categories occupy largely separate subspaces, with little feature overlap between them. The authors interpret these irregular geometric properties as a consequence of tools being supplied entirely at inference time, and they note that the precise relationship between this geometry and the observed behavioral control remains unresolved.
Why it matters
This research provides advanced techniques for controlling LLM agent behavior, which is crucial for Dutch AI researchers developing reliable and efficient AI systems. Understanding and steering tool use aligns with the EU's push for transparent and predictable AI deployments.
