Probabilistic Concept-Aware Steering for Trustworthy LLM Inference
06:00 · July 22, 2026 · arXiv cs.AI RSS
Steering vectors (SVs), an inference-time intervention technique for large language models (LLMs), guide the generation process by adding a concept-specific direction vector to intermediate activations during inference. However, existing SV methods frequently yield representation-incoherent behaviors that undermine interpretability and fine-grained control, largely because prior work has focused on binary positive-negative steering evaluation while employing discrete clustering metrics that fail to capture the continuous spectrum of semantic alignment. In this work, we present the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference. PCS preserves original task competence while providing controllable, safety-oriented semantic bias through concept-driven steering-vector retrieval and probabilistic strength calibration.
Summary
Steering vectors offer a lightweight way to guide large language models at inference time by adding a learned direction to hidden activations, yet conventional approaches often produce incoherent outputs because they rely on fixed coefficients and binary positive-negative evaluations that ignore the continuous nature of semantic alignment. The Probabilistic Concept-Aware Steering (PCS) framework addresses these shortcomings through an automated pipeline that first detects the target concept in an input prompt, retrieves the corresponding steering vector, and then applies an affine transformation at a chosen mid-layer of the transformer.
Instead of a static intervention strength, PCS models the coefficient as a Gaussian distribution whose mean and variance are conditioned on the cosine similarity between the prompt’s concept representation and the steering direction. This similarity-conditioned sampling replaces manual tuning or brute-force search, reducing both under- and over-steering while preserving the model’s original task performance. The method remains model-agnostic and requires no parameter updates, making it applicable across different scales without retraining.
Evaluations across multiple model sizes and ten alignment and safety datasets show that the adaptive calibration yields more than 30 percent higher direction accuracy in previously anti-steerable domains, exceeds 89 percent steering accuracy under established metrics, and delivers roughly threefold gains in inference efficiency relative to dynamic baselines. The work further indicates that steering efficacy tends to peak at moderate rather than maximal semantic alignment, underscoring the value of context-sensitive rather than uniform intervention.
Why it matters
Directly addresses trustworthy, interpretable LLM control—an EU/NL priority—via a reproducible, model-agnostic method that Dutch researchers and advanced practitioners can apply to ethical AI deployment and SME solutions.
