Forecasting Side Effects of Activation Steering
06:00 · August 13, 2026 · arXiv cs.AI RSS
Activation steering modifies a language model by adding a learned direction to its hidden activations, enabling targeted behavioral changes without retraining. While effective, steering often produces unintended side effects on other behaviors, making it difficult to deploy safely. We therefore ask: can these side effects be forecasted before steering is applied? We answer this question by constructing a cross-effect matrix over a taxonomy of 67 behaviors across three open-weight language models. We find that side effects are common, structured, and often asymmetric, revealing interactions that cannot be explained by existing similarity-based heuristics. Despite this complexity, we show that side effects are largely predictable before steering is performed. Their magnitude depends primarily on the target behavior, while their direction can be forecasted from the model's unsteered representations with substantially higher accuracy than simple baselines. Our results demonstrate that activation steering has systematic and forecastable side effects, enabling proactive safety auditing and more informed deployment of steering interventions.
Summary
Activation steering alters large language models at inference time by adding a learned direction to hidden activations, allowing targeted changes to behaviors such as concision or refusal without retraining or prompt modification. While this approach is computationally lightweight, it frequently produces unintended changes in other behaviors. The paper addresses whether these side effects can be anticipated before any steering is applied.
To investigate this, the authors construct cross-effect matrices that record how steering each behavior in a taxonomy of 67 influences every other behavior. Experiments cover three open-weight models—Gemma-3-4B, Gemma-3-12B, and Qwen2.5-7B—using validated steering directions at selected layers and multiple prompt contexts designed to surface a range of behaviors. The resulting matrices reveal that side effects are widespread, low-dimensional, and often asymmetric: steering behavior A may amplify B while steering B suppresses A. Existing similarity-based heuristics, such as cosine similarity between steering vectors, account for at most 23 percent of observed couplings and therefore fail to capture the structure.
Despite this complexity, side effects prove largely forecastable from unsteered model representations alone. The magnitude of an effect depends primarily on the target behavior, while its direction—amplification or suppression—can be predicted by propagating a steering vector through a learned map and decoding the result with linear probes. This method achieves 68–78 percent accuracy on major side effects, substantially above simple baselines, and extends naturally to behaviors that cannot themselves be steered. The framework therefore supports systematic risk assessment of steering interventions prior to deployment.
Why it matters
Directly addresses ethical and safe LLM deployment central to Dutch/EU AI priorities; the forecasting method is actionable for researchers auditing steering interventions on open models.
