How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories
06:00 · August 3, 2026 · arXiv cs.AI RSS
Understanding how computational effort is allocated across individual chain-of-thought (CoT) reasoning steps remains an open challenge: existing interpretability methods rely on output-level signals or collapse processing depth into a single trajectory-level scalar, leaving step-wise effort opaque. We propose Step-Aware Reasoning Energy (SARE), a geometric framework that quantifies effort at the granularity of individual CoT steps via Centered Kernel Alignment (CKA) between Gram matrices of token hidden states across adjacent transformer layers, capturing inter-token relational structure without requiring eigenvector alignment or cluster correspondence. SARE further contextualizes this energy within reasoning's semantic progression by modeling CoT trajectories as transitions among latent semantic states. Across six reasoning benchmarks and three open-weight LLMs, we find that reasoning energy is highly non-uniform across step types, exhibiting phase-like transitions invisible to trajectory-level metrics; incorrect trajectories show systematically lower energy at critical reasoning junctions; and SARE-based features match or outperform output-based confidence baselines in most settings, indicating that internal geometric dynamics encode predictive information beyond surface-level signals.
Summary
The paper addresses a persistent gap in understanding how large language models distribute internal computation during chain-of-thought reasoning. While CoT prompting produces explicit intermediate steps, most interpretability techniques either examine final token probabilities or reduce an entire trajectory to a single scalar, leaving the effort invested at each individual step opaque.
Step-Aware Reasoning Energy (SARE) tackles this by treating each reasoning step as a set of tokens whose hidden-state representations evolve across transformer layers. For every pair of adjacent layers, the method constructs Gram matrices that encode pairwise token similarities and then applies Centered Kernel Alignment to quantify how much that relational geometry changes. A step whose token relationships continue to reorganize through many layers registers high energy; one that stabilizes early registers low energy. This geometric signal preserves inter-token structure without requiring alignment of eigenvectors or cluster assignments across layers.
The framework further situates these energy measurements within the semantic progression of reasoning. CoT trajectories are modeled as sequences of transitions among latent semantic states discovered through unsupervised clustering of final-layer representations. This dual view—geometric effort at each step combined with the semantic role of that step—reveals structured, non-uniform energy profiles that trajectory-level metrics obscure. Early setup and final synthesis steps typically anchor the extremes, while mid-trajectory factual retrieval steps show moderate, stable energy.
Evaluations across six benchmarks spanning mathematical, commonsense, and multi-hop reasoning, conducted on LLaMA-3.2-3B, Phi-4-mini, and Gemma-3-4B, show that incorrect trajectories consistently exhibit lower energy at critical junctions such as verification and compositional reasoning. When used as features for failure prediction, SARE-based signals match or exceed output-based baselines including token log-probability, entropy, and perplexity on most model–benchmark combinations. The work supplies the underlying mathematical formulations, experimental protocols, and reproducibility artifacts needed for further investigation of step-level computational dynamics in open-weight models.
Why it matters
Provides actionable interpretability tools for Dutch researchers and advanced practitioners working on reliable LLM reasoning; aligns with NL/EU emphasis on transparent and ethical AI; novel step-level geometric analysis offers insights beyond trajectory-level metrics.
