Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
06:00 · June 25, 2026 · arXiv cs.AI RSS
Large language models (LLMs) reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. Prior work analyzes failure at the step, chunk, or sentence level, or at tokens where failure has already occurred. Neither identifies the precise token that triggers the shift toward failure. We introduce the cliff token, a token where the token-wise potential drops significantly under an adaptive threshold that scales with the local token-wise potential, based on a one-sided two-proportion z-test. Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to between 0.71 and 1.00. We further introduce a cliff taxonomy of deterministic, uncertain, and sampled-off cliffs, defined by greedy choice and token entropy. Each type has distinct probabilistic characteristics, and the taxonomy generalizes across model scales. Finally, we validate the taxonomy via single-token preference optimization at cliff positions (Cliff-DPO). Trained on GSM8K, Cliff-DPO improves accuracy across benchmarks by up to +6.6. Optimizing at uncertain and sampled-off cliffs improves reasoning, while deterministic cliffs do not.
Summary
Large language models often produce correct answers on mathematical reasoning tasks yet generate divergent traces on identical problems, with some paths failing due to isolated decisions. Prior analyses have examined failures at the level of steps or sentences, or after success probability has already collapsed, without isolating the exact token that initiates the downward shift. The paper defines a cliff token as the position where token-wise potential—the probability, estimated from 64 rollouts, that the trace will still reach the ground-truth answer—drops by a statistically significant margin.
The detection method replaces fixed probability thresholds with an adaptive criterion derived from a one-sided two-proportion z-test at 95 percent . A token qualifies as a cliff when the observed drop exceeds a baseline of 0.1 plus a term that grows with local sampling variance, thereby reducing false positives in high-uncertainty regions. Experiments spanning seven models from the Qwen3, Llama-3, and Gemma-3 families and three benchmarks (GSM1K, MATH500, AIME 2025) demonstrate that these tokens function as causal triggers: deleting the first cliff token and resampling restores pass@64 to 1.0, whereas retaining it caps recovery between 0.71 and 1.00.
The authors further partition cliff tokens into a taxonomy of deterministic, uncertain, and sampled-off types according to whether the token matches the greedy choice and the magnitude of its entropy. Deterministic cliffs remain stable across model scales, while uncertain and sampled-off cliffs vary with capacity and training gaps. Single-token preference optimization performed exclusively at cliff positions (Cliff-DPO) and trained on GSM8K lifts accuracy on held-out benchmarks by as much as 6.6 points when uncertain or sampled-off cliffs are targeted; optimization at deterministic cliffs yields no gain. The results establish that token-level statistical signals can both diagnose and selectively mitigate reasoning failures.
Why it matters
High technical depth and novelty in LLM failure analysis make it directly actionable for Dutch AI researchers developing reliable reasoning systems; the methodology and Cliff-DPO results are reproducible with public benchmarks and generalize across model scales.
