When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models
06:00 · July 1, 2026 · arXiv cs.AI RSS
Reasoning models spend different amounts of useful computation across instances, but it remains unclear when a learned stopping rule improves over simple confidence or convergence thresholds. We study this question with LearnStop, a hidden-state-free checkpoint stopper for reasoning language models. At fixed budget checkpoints, LearnStop probes a short answer from the current reasoning prefix and predicts prefix correctness from online features such as answer confidence, entropy, prefix vote share, answer stability, and backtracking-marker density. Across 18 task-model settings spanning GSM8K, MATH-500, MMLU-Pro, AIME-90, GPQA, Qwen3, and DeepSeek-R1 distillations, the answer is task-dependent. On free-form math, learned multi-feature stopping improves the fixed-budget frontier and often beats scalar exits: on GSM8K with Qwen3-32B, the empirical frontier reaches a post-hoc peak adapt gain of +0.157, validation-selected operating points preserve positive gains, and the paired gain over the strongest scalar baseline is +0.028. On multiple-choice and very hard settings, scalar confidence, entropy, or stability rules are competitive or stronger. We therefore frame learned stopping not as a universal replacement for scalar exits, but as a tool whose value depends on trajectory structure. We further provide validation-selected operating points, paired bootstrap tests, finite-grid lost-correct risk calibration, cost accounting under KV-fork, prefix-cache, and black-box regimes, H100 serving profiles, checkpoint-schedule sweeps, transfer analyses, and robustness checks. The main practical finding is that learned stopping is useful when many questions become correct before full budget but do not exhibit a single reliable scalar stopping signal; its benefits largely disappear when confidence or answer convergence already solves the stopping problem.
Summary
Reasoning models allocate test-time compute unevenly across instances, often wasting tokens on easy problems while risking premature termination on harder ones. A range of early-exit heuristics—confidence thresholds, entropy monitors, answer-stability checks, and learned stoppers—have been proposed to reclaim that waste, yet prior evaluations use inconsistent protocols that make direct deployment comparisons difficult. The paper addresses this gap with a controlled study that enforces equal lost-correct risk across policies and accounts for probe overhead under different serving regimes.
LearnStop serves as the learned-policy instrument: at each checkpoint it forces a short answer from the current reasoning prefix, extracts eight prefix-observable features such as confidence, entropy, answer stability, and backtracking markers, and applies a logistic classifier to decide whether to stop. Because the method uses no hidden states, it remains portable across model families. All policies, learned and scalar alike, are calibrated on identical data splits with the same finite-grid correction for risk control and then evaluated on held-out test data at a matched lost-correct tolerance of α = 0.15.
Across 18 task–model combinations spanning GSM8K, MATH-500, MMLU-Pro, AIME-90, and GPQA on Qwen3 and DeepSeek-R1-distilled models, the results fall into three regimes. Learned multi-feature stopping delivers additional total-token savings of 3.2 to 21.2 percentage points on the four primary Qwen3 free-form math settings. Calibrated scalar exits remain competitive on multiple-choice MMLU-Pro, where many instances are already solved at the first checkpoint. On small, hard benchmarks such as AIME-90 and GPQA, no policy can be certified for aggressive early exit under the finite-sample risk bound.
A trajectory decomposition into early-solved, beneficial-thinking, harmful-overthinking, unsolved, and oscillating categories predicts which regime applies. Free-form math trajectories frequently oscillate and distribute correctness evidence across complementary signals, favoring the learned stopper; multiple-choice tasks concentrate evidence in a single confidence signal; and hard benchmarks contain too many permanently unsolved instances for any stopping rule to help. Cost accounting further qualifies the gains: the same policy that saves 32 % of tokens under KV-cache forking incurs a 121 % overhead under black-box repeated prefilling, underscoring that serving architecture determines whether learned stopping is worthwhile in practice.
Why it matters
This research is highly relevant for Dutch AI researchers and engineers focused on optimizing LLM inference costs and promoting sustainable AI. The detailed cost-aware analysis and practical serving profiles offer actionable methodologies for deploying efficient AI models in resource-constrained or enterprise environments within the Netherlands.
