What Drives Interactive Improvement from Feedback?
06:00 · July 1, 2026 · arXiv cs.AI RSS
We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent setting, higher final accuracy can reflect useful feedback, but it can also arise from resampling, format correction, or additional test-time computation. To separate these effects, we introduce a controlled student-teacher protocol across Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1, evaluating thirteen open-weight models in both student and teacher roles. We compare external feedback, self-feedback, and unguided self-refinement, while varying interaction history, task difficulty, and teacher access to privileged task information. Across settings, we find that multi-turn improvement is often not evidence of feedback use: self-generated feedback adds little beyond unguided self-refinement, whereas the strongest external teachers produce substantially larger feedback-specific gains, suggesting that useful feedback must provide guidance beyond generic retry. Dense student-teacher interaction matrices further show that interactive gains are driven more by the student's ability to use feedback than by the teacher's identity, although teacher choice remains important for a fixed student. These results suggest that feedback-based agents should be evaluated against repeated-attempt baselines, and that ability to act on feedback, not merely feedback availability, is a central bottleneck for interactive improvement. We release our controlled student-teacher evaluation framework at https://j-lojek.github.io/feedback-generation-is-a-bottleneck/.
Summary
The paper examines whether natural-language feedback genuinely drives gains in multi-turn language agents or whether observed improvements largely reflect repeated sampling, format corrections, or extra test-time computation. To isolate these factors, the authors introduce a controlled student-teacher protocol that runs across four verifiable reasoning benchmarks—Omni-MATH, Codeforces, BBEH Linguini, and ARC-AGI1—and evaluates thirteen open-weight models in both student and teacher roles. In each episode a student model produces an initial answer; if incorrect, a teacher model supplies natural-language feedback without revealing the solution, and the student revises its attempt until success or a turn limit is reached.
The experiments compare three conditions: external teacher feedback, self-generated feedback, and unguided self-refinement. They also vary interaction history length, task difficulty, and whether the teacher receives privileged information such as ground-truth solutions. Results show that self-generated feedback rarely outperforms unguided retry, whereas the strongest external teachers produce clear additional gains. Dense interaction matrices across all model pairs further indicate that most performance variation stems from the student’s capacity to interpret and apply feedback rather than from differences in teacher quality, although teacher choice still matters once the student is fixed.
The authors conclude that feedback-based agents should be measured against repeated-attempt baselines and that the central bottleneck lies in the student’s ability to act on feedback, not in the mere availability of corrective signals. They release the evaluation framework to support such controlled comparisons.
Why it matters
This research is highly relevant for Dutch AI researchers and developers building LLM agents, as it provides a rigorous framework to evaluate feedback mechanisms. It aligns with the EU's push for robust, transparent AI by highlighting the need for proper baselines (repeated attempts) rather than misleading multi-turn accuracy metrics.
