Towards Evaluation of Implicit Software World Models in Coding LLMs
06:00 · June 29, 2026 · arXiv cs.AI RSS
Software engineering, whether performed by humans or by AI agents, requires reasoning about how software behaves. We call the internal model that supports such reasoning the software world model, and view current code-execution benchmarks as covering one well-studied slice of it -- control flow. In this paper, we take a step toward a broader evaluation by shifting the observable axis to execution resources: alongside test outcome and exception class, we predict peak memory, wall-clock time, and ranked profiler outputs at method and line granularity. We use SWE-bench Verified as the source of data to hold the test close to real-world software engineering tasks. All tested models, frontier ones included, show modest performance and brittle behaviour, suggesting a notable lack of understanding of how software is executed, as opposed to how its source code is written.
Summary
This paper proposes a broader way to probe the implicit software world models that coding LLMs acquire during pre-training. Rather than treating successful code generation or control-flow prediction as sufficient evidence of execution understanding, the authors argue that genuine software reasoning also requires anticipating resource consumption and profiling behaviour under realistic conditions. They therefore shift evaluation from isolated function outputs to the prediction of test outcomes, exception types, peak memory, wall-clock time, and ranked method- and line-level profiles.
The evaluation is grounded in SWE-bench Verified, a curated set of 500 real GitHub issues spanning twelve Python repositories. From failing-then-fixed test pairs the authors derive 435 library-level examples that remain within a 500 k-character context window. Ground-truth observables are collected by injecting a custom tracer into the original SWE-bench Docker environments and executing each test both before and after the gold patch. Resource quantities are recorded on a clean run to reduce measurement noise, while profiler rankings capture the top twenty entries for time and memory at both method and line granularity.
When frontier and open-weight models are asked to return these quantities in a single JSON response, performance remains modest across all metrics. Binary classification of test failure yields limited F1 scores, linear calibration on log-scaled time and memory reveals poor slope and high error, and ranking metrics for profiler outputs show that models rarely surface the actual hottest methods or lines. The consistent brittleness suggests that current coding LLMs still lack a robust internal model of how software consumes resources once it leaves the textual domain of its source.
Why it matters
It provides AI researchers with a new framework for evaluating coding LLMs beyond standard metrics. For the Dutch AI ecosystem, which emphasizes efficient and robust AI engineering, improving how models predict execution resources is crucial for developing sustainable and optimized software.
