Request-Level Energy Attribution for Batched LLM Serving
06:00 · August 4, 2026 · arXiv cs.AI RSS
Batched LLM serving improves throughput but complicates energy accounting. GPU power telemetry is aggregate, whereas sustainability reporting, chargeback, and workload analysis often require request-level energy charges. Existing inference-energy benchmarks report model-, phase-, or token-level energy, and recent carbon-accounting work motivates Shapley fairness conceptually. Neither provides measured request-level ground truth, so how far the accounting rules used in practice deviate from a fair allocation has remained unknown. We present JouleShare, an attribution framework with two components. An offline harness establishes this ground truth by replaying request subsets under vLLM with a reproducible protocol, integrating GPU power telemetry, and computing exact Shapley energy for each request. A lightweight calibration model, JCalib, then learns to predict Shapley shares from cheap request features for use at serving time. Across 16 model/workload runs, token-proportional attribution differs from exact Shapley by 0.440 normalized L1 on average under static batching and by 0.458 under continuous batching, a gap that reproduces across three data-center GPUs. JCalib reduces this error to 0.116 under static batching and 0.177 under continuous batching, below even a standalone-measurement baseline that is unavailable online, while preserving exact batch-energy efficiency. Sampled Shapley extends the measured reference to larger group sizes, where the gap persists and a single offline calibration remains the most accurate deployable rule. The results show that token attribution is not a reliable proxy for marginal energy under batched execution, and that measured Shapley ground truth can calibrate low-cost request features toward fairer attribution.
Summary
Batched LLM serving improves GPU utilization by processing multiple requests together, yet this interaction makes it difficult to assign energy consumption accurately to individual requests. GPU power telemetry provides only aggregate measurements, while sustainability reporting, chargeback mechanisms, and workload analysis require request-level attribution that reflects each request’s marginal contribution. Token-proportional allocation, a common practical rule, divides batch energy according to token counts but ignores how requests interact during prefill and decode phases under shared execution.
JouleShare addresses this gap with an offline measurement harness and a lightweight predictor. The harness replays every subset of a request group on vLLM under a fixed protocol, records active GPU energy via power telemetry, and computes exact Shapley values for each request. These measured values serve as ground truth. A calibration model called JCalib is then trained on cheap request features to approximate the Shapley shares for online use without repeating the full subset measurements.
Across 16 model and workload combinations covering static and continuous batching, token-proportional attribution deviates from the measured Shapley allocation by an average normalized L1 error of 0.440 under static batching and 0.458 under continuous batching. The discrepancy persists across three data-center GPUs and model scales up to 14B parameters. JCalib reduces the error to 0.116 and 0.177 respectively, with negligible added latency, while still summing exactly to the observed batch energy. Sampled Shapley evaluations confirm that the gap remains at larger group sizes and that a single offline calibration continues to outperform simpler rules in practice.
Why it matters
Directly actionable for Dutch AI teams optimizing sustainable LLM inference under EU energy-reporting rules; provides measured fairness baselines and reproducible protocols relevant to ethical AI and data-center efficiency goals.
