Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment
06:00 · July 24, 2026 · arXiv cs.AI RSS
We present an empirical benchmark evaluating how five large language models assess multisensor physical hazard data. Testing 60 scenarios across three categories - multi-sensor joint assessment, response proportionality, and pattern disambiguation - with 1,800 API calls at temperature 0.0, we find that all tested models consistently produced no precautionary warning signal across the tested scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near-perfect accuracy on single-sensor threshold violations. All five models (ChatGPT-4o, Gemini 2.5 Flash, DeepSeek, Kimi, Llama 3.1 8B) score near zero on Category A multi-sensor scenarios (Q2: 0.000-0.208; Q3: 0.000-0.592) compared to strong performance on single-sensor scenarios (Category B Q1: 0.975-1.000). Structured tabular formatting shows no consistent advantage over plain prose; ChatGPT-4o performs significantly better under prose (p = 0.001). These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems.
Summary
The paper reports an empirical benchmark that measures how five large language models interpret streams of numerical sensor readings against established occupational safety thresholds. Sixty scenarios were constructed across three categories—multi-sensor joint assessment, response proportionality, and pattern disambiguation—producing 1,800 API calls at temperature 0.0. The models examined were ChatGPT-4o, Gemini 2.5 Flash, DeepSeek, Kimi, and Llama 3.1 8B.
All models performed near ceiling on single-sensor threshold checks, scoring between 0.975 and 1.000. In contrast, they produced almost no precautionary signals in the twenty multi-sensor scenarios where each individual reading remained below its limit yet the combined exposure, calculated with the OSHA additive index, exceeded unity. Scores on the corresponding hazard-classification and action-recommendation questions fell to the ranges 0.000–0.208 and 0.000–0.592 respectively. Structured tabular input offered no consistent benefit over plain prose; only ChatGPT-4o showed a statistically significant preference for the prose format.
The results indicate that current models reliably execute isolated arithmetic comparisons yet fail to integrate multiple sub-threshold elevations into the combined-exposure logic used by occupational-hygiene standards. The authors note direct consequences for any deployment of these models in industrial monitoring, building automation, or environmental safety systems that must detect cumulative rather than isolated hazard conditions.
Why it matters
The research is highly relevant for Dutch AI researchers and practitioners developing IoT and industrial safety systems, especially given the EU's strict occupational health standards and the AI Act's focus on high-risk safety components. It provides actionable insights and a reproducible benchmark to test LLM reliability in multi-sensor environments.
