Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models
06:00 · August 19, 2026 · arXiv cs.AI RSS
Small Language Models (SLMs) are increasingly deployed in resource-constrained, privacy-sensitive settings, where safety and bias failures can cause security and societal risks. However, existing AI safety\slash security\slash compliance benchmarks are designed for large language models that may not transfer reliably to SLMs. We therefore ask: Can these benchmarks effectively and reliably evaluate SLMs? To answer this question, we conduct a large-scale assessment of the effectiveness and robustness of these automated pipelines by evaluating five widely used benchmark suites across 26 open-source SLMs under a unified judging rubric, which assigns a score of 0, 1, or 0.5 to harmful, safe, or ambiguous/irrelevant responses, respectively. Across the benchmarks, ambiguous judgments dominate and correlate with prompt complexity and model architecture, indicating that {\em LLM-centric safety benchmarks are insufficient as standalone evidence for SLM safety assessment}. In general, the ambiguity rate increases with lexical density, output perplexity, and output length and decreases with lexical sophistication, self-coherence, and reply-prompt similarity. This reveals a capability-safety confound that mixes model capability with apparent safety. Since ambiguity is prevalent, aggregate mean-score leaderboards are mathematically brittle: model rankings change significantly under reasonable ambiguity treatments, even when the underlying outputs remain unchanged.
Summary
This study examines whether safety benchmarks originally developed for large language models remain valid when applied to small language models. The authors evaluate five widely used benchmark suites on 26 open-source SLMs, generating more than 741,000 model-prompt pairs and scoring responses with a unified automated judge that assigns 0 for harmful output, 1 for safe output, and 0.5 for ambiguous or irrelevant replies. Across all suites, ambiguous judgments form the dominant category and correlate strongly with both prompt complexity and model architecture.
The analysis reveals a clear capability-safety confound. Ambiguity rises with lexical density, output perplexity, and response length, while it falls with greater lexical sophistication, self-coherence, and similarity between reply and prompt. These patterns indicate that automated judges often flag evaluation difficulty rather than genuine safety violations. Because ambiguous cases are prevalent, conventional mean-score leaderboards prove mathematically unstable: modest changes in how ambiguous outputs are treated can reorder model rankings even when the underlying generations remain identical.
The authors conclude that existing LLM-centric pipelines cannot serve as standalone evidence of SLM safety. They recommend developing evaluation methods that explicitly account for output quality and prompt difficulty, and they note that simpler prompt sets may temporarily mask the problem while more demanding suites expose it. The work underscores the need for SLM-specific safety assessments that separate capability limitations from actual risk behavior.
Why it matters
This article is highly relevant for Dutch AI researchers and compliance officers navigating the EU AI Act, as it exposes critical flaws in using standard LLM safety benchmarks for SLMs. It provides actionable insights into the capability-safety confound, urging practitioners to rethink how they validate SLMs for privacy-sensitive and resource-constrained deployments in the Netherlands.
