Beyond Liars' Bench: The Impact of Lie Typology, Depth, and Sparsity on Deception Detection in LLMs
06:00 · July 24, 2026 · arXiv cs.AI RSS
Training probes to detect deceptive outputs from large language models is still an open problem. Recent work has demonstrated that detection probes fail especially in out-of-domain scenarios -- training on one type of lie does not transfer well to deception scenarios involving other types of lies. In this work, we conduct a systematic study on how various factors impact detection performance: representation depth, probe expressivity, sparse feature representations, and the lie typology of the training data. To this end, we augment standard benchmark training data with a supplementary dataset containing diverse types of deception, including fabrication, omission, and exaggeration examples. Analyzing these factors across seven probe types, our experimental results show that the optimal representation depth is highly dataset-dependent, more expressive probes provide only selective gains over linear baselines, and sparse autoencoder features perform similarly to dense hidden states. Ultimately, we demonstrate that the choice of training data and lie typology substantially changes detectability, highlighting that deception detection is a highly representation-dependent problem.
Summary
Recent research on detecting deceptive outputs from large language models has shown that probes trained on internal activations often fail to generalize across different deception scenarios. This paper extends that line of work by systematically examining how four factors shape detection performance: the depth at which representations are extracted, the expressivity of the probe, the use of sparse autoencoder features, and the typology of the lies used for training.
The authors build on the Liars’ Bench benchmark and supplement it with the DolusChat dataset, which introduces examples of fabrication, omission, and exaggeration. They evaluate seven probe families on these combined resources, comparing linear baselines against more expressive models and testing both dense hidden states and sparse autoencoder representations. The experiments track how performance varies with layer depth and with the specific category of deception present in the training data.
Results indicate that the best representation depth shifts depending on the dataset, that gains from non-linear probes remain selective rather than consistent, and that sparse autoencoder features yield detection performance comparable to standard dense activations. Most notably, swapping the lie typology in the training set produces substantial changes in detectability, confirming that no single probing strategy works reliably across contexts.
The study therefore frames deception detection as a representation-dependent problem whose success hinges on the alignment between training data characteristics and the target deployment setting.
Why it matters
This research is highly relevant for Dutch AI researchers and practitioners focused on AI safety, transparency, and alignment. Given the EU AI Act's strict requirements for AI accountability, methodologies to detect internal model deception are crucial for developing compliant and trustworthy LLM applications in the Netherlands.
