Scenario Generation for Testing of Autonomous Driving Systems Using Real-World Failure Records
06:00 · July 1, 2026 · arXiv cs.AI RSS
To ensure safe on-road behavior, pre-deployment testing and failure discovery of Autonomous Driving Systems (ADS) is crucial. Present day simulation based testing methods focus largely on mathematical models for efficient search of optimal scenarios, assuming a fixed scenario representation. On the other hand, real-world testing involves substantial manual effort to design scenario templates for testing. These templates represent distinct failure scenarios consisting of pre-deployment vehicle movements, map types, etc. Historical failure records for ADS are a reliable source of real-world failure conditions, which can be used for scenario generation. In this work, we propose a scenario generation pipeline using categorical and contextual information available from historical records in natural language format. Our approach consists of modular LLM based synthetic scenario generation, compatible with the testing constraints of a given system. We successfully apply our method to generate a diverse set of scenarios for testing autonomous navigation on Metadrive simulator using the NHTSA ADS crash records. Our approach results in accurate and diverse scenario generation with a combination of 4 road types, 3 non ego vehicle movement types, including on road anomalies in the form of working zones. Generated scenarios align with the provided testing conditions, and reveals interesting failures of the system within a limited testing budget of 20 scenarios. Code is available at https://github.com/anjaliParashar/crash2scenario.
Summary
The article introduces a pipeline that converts historical crash records into simulation-ready test scenarios for autonomous driving systems. Rather than relying on fixed mathematical templates or exhaustive manual design, the method extracts both categorical attributes—such as road type and pre-crash vehicle movements—and contextual details from natural-language narratives in existing incident reports. Large language models then translate these elements into synthetic scenarios that respect the operational constraints of a given ADS under test.
The approach is modular, allowing the generated scenarios to be adapted to different simulators and testing budgets. In the reported validation, the pipeline processed NHTSA ADS crash records to produce scenarios for the MetaDrive simulator running an IDM-based navigation policy. The resulting suite combined four road types, three categories of non-ego vehicle motion, and road anomalies such as work zones, while remaining aligned with the original crash conditions.
Within a budget of only twenty scenarios, the generated tests exposed several system failures that standard combinatorial templates would have missed. The authors release the full pipeline as open-source code, enabling reproducible application to other ADS platforms and additional crash datasets.
Why it matters
This research is highly relevant for Dutch AI researchers and automotive tech companies focusing on smart mobility and AI safety. It provides an actionable, open-source methodology using LLMs to improve the safety and regulatory compliance of autonomous systems, aligning well with EU AI safety standards.
