How Can AI Find My Model? A Model-Finding Experimental Study Considering Data Formats, Embeddings, and Retrieval Strategies
06:00 · July 1, 2026 · arXiv cs.AI RSS
Discovering simulation models for reuse remains a fundamental challenge in Modeling and Simulation (M&S). When many models coexist, identifying those that align with a given modeling intent remains difficult. Recent advances in Artificial Intelligence (AI), particularly retrieval-based approaches, offer a promising pathway to operate at this semantic layer. In this paper, we present an experimental study investigating the impact of data representation, transformer-based embedding models, and retrieval strategies on the discovery of simulation models using natural language queries. We evaluated performance across multiple query types using standard information retrieval metrics, including recall@5 and nDCG@5. Results show that data representation matters, open-source embedding models can achieve high performance, and reranking methods are important, especially as query complexity increases. This work provides a baseline for AI-driven model discovery and discusses its role in advancing toward AI-driven composability and interoperability.
Summary
Discovering simulation models suitable for reuse has long been a core difficulty in Modeling and Simulation. When repositories contain many models, locating those that match a particular modeling intent through natural language remains nontrivial. The paper examines whether retrieval-based AI techniques can address this gap by operating directly on semantic representations rather than keyword or metadata matching.
The authors conducted a controlled experiment that systematically varied three elements: the way model information is represented as input data, the choice of transformer-based embedding models, and the retrieval pipelines applied after initial ranking. Performance was measured on several categories of natural-language queries with standard information-retrieval metrics, notably recall@5 and nDCG@5. The study therefore isolates the contribution of each design choice while reflecting realistic query complexity.
Results indicate that the format in which model descriptions are stored materially affects retrieval quality. Several openly available embedding models reached competitive accuracy without proprietary components, and the addition of reranking stages proved especially valuable once queries grew more elaborate. The work supplies an empirical baseline for AI-supported model search and outlines how such retrieval capabilities could eventually support automated model composition and interoperability.
Why it matters
The research provides actionable insights into semantic search and model discovery, which is highly relevant for Dutch research institutions and enterprises utilizing digital twins and complex simulations. Its validation of open-source embedding models also aligns with the European push for transparent, cost-effective, and sovereign AI infrastructure.
