ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science
06:00 · July 30, 2026 · arXiv cs.AI RSS
Clinical data-science agents must transform heterogeneous longitudinal records into auditable analyses, yet existing benchmarks largely isolate medical question answering, structured-table reasoning, or generic scientific repositories. We introduce CLINLENS, a benchmark of 200 executable tasks over five linked MIMIC resources spanning structured electronic health records, notes, electrocardiograms, chest radiographs, and echocardiograms. A 4 x 5 taxonomy crosses four patient-time scopes with five analysis capabilities. Program-first reverse synthesis pairs each bounded semi-raw package with an evaluator-private reference workflow and checks required artifacts, cohort and temporal semantics, and the final answer. On a fixed 126-task suite, the strongest of 24 standardized model-scaffold configurations achieves 56.3% scope-macro STRICTPASS despite 100% EXECSUCCESS. For reference, a separately configured coding agent solves 83 of 126 tasks, while five biomedical systems adapted to GPT-4o-mini reach at most 2.9% scope-macro STRICTPASS. These results expose a substantial gap between runnable submissions and correct clinical analyses.
Summary
ClinLens is a benchmark that evaluates coding agents on long-horizon clinical data-science tasks requiring longitudinal, multimodal records. It assembles 200 executable tasks from five linked MIMIC resources that retain structured electronic health records, clinical notes, electrocardiograms, chest radiographs, and echocardiograms without flattening them into pre-joined tables. Each task supplies a natural-language request together with a bounded package of semi-raw files; the agent must produce specified artifacts and a final answer while respecting patient identifiers, encounter hierarchies, and temporal windows.
Tasks are organized by a 4 × 5 taxonomy that crosses four patient-time scopes—whole-patient, admission, ICU-stay, and event or study—with five analysis families: profiling, association analysis, event-aligned change estimation, prediction, and phenotyping. Construction follows a program-first reverse-synthesis procedure that generates each task from an evaluator-private reference workflow, enabling automated checks on cohort definitions, temporal constraints, artifact schemas, and answer correctness.
On a fixed 126-task subset, the strongest of 24 standardized model-scaffold combinations reaches 56.3 percent scope-macro StrictPass while attaining 100 percent ExecSuccess. A separately tuned coding agent solves 83 of the 126 tasks, whereas five biomedical systems adapted to GPT-4o-mini achieve at most 2.9 percent StrictPass. The results therefore isolate a persistent gap between runnable code and analyses that satisfy the required clinical semantics, cohort semantics, and answer fidelity.
Why it matters
This research is highly relevant for Dutch AI researchers and clinical data scientists developing healthcare LLMs, as it provides a rigorous benchmark for evaluating the actual correctness of multimodal AI agents. This aligns with the Netherlands' strong emphasis on transparent, reliable, and ethically sound AI deployment in medical settings, especially under the EU AI Act.
