How Far Can Root Cause Analysis Go on Real-World Telemetry Data?
06:00 · July 16, 2026 · arXiv cs.AI RSS
Identifying root causes in production microservice failures requires reasoning over large-scale, multimodal telemetry spanning metrics, logs, and traces, a problem that has proved resistant to both classical and LLM-based approaches. The OpenRCA dataset exemplifies these challenges: it is large-scale, multimodal, and lacks detailed domain knowledge, and yields consistently low accuracy across all existing methods. We show that classical causal discovery methods and existing LLM-based multi-agent systems fail to reliably identify root causes on this benchmark, and present a Structured Multi-Agent RCA pipeline that substantially outperforms existing LLM-based and classical baselines, supporting both domain-knowledge and knowledge-free operating modes. To diagnose where failures originate, we introduce a reverse reasoning agent that, given the correct answer, identifies which signals in the extracted anomalies support it and determines whether Stage~1 had access to those signals, classifying each failure as Reasoning Gap (evidence present but unused) or Data Ambiguity (evidence genuinely absent). This analysis reveals that the required evidence is present in the vast majority of failures: the bottleneck is not data access but the agent's ability to reason over it correctly. We further introduce an automated rule mining pipeline that systematically extracts discrimination rules from reverse reasoning reports, reducing reliance on manual knowledge curation. Across all configurations, model reasoning capability and domain knowledge are the primary constraints: stronger models embed more domain expertise, and explicit knowledge injection partially compensates for this gap. Reasoning performance remains practically bounded even when evidence extraction is perfect: scaffold engineering and better data pipelines alone cannot close this gap; progress requires improvements at the model level.
Summary
Identifying root causes in large-scale microservice failures requires reasoning across heterogeneous telemetry that includes metrics, logs, and traces, often distributed over many files and recorded at incompatible time granularities. The OpenRCA benchmark captures these production realities: a 64 GB multimodal corpus spanning three domain-specific environments, with each failure instance limited to a 30-minute window that still produces roughly 2 GB of raw data once extracted. Existing classical causal-discovery algorithms and current LLM-based multi-agent systems both yield low accuracy on this corpus, the former because short windows and high dimensionality violate their statistical assumptions, the latter because aggressive context reduction and ad-hoc anomaly detection discard or mis-rank critical signals.
A Structured Multi-Agent RCA pipeline addresses these constraints by first condensing each window into a fixed, pre-computed set of anomalous rows using consistent per-KPI thresholds. Subsequent agents then perform staged enumeration, clustering, cross-modal verification with log and trace tools, and iterative reflection over that compact evidence. The pipeline supports two operating modes—one that supplies explicit component taxonomies and inference rules, and one that relies solely on telemetry-derived patterns—allowing isolation of the contribution of domain knowledge from raw reasoning ability.
To determine why failures persist, the authors introduce a reverse-reasoning agent that, given the ground-truth root cause, reconstructs the supporting evidence chain and classifies each error as either a Reasoning Gap (the necessary signals were present but not used) or Data Ambiguity (the signals were genuinely absent). Across the examined cases the required evidence resides in the extracted anomalies for the great majority of instances, indicating that the dominant bottleneck lies in model reasoning and available domain knowledge rather than in data access. An automated rule-mining pipeline further distills discrimination rules from these reverse-reasoning traces, reducing dependence on manual knowledge curation.
Empirical results show that the structured pipeline substantially outperforms both classical baselines and prior LLM-based systems on OpenRCA. Nevertheless, even perfect anomaly extraction leaves residual ambiguity that stronger base models and explicit domain knowledge can only partially mitigate. The findings therefore bound the practical gains obtainable from scaffold engineering alone and point to model-level improvements as the next necessary step.
Why it matters
This research is highly relevant for AI researchers and AIOps practitioners in the Netherlands managing complex cloud-native environments. It provides actionable insights into improving LLM-based multi-agent systems for automated diagnostics, a critical area for Dutch tech enterprises and infrastructure providers.
