RIFT-Bench: Dynamic Red-teaming For Agentic AI Systems
06:00 · June 24, 2026 · arXiv cs.AI RSS
Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities. Existing security evaluations are often tied to specific implementations or domains, limiting unified comparison across heterogeneous systems. To address this gap, we introduce RIFT-Bench, a graph representation-driven methodology for dynamic red-teaming that enables unified evaluations across diverse agentic architectures. Building on a novel hierarchical representation, RIFT-Bench operates in two automated phases: Discovery, which extracts system structure, and Scanning, which deploys adaptive adversarial attacks and produces a comprehensive evaluation report. It evaluates the examined system itself, leveraging a broad set of dynamically adaptable adversarial probes across diverse attack vectors and objectives. We demonstrate the effectiveness of the proposed evaluation pipeline across 45 agentic systems spanning a diverse range of implementations, showing that the approach generalizes effectively to heterogeneous agentic architectures. Beyond systems and attacks, RIFT-Bench also supports direct evaluation of mitigation strategies. These key capabilities make RIFT-Bench a scalable foundation for security evaluation of agentic AI systems.
Summary
Agentic AI systems built on large language models increasingly operate as autonomous decision-makers that combine tool use, memory, and multi-step coordination. These capabilities extend beyond conventional LLM vulnerabilities such as prompt injection or jailbreaks, introducing system-level risks including goal hijacking, tool misuse, privilege escalation, and memory poisoning. Existing security evaluations remain tied to particular frameworks or simulated environments, which prevents direct comparison across heterogeneous implementations and limits reuse of attack patterns.
RIFT-Bench addresses this limitation through a graph-based methodology centered on NodeSpec, a hierarchical representation that maps agentic components and their interactions to concrete code blocks. The framework runs in two automated phases. Discovery extracts the NodeSpec from a target codebase and integrates configurable tool emulation to support safe execution. Scanning then instantiates structure-aware adversarial probes, executes them against the live system, traces outcomes, and generates a vulnerability report covering multiple attack vectors and objectives.
The approach was applied to a benchmark of 45 agentic systems drawn from varied frameworks and architectures. More than 100 adversarial probes were instantiated into over 10,000 distinct tests, demonstrating that the method generalizes across implementation differences. In addition to assessing systems and attacks, RIFT-Bench supports direct measurement of mitigation strategies under realistic conditions, establishing a reusable foundation for standardized security evaluation of agentic AI.
Why it matters
This research is highly relevant for Dutch AI practitioners and researchers focusing on AI safety and compliance with the EU AI Act. RIFT-Bench provides a scalable, unified framework for red-teaming autonomous LLM agents, which is critical for deploying secure and trustworthy AI systems in enterprise environments.
