L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
06:00 · July 13, 2026 · arXiv cs.AI RSS
While multi-agent debate (MAD) frameworks have shown significant potential in general reasoning, their effectiveness in highly structured, knowledge-heavy legal domains remains under-explored. In this work, we introduce the Legal Multi-Agent Debate (L-MAD) framework to systematically evaluate different debate structures and aggregation methods within Legal Textual Entailment. By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8\%. Furthermore, analyzing how debate scales reveals a clear trade-off: increasing the agent population reduces inconsistency and improves accuracy, whereas extending discussion rounds induces a detrimental \textit{over-deliberation drift} where agents reinforce each other's mistakes. Ultimately, our findings outline the practical boundaries and safety margins of deploying collaborative multi-agent systems in high-stakes legal reasoning environments.
Summary
The Legal Multi-Agent Debate (L-MAD) framework evaluates how structured multi-agent interaction affects performance on Legal Textual Entailment, a binary classification task that requires models to determine whether given statutory articles support or refute a legal hypothesis. By assigning distinct expert personas to several LLM agents and allowing them to exchange arguments over discrete rounds, L-MAD improves accuracy over strong single-agent baselines by up to 8 percent. The improvement stems from the injection of diverse inductive biases rather than from external tool use or parameter updates.
Two families of decision protocols were compared. Consensus protocols require agents to continue debating until a chosen agreement threshold—majority, supermajority, or unanimity—is reached. Voting protocols instead fix the number of discussion rounds and then aggregate independent final votes. Results indicate that the preferred protocol depends on model scale: forcing consensus benefits larger models (30 B parameters and above) by encouraging constructive refinement, while independent voting shields smaller models (around 8 B parameters) from premature or superficial agreement.
Scaling experiments reveal a clear asymmetry. Increasing the number of participating agents reduces output inconsistency and raises accuracy. Extending the number of debate rounds, however, produces over-deliberation drift: agents begin to echo and reinforce one another’s errors, eventually degrading performance. The study therefore supplies concrete operational boundaries—agent population size, discussion depth, and aggregation method—for the safe deployment of collaborative LLM systems in high-stakes legal reasoning.
Why it matters
This research is highly relevant for Dutch AI researchers and LegalTech developers building multi-agent systems for high-stakes, regulatory, or compliance domains. It provides actionable insights into preventing hallucination and over-deliberation, aligning with the Netherlands' strong focus on transparent, ethical, and reliable AI.
