SCOPE and SCION: A Benchmark and an Auditable Reference Pipeline for Schema Induction and Fusion from Text
06:00 · July 27, 2026 · arXiv cs.AI RSS
Schema graphs are an upstream bottleneck of schema-grounded information extraction and knowledge graph construction, yet most extraction systems assume the schema is already available. We introduce SCOPE (Schema Construction and Ontology-induction Pipeline Evaluation), a train-text-only benchmark for corpus-to-schema induction and optional schema fusion from raw text, built from 24 public information extraction sources (15 RE and 9 EE) normalized into evaluation-only gold schema graphs; its core event-extraction target covers event types and within-event argument roles, with inter-event links reported separately. We present SCION (Schema Construction and Induction with Ontology Normalization), an auditable reference pipeline rather than a new extraction architecture; it constructs candidate spaces from train text and restricts naming, merging, filtering, validation, and conservative fusion to candidate-linked evidence under strict JSON contracts. On the SCOPE core suite, SCION-lite attains the highest F1 among released source-schema references, Text2Onto-style, LLM-only, and matched extract-then-aggregate baselines under Literal, Fuzzy, Continuous, and Graph schema-graph metrics, while the compact open-model SCION-RL variant reduces reliance on proprietary LLM schema engineers. These results are reported against normalized typed-edge targets rather than as claims that induced schemas surpass human ontology design; the release includes evidence-linked outputs, parse/fallback logs, candidate retention/merging logs, run manifests, code, and benchmark packages at https://github.com/wandugu/paper_scion.
Summary
Schema graphs serve as a foundational but often overlooked prerequisite for schema-grounded information extraction and knowledge-graph construction. Most existing systems presuppose that an appropriate schema already exists, leaving the upstream task of inducing one from raw text comparatively underexplored. To address this gap, the authors present SCOPE, a train-text-only benchmark constructed from 24 publicly available information-extraction datasets—15 relation-extraction and 9 event-extraction sources—whose schemas have been normalized into evaluation-only gold graphs. The benchmark’s primary focus is event extraction, specifically event types and the argument roles that appear within individual events; inter-event relations are tracked separately.
Alongside the benchmark they release SCION, an auditable reference pipeline rather than a novel extraction architecture. SCION first assembles candidate schema elements directly from the training text, then applies naming, merging, filtering, validation, and conservative fusion steps exclusively to evidence-linked candidates. All operations are constrained by strict JSON contracts that record every decision, enabling inspection of parse logs, fallback behavior, candidate retention, and merging history. On the SCOPE core suite, the SCION-lite configuration records the highest F1 scores among published baselines—including source-schema references, Text2Onto-style pipelines, LLM-only approaches, and extract-then-aggregate methods—across Literal, Fuzzy, Continuous, and Graph schema-graph metrics.
A compact open-model variant, SCION-RL, achieves comparable results while eliminating dependence on proprietary large-language-model schema engineers. The authors emphasize that performance figures are measured against the normalized typed-edge targets supplied by SCOPE and do not constitute claims of superiority over human-designed ontologies. The full release comprises evidence-linked outputs, run manifests, benchmark packages, and source code, supporting reproducible experimentation in schema induction for knowledge-graph construction.
Why it matters
This research is highly relevant for Dutch AI researchers and enterprises developing knowledge graphs and information extraction systems. Its emphasis on an auditable pipeline and open models aligns perfectly with the Netherlands' strategic focus on transparent, ethical, and sovereign AI solutions.
