SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
06:00 · August 7, 2026 · arXiv cs.AI RSS
LLM-agent ecosystems are rapidly growing around reusable skills: mixed-modality packages of metadata, natural-language instructions, code, tools, references, and operational workflows. As skills become marketplace artifacts, auditing their reuse is no longer the same problem as ordinary code clone detection. Existing detectors target single-modality source code or whole-package similarity, yet skill reuse evidence is distributed across authored text, implementation fragments, and operational structure. As a result, they can miss reuse that preserves only one part of a skill. We present SKILLTRACE, a multi-trace provenance auditing framework for LLM-agent skill reuse. SKILLTRACE extracts three provenance traces: Expression, Implementation, and Operational. It represents the Operational Trace as a Skill Operational Graph (SOG) that captures activation, procedure, and resource-flow structure. An LLM assists only the Operational-trace extraction, once at ingestion; at audit time SKILLTRACE compares cached traces deterministically, calibrates each trace against same-function strict negatives, and reports which trace supports a reuse decision. On SKILLTRACE-BENCH, with 820 transformed reuse positives over 100 marketplace anchors and 751 negative controls, SKILLTRACE achieves AUROC 0.938 and F1 0.898. A 36,446-skill wild audit further shows that trace-attributed evidence surfaces actionable reuse review queues beyond repository-level baselines.
Summary
SkillTrace addresses the growing challenge of auditing reuse within LLM-agent skill marketplaces, where skills function as mixed-modality packages containing natural-language instructions, executable code, tool interfaces, and operational workflows. Unlike conventional code-clone detectors that operate on single-modality source or whole-package similarity, the framework recognizes that reuse evidence can survive in only one of these layers after transformation. It therefore extracts three distinct provenance traces from each skill: an Expression trace for authored text, an Implementation trace for scripts and API patterns, and an Operational trace that records activation logic, task procedures, and resource flows.
The Operational trace is encoded as a Skill Operational Graph (SOG) whose three views capture the structural elements most likely to persist under rewriting. Extraction of the SOG relies on an LLM only once, at ingestion time; subsequent audits compare cached traces deterministically and calibrate each modality against same-function negative examples to reduce false attribution. The resulting report indicates which trace, if any, supports a reuse finding, allowing reviewers to distinguish inherited artifacts from independent implementations that merely solve the same task.
Evaluation on SkillTrace-Bench, comprising 820 transformed reuse cases derived from 100 marketplace anchors together with 751 strict negative controls, yields an AUROC of 0.938 and an F1 score of 0.898. A separate audit of 36,446 publicly available skills demonstrates that the trace-attributed evidence produces review queues containing reuse relations that repository-level fingerprinting methods overlook or rank too low. By separating provenance signals from functional similarity, SkillTrace supplies the visibility required for governance, deduplication, and remediation tasks in expanding LLM-agent ecosystems.
Why it matters
This research is highly relevant for Dutch AI researchers and enterprises focused on AI governance, IP protection, and compliance with EU transparency regulations. It provides a rigorous, actionable methodology for auditing LLM-agent ecosystems, which is crucial for maintaining ethical and transparent AI marketplaces.
