FinSkillBench: Evaluating AI Agents and Domain Skills for Investment Management
06:00 · August 20, 2026 · arXiv cs.AI RSS
Investment management is a high-stakes domain in which agentic AI systems must do more than generate plausible text. They must retrieve point-in-time data, assemble correct computational inputs, invoke specialized methods, and produce auditable structured outputs. We introduce FinSkillBench, an evaluation suite designed to measure whether language model agents can effectively use financial domain skills to solve investment management tasks. The benchmark spans three domains, portfolio construction, risk management, and fundamental analysis, and includes 12 subtasks with 2,603 task episodes. Each episode provides point-in-time inputs, hidden ground truth, and a task-specific verifier.We compare three conditions: no skill, curated skill packages consisting of procedural documents and executable components, and self-generated skills in which the agent writes and reuses its own procedures within an episode. Across 9 models and a large-scale evaluation, curated skills consistently improve performance, raising mean scores from 0.366 to 0.528, with the largest gains in portfolio construction and risk management. In contrast, self-generated skills provide little benefit despite higher computational cost. An independent evaluation using a separate agent framework (Hermes Agent, 8 models, 5,280 episodes total) reproduces the directional pattern across all three domains, with the magnitude of skill effects varying by subtask and harness. These results showthat in investment management agents, access to reliable procedural skills can be as important as model choice, while naive self-generation of skills is often ineffective. We release the benchmark, evaluation tools, curated skill packages, and full trajectories to support further research.
Summary
FinSkillBench provides a structured evaluation of language-model agents on investment-management workflows that demand point-in-time data retrieval, precise numerical inputs, domain-specific procedures, and verifiable structured outputs. The benchmark covers three domains—portfolio construction, risk management, and fundamental analysis—through twelve subtasks and 2,603 task episodes. Each episode supplies an explicit as_of_date, hidden ground truth, and an automated verifier that scores numeric accuracy, constraint satisfaction, or accounting-derived metrics.
The study isolates the contribution of procedural skills by comparing three conditions across nine models. In the baseline, agents receive only the task description. In the curated condition, they can load human-authored skill documents and executable components. In the self-generated condition, agents must first write and then reuse their own procedures. Curated skills raised mean performance from 0.366 to 0.528, with the largest improvements appearing in portfolio-construction and risk-management subtasks. Self-generated skills yielded negligible gains while incurring higher computational cost.
An independent replication using the Hermes Agent framework, covering eight models and 5,280 episodes, reproduced the same directional pattern across all three domains, although the size of the skill effect varied by subtask and evaluation harness. The authors release the full benchmark, evaluation harnesses, curated skill packages, and complete agent trajectories to enable further controlled study of how reliable procedural knowledge interacts with model choice in high-stakes financial settings.
Why it matters
Provides actionable, reproducible evaluation methods and skill packages that Dutch AI researchers and fintech teams can directly apply or extend for regulated financial workflows under EU standards.
