Securing the frontier: How JetBrains evaluates and deploys Claude Fable 5
02:00 · August 13, 2026 · Claude Blog
Summary
JetBrains, the company behind IntelliJ IDEA, PyCharm and the Kotlin language, serves more than 12.5 million developers and most of the Fortune Global 100. In an interview with Anthropic, CTO Vladislav Tankov described how the firm now evaluates frontier models for its coding tools and when it chooses to deploy Claude Fable 5 in production workflows.
Evaluation relies on large private-repository test suites that include JetBrains’ own monorepo. The team maintains separate leaderboards for output quality, cost per completed task and speed. Claude Fable 5 recorded a 44.3 % Python pass rate on these internal sets, compared with 28.2 % for Opus 4.8, and solved 18 tasks the earlier model missed while failing only two. It also required roughly 22 % fewer steps to reach a working solution and avoided unproductive attempts to import external resources that do not exist inside JetBrains’ environment.
The model is reserved for tasks that demand sustained reasoning or exploratory agentic coding. Examples include implementing a rich-text editor component that had resisted prior attempts and running long-horizon experiments in which an agent receives specifications, generates its own follow-up requirements, and rewrites an application across runtimes or languages in a largely black-box setting. Opus remains the default for routine work where predictability matters more than peak capability.
On safety and data handling, JetBrains relies on Anthropic’s red-teaming rather than modifying the model itself. The company instead builds surrounding infrastructure and review processes. It uses the model for internal white-box security testing while preparing for external actors who may employ similar systems against JetBrains products. Tankov noted a preference for zero data retention but accepts limited review of the most serious flagged cases as a practical trade-off for access to frontier performance, especially when serving regulated enterprise customers.
Why it matters
Provides concrete evaluation methods, metrics, and safety trade-offs that Dutch product teams can replicate for model selection and compliant deployment under EU rules.
