FedPref: Federated Preference Learning for Structured Radiology Report Extraction
06:00 · August 19, 2026 · arXiv cs.AI RSS
Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema. Learning this extraction requires labels that are unevenly distributed across institutions: smaller hospitals have less local evidence, and pooling data may be infeasible. We introduce FedPref: frozen public language models propose alternative JSON extractions, local annotations rank them, and sites collaboratively train compact Qwen3-8B adapters while sharing only model updates. A heterogeneous teacher pool provides cross-model contrast when repeated single-model samples collapse. On development data from six simulated hospitals with unequal data volume and disease prevalence, FedPref improves client-mean F1 by 2.49 points and worst-site F1 by 9.10 points compared with training each site in isolation, with the largest gains at the sites holding the least data. Central training on the pooled preference-pair union is 2.66 points higher on client-mean F1. On a locked, 400-report manually validated gold test set, FedPref reaches 68.68 F1 and pooled training 71.67, preserving that same ordering. FedPref thus lets institutions with unequal, unpooled data benefit from collaboration without ever sharing reports or annotations.
Summary
Radiology reports encode clinical findings together with their anatomical locations in free text, yet many downstream applications require these relations in a fixed JSON schema. Because smaller hospitals typically hold fewer cases and different disease prevalences, local training data are unevenly distributed, while pooling raw reports across institutions is often prohibited. FedPref addresses this constraint by letting each site keep its reports and annotations private while still benefiting from collaborative model improvement.
The method begins with a pool of four frozen teacher models that generate alternative structured extractions for every local report. These candidates are projected into a common schema and ranked by the site’s existing structured annotation, producing chosen and rejected pairs. A compact Qwen3-8B target model is first aligned to the chosen responses through one epoch of supervised fine-tuning, after which Direct Preference Optimization refines its preference ordering. Only the resulting low-rank adapter updates are exchanged; these updates are aggregated with pair-count-weighted federated averaging to obtain a shared global adapter.
Experiments simulate six hospitals that differ in both data volume and disease prevalence, using development data derived from MIMIC-CXR. Relative to training each site in isolation, FedPref raises mean client F1 by 2.49 points and worst-site F1 by 9.10 points, with the largest gains observed at the two smallest sites. Centralized training on the pooled preference pairs yields an additional 2.66 points on the client-mean metric. On a held-out, manually validated test set of 400 reports, FedPref reaches 68.68 F1 while pooled training reaches 71.67 F1, confirming that the federated ordering is preserved. The framework therefore supplies a practical route for institutions that cannot share data yet still need to improve structured extraction performance.
Why it matters
Strong actionability for Dutch/EU hospitals under GDPR constraints; directly addresses privacy-preserving collaboration on medical data with unequal distributions, high technical depth, novelty in combining federated learning with preference optimization, and full reproducibility via GitHub.
