Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing
06:00 · July 3, 2026 · arXiv cs.AI RSS
Finetuning a language model on documents that are explicitly annotated as fictional results in a model that still actually believes the documents' core claims, an effect known as Negation Neglect. In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time. To address this, we introduce Goggles, a learned module that intervenes on the finetuning gradient rather than the data. During supervised finetuning, a Goggles module edits the gradients an LLM LoRA receives, imparting a chosen epistemic frame (the stance the model takes toward the nature of what it reads) to whatever the documents teach. A Goggles instance is trained once for a given base model, frame, and LoRA configuration, then applied frozen to documents it was never trained on. Trained through Goggles on those same documents, now carrying no fictional annotation, the model flags the content as fictional roughly 91% of the time, while preserving capability (GPQA and TruthfulQA match or exceed baseline). The same architecture supports other frames: a Goggles instance can be trained to treat documents as "part of an AI safety evaluation by Redwood Research" rather than simply as fiction. The imparted frame persists under continued finetuning that pushes back toward the claim, where prior interventions revert. Goggles suggests a path toward training language models on known-misaligned data without absorbing the behaviors that data demonstrates.
Summary
Finetuning language models on explicitly labeled fictional documents still leads them to internalize the core claims those documents contain, a failure mode termed Negation Neglect. In the reported evaluations, models trained on documents carrying clear prefix and suffix disclaimers correctly treat the embedded assertions as non-factual only about nine percent of the time. The effect appears tied to the inductive bias of supervised fine-tuning: the cross-entropy objective favors representing the substantive content as true, even when an epistemic frame—i.e., an explicit stance on the factual status of the material—is supplied in the textual channel.
Goggles addresses the limitation by operating directly on gradients rather than on tokens. For a chosen base model and LoRA configuration, a small set of editor networks is trained once to read activations, gradients, and LoRA outputs at selected layers and to emit a residual that is added to the gradients flowing into the adapter. The resulting module is then frozen and reused on new documents. When the same fictional corpus is presented without any textual annotation, training through the Goggles editor causes the model to flag the claims as fictional roughly ninety-one percent of the time. Performance on GPQA and TruthfulQA remains at or above the level achieved by standard LoRA fine-tuning without the editor.
The same architecture supports other epistemic frames. One variant conditions the model to treat incoming documents as material generated for an AI-safety evaluation conducted by Redwood Research. The induced frame also proves more stable than text-based interventions: it survives subsequent fine-tuning steps that attempt to push the model back toward accepting the planted claims. Because a single Goggles instance generalizes across documents it never encountered during its own training, the approach offers a practical route for incorporating known-misaligned or otherwise undesirable data into supervised fine-tuning runs without absorbing the behaviors those data exhibit.
Why it matters
Novel gradient-editing technique for epistemic control directly supports ethical and transparent AI goals emphasized in Dutch and EU policy. Researchers can reproduce and extend the method using the provided code and datasets. The approach offers practical value for Dutch labs and SMEs working on safe fine-tuning pipelines.
