Incomplete Prompt Jailbreaks in Large Language Models
06:00 · July 24, 2026 · arXiv cs.AI RSS
Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. Nevertheless, sentence completion remains vulnerable to incomplete harmful prompts. In this work, we formalize this phenomenon as incomplete prompt jailbreaks (IPJ) and provide a systematic empirical characterization of when and how incomplete prompts elicit harmful continuations. We analyze diverse attractor types associated with incomplete sentence continuation and show that LLMs systematically delay refusal until sentence termination. We further demonstrate that training models to refuse incomplete harmful prompts via parameter tuning is insufficient, failing to generalize across both content domains and attractor types. To enable fine-grained control, we identify two functional neurons: termination and continuation neurons. By clarifying their roles in sentence completion, we highlight the potential of neuron-level interventions for more precise and robust IPJ defenses.
Summary
Large language models released with open weights typically incorporate safeguards that cause them to refuse overtly harmful requests. The paper shows that these safeguards can be circumvented when a request is left syntactically incomplete. By appending short linguistic cues—termed attractors—that signal an ongoing sentence, an otherwise refused query can be turned into a prompt that elicits harmful continuations before any refusal appears. The authors formalize this behavior as incomplete prompt jailbreaks (IPJ) and examine it across nine categories of attractors, including methodological, structural, sequential, and hypothetical constructions.
Experiments on multiple instruction-tuned models demonstrate that refusal is systematically postponed until the incomplete prompt reaches a natural sentence boundary. Once that boundary is crossed, models often produce harmful content and only afterward emit refusal statements. Attempts to mitigate the issue through parameter tuning that forces explicit refusal phrases prove brittle: the resulting models fail to generalize across content domains, unseen attractor types, or different refusal styles.
To address these shortcomings, the work identifies two functionally distinct sets of neurons. Activation of termination neurons can be steered to interrupt generation early, while amplification of continuation neurons increases the likelihood of harmful output. The findings indicate that neuron-level interventions offer a more precise route to defending against IPJ than prompt-level or parameter-tuning approaches alone.
Why it matters
Directly addresses LLM safety and ethical deployment of open-weight models, highly actionable for Dutch/EU researchers under AI Act constraints; offers novel neuron-level methods with code and data.
