From Monolithic to Modular: Segment-level Automatic Prompt Optimization
06:00 · August 13, 2026 · arXiv cs.AI RSS
Automatic Prompt Optimization (APO) often rewrites prompts monolithically, which can improve one behavior while degrading others. We present SAPO, a segment-level APO method that decomposes prompts into role, context, tasks, and output format, then applies targeted improvements based on top-5 and bottom-5 examples. The optimization loop uses one LLM with static meta-prompts and structured outputs for segmentation, weakness analysis, and candidate generation. We describe a train/validation protocol and a two-stage generation process: (1) segment-level diagnosis and recommendation extraction, (2) candidate synthesis constrained by weak/strong segment signals. Using the evaluation setup across SQuADv2, TweetEval, XSUM, CommonGen, and GSM8K on GPT-3.5-Turbo and GPT-4o-mini, SAPO achieves the best average score against Zero-shot and strong APO baselines including APE, OPRO, EvoPrompt, GEPA, and StraGO.
Summary
Automatic Prompt Optimization methods have typically treated prompts as single, indivisible strings. This monolithic approach can strengthen one aspect of model behavior while unintentionally weakening others, leading to unstable iteration and prompt drift. SAPO addresses the issue by decomposing a prompt into four explicit segments—role, context, tasks, and output format—then optimizing each segment on the basis of empirical evidence rather than global rewriting.
The method proceeds in two stages within an iterative loop. First, the current prompt is evaluated on a training set; the top-five and bottom-five examples are extracted to identify segments associated with correct versus incorrect outputs. These contrastive signals produce targeted recommendations that preserve segments linked to strong performance and flag segments requiring revision. In the second stage, an LLM generates a small set of candidate prompts under explicit constraints derived from the diagnosis. Candidates are scored on a held-out validation set; the best candidate is accepted only if it improves validation performance, with edit distance used as a tie-breaker to favor conservative changes.
Evaluations on SQuADv2, TweetEval, XSUM, CommonGen, and GSM8K using GPT-3.5-Turbo and GPT-4o-mini show that SAPO attains the highest average score among compared baselines, including APE, OPRO, EvoPrompt, GEPA, and StraGO. Relative gains over the strongest baseline reach 5.13 percent on GPT-3.5-Turbo and 7.25 percent on GPT-4o-mini. The approach therefore demonstrates that segment-level diagnosis and constrained candidate synthesis can improve both average task performance and robustness against destructive edits.
Why it matters
SAPO provides a highly actionable, structured approach to prompt engineering that Dutch AI researchers and enterprise teams can use to build more reliable and interpretable LLM applications. Its focus on modular, non-destructive prompt updates aligns with the EU's demand for robust, transparent, and controllable AI systems.
