Self-refine: a complete guide

Language models, like us, rarely nail it on the first try. Self-refine fixes that with one move: let the same model read its own draft, write specific feedback, and rewrite, looping until the output is good. No fine-tuning, no reward model, no second system. Madaan et al. (2023) report a roughly 20% average absolute improvement across seven tasks on GPT-3.5, ChatGPT, and GPT-4.

See it work

Ask a model to coin an acronym, and the first pass is often clumsy. Self-refine lets it grade its own work and try again.

Prompt: Generate an acronym for "Radio Detection And Ranging".

Draft (round 0): RDR
  - Pronounceable? No, it's just initials.
  - Memorable? Weak.
  - Relevant? It maps to the words but reads like noise.

Self-feedback: "RDR" isn't pronounceable and doesn't evoke the domain.
A good acronym should be a real word tied to detection or signals.

Refined (round 1): RADAR
  - Pronounceable? Yes.
  - Memorable? Very, it's already a word.
  - Relevant? Strongly evokes detection and ranging.

Same model, same prompt, three roles. The only new ingredient is a feedback step wedged between draft and rewrite. That's the whole technique.

The mental model

Think of a writer with no editor on call. They draft a paragraph, lean back, ask "what's weak here?", scribble notes in the margin, then rewrite from those notes. They repeat until nothing obvious is left to fix.

Self-refine turns a single model into draft-writer, critic, and reviser, all in one conversation, so quality comes from iteration instead of getting it perfect in one shot.

The trick is that spotting a flaw is easier than avoiding it up front. Critiquing "this code allocates inside the loop" is a smaller ask than emitting optimal code on the first pass. Self-refine cashes in that gap.

How it works

Three prompts, run in a loop with the same underlying model.

1. Generate. Prompt the model for an initial answer using a few in-context examples. This is your baseline draft, no different from ordinary prompting.
2. Feedback. Feed the prompt and that draft back to the same model and ask it to critique its own work. The feedback must be specific and actionable ("line 4 recomputes the sum each pass") rather than vague ("make it better"). This is the load-bearing step.
3. Refine. Hand the model the draft plus its feedback and ask for an improved version. The history of past drafts and feedback carries forward, so the model doesn't repeat mistakes it already flagged.

Steps 2 and 3 repeat. Each round appends the previous answer and feedback to the context, then stops when the feedback says the output is good enough or a max iteration count is hit (the paper caps at four).

Why it works

The gains aren't uniform, and the ranking below explains where the lift comes from.

Where it shines

Self-refine works best on open-ended generation where you can articulate what "better" looks like. Madaan et al. (2023) measured seven tasks across GPT-3.5, ChatGPT, and GPT-4. The standout results:

• Constrained generation (cram many required concepts into a sentence): GPT-4 jumped from 15.0 to 45.0, a 30-point gain.
• Acronym generation: GPT-4 rose from 30.4 to 56.0.
• Sentiment reversal (flip a review's polarity): improved by at least 21.6 points in preference.
• Code readability improvement: gained at least 13.9 points.
• Code optimization (PIE, make programs faster): up 8.7 points.
• Dialogue response generation: human raters preferred the refined replies by a wide margin.

The weak spot is telling. On math reasoning (GSM8K), GPT-4 moved only 92.9 to 93.1, because the feedback step couldn't reliably spot arithmetic errors and answered "everything looks good" on most instances. No reliable critique, no refinement.

When to use it (and when not)

Reach for it when:

• The task is open-ended and you can describe what a better answer looks like.
• The model can judge its own output (style, constraints, readability, structure).
• Quality matters more than latency or token cost.

Skip it when:

• Errors are hard for the model to detect, like subtle arithmetic, where self-feedback gives false all-clears.
• You need a fast, single-shot answer; each loop multiplies calls and latency.
• The base model is weak; it'll produce shallow critiques and stall.

Model fit. Self-refine needs a model strong enough to both critique and revise. It pairs naturally with frontier models (GPT-4-class and up); on small or instruction-weak models the feedback is too shallow to help, and you're better off with few-shot prompting or fine-tuning.

Variants and alternatives:

The refinement loop in code

The canonical algorithm is short. One model object plays all three roles.

def self_refine(task, max_iters=4):
    draft = model.generate(task)            # 1. initial draft
    for _ in range(max_iters):
        feedback = model.feedback(task, draft)   # 2. critique own work
        if feedback.is_satisfactory:             # stop condition
            break
        draft = model.refine(task, draft, feedback)  # 3. rewrite
    return draft

The feedback prompt does the heavy lifting. It should force specific, localized critique and include an explicit "stop" escape hatch so the loop can terminate.

You wrote the draft below for the task. Critique it.
For each issue, name the exact location and a concrete fix.
If the draft cannot be meaningfully improved, reply exactly: GOOD.

Task: {task}
Draft: {draft}
Feedback:

Configuration that matters

Implementation workflow

1. Write the three prompts: generate, feedback, refine. Each gets its own few-shot examples.
2. Make the feedback examples ruthlessly specific; this is where quality is won or lost.
3. Add a clear stop sentinel and a hard iteration cap so the loop always terminates.
4. Carry the full draft-plus-feedback history into each refine call.
5. Measure against a single-pass baseline before trusting the loop.

Do and don't:

• Do demand located, actionable feedback ("line 4, recompute outside the loop").
• Do cap iterations and give the model a way to say "stop".
• Don't use vague feedback prompts; "improve this" yields cosmetic edits.
• Don't apply it to tasks where the model can't tell right from wrong.

Debugging a stalled loop

• Feedback always says "good" on round 0 to a flawed draft: the model can't detect this error type (common on math). Self-refine won't help; switch techniques.
• Output oscillates or degrades across rounds: the refine step is over-editing. Tighten the refine prompt to "apply only the listed fixes".
• Loop never stops: your stop sentinel isn't being honored. Make it exact-match and parse strictly.
• Cosmetic-only changes: feedback is too generic. Add few-shot examples of sharp, located critiques.

Proving it beats the baseline

Run both arms on a held-out set and compare. For preference-style tasks, use human raters or an automatic metric; for code, run the actual benchmark.

def evaluate(items):
    base = [model.generate(x) for x in items]
    refined = [self_refine(x) for x in items]
    return {
        "base_score": score(base, items),
        "refine_score": score(refined, items),
    }

If the refined arm doesn't clear the baseline by a margin worth the extra calls, the task probably isn't a fit.

Limitations

• The model must be able to detect its own errors. When it can't, as with subtle arithmetic, feedback gives false all-clears and refinement does nothing.
• It needs a strong base model. Critique and revision both demand capability; weak models stall or drift.
• Cost and latency multiply with each iteration, since every round is extra calls over growing context.
• Evaluation was English-only. The original study tested English datasets; behavior elsewhere is unverified.
• It can be misused. The same self-improvement loop could be turned toward polishing harmful or deceptive content.

How it fits with other techniques

Self-refine sits in the self-correction family, defined by having no external signal: the same model generates, critiques, and revises. Useful contrasts:

It composes well, too. Use chain-of-thought inside the generate step, then self-refine the result. Or run self-refine over a retrieval-augmented draft so the feedback step can flag unsupported claims. The loop is technique-agnostic; it just needs a draft to chew on and a way to judge it.

Summary

• Self-refine loops one model through three roles, generate, self-feedback, refine, with no external signal or training.
• It works because spotting a flaw is easier than avoiding it, so critique-then-rewrite beats one-shot.
• Madaan et al. (2023) report roughly 20% average absolute improvement across seven tasks on GPT-3.5, ChatGPT, and GPT-4.
• Biggest wins are on checkable open-ended tasks (constrained generation 15.0 to 45.0, acronym 30.4 to 56.0 on GPT-4); the weakest is math, where the model can't reliably catch its own errors.
• Feedback specificity is everything; vague critique yields cosmetic edits.
• It costs extra calls and latency per round, needs a strong base model, and should be paired with an external check for factual or safety-critical work.