Self-verification: a complete guide

Chain-of-thought reasoning is brittle. One slip in a multi-step solution and the whole answer falls over, but the model still hands it to you with total confidence. Self-verification flips the problem around: generate a few candidate answers, then check each one backward — plug the answer back in, hide an original number, and see if the model can reconstruct it. The answer that survives the most checks wins. On GSM8K this lifts code-davinci-002 from 60.81% to 65.14% (+4.33), with similar gains across arithmetic and commonsense tasks (Weng et al., EMNLP 2023 Findings).

See it work

Take a grade-school word problem. Sample the model a few times with chain-of-thought and you get a spread of answers — that's normal, decoding is stochastic.

Question: Jackie has 10 apples. Adam has 8 apples. How many more
apples does Jackie have than Adam?

Forward reasoning (sampled 3 times):
  Chain A → "10 - 8 = 2"        → answer: 2
  Chain B → "10 + 8 = 18"       → answer: 18   (misread "more than" as "total")
  Chain C → "10 - 8 = 2"        → answer: 2

Two chains say 2, one says 18. A majority vote already leans toward 2 — but self-verification gives you a reason, not just a count. Take each candidate, fold it into the problem as a new fact, mask an original number, and ask the model to recover it:

Verify candidate "2":
  "Jackie has X apples. Adam has 8. Jackie has 2 more than Adam.
   What is X?"   → model predicts X = 10  ✓ matches original (10)

Verify candidate "18":
  "Jackie has X apples. Adam has 8. Jackie has 18 more than Adam.
   What is X?"   → model predicts X = 26  ✗ original was 10

Candidate 2 reconstructs the hidden condition; candidate 18 doesn't. So you keep 2 — and you can show your work for why.

The mental model

Think of how you check your own arithmetic. You solved 47 - 19 = 28, so you sanity-check by adding back: 28 + 19 should give 47. If it doesn't, you made a mistake. Working the problem in reverse uses different muscles than working it forward, so it catches errors the forward pass missed.

Self-verification treats a candidate answer as a hypothesis, then asks: if this were true, would the original problem still hold together? The answer that best rebuilds its own premises is the one to trust.

How it works

The method has two phases — forward reasoning to propose answers, then backward verification to score them. Nothing is fine-tuned; it's all prompting.

1. Forward reasoning. Prompt the model with few-shot CoT and sample it K times (the paper uses K=5). You get several reasoning chains and their answers — some agree, some don't.
2. Rewrite as conditions. For each candidate, turn the question plus that answer into a set of declarative statements ("Jackie has 10 apples", "Jackie has 2 more than Adam").
3. Mask and verify backward. Replace one of the original conditions with a variable X, then ask the model to solve for X using the candidate answer as a given. Repeat across each maskable condition, P times each (the paper uses P=10) to smooth out sampling noise.
4. Score and select. The verification score is how often the model's re-predicted X matches the true masked value. The candidate with the highest score is your final answer.

Why it works

The gain comes from a few compounding effects, roughly in order of impact.

Where it shines

The technique was validated on three task families, all with the 175B Instruct-GPT (code-davinci-002) base model and CoT as the baseline.

Arithmetic is the sweet spot, because numeric conditions are easy to mask and check:

Commonsense and logical reasoning also improve, though by less, because there's no clean number to mask — Date Understanding went from 65.43% to 66.57% (+1.14) and CommonsenseQA from 77.42% to 77.83% (+0.41).

The method also stacks on top of self-consistency: the paper reports a combined "SC + self-verification" setting that improves over self-consistency alone, since verification re-ranks the sampled chains rather than just counting them.

When to use it (and when not)

Reach for it when the task has a checkable answer (math, structured logic, factual QA), you're already using CoT, and a wrong answer is costly enough to justify extra calls. It's strongest exactly where error accumulation bites — long multi-step arithmetic.

Skip it when the base model is weak, the output is open-ended (essays, summaries, creative text) with no premise to reconstruct, or you're latency-bound.

Model fit: this needs a model strong enough to verify, not just generate. On the weaker GPT-3 code-davinci-001, GSM8K barely moved (13.84% to 13.92%, +0.08) — verification is only as good as the verifier. Reach for a capable, instruction-tuned model.

Escalation path: start with plain CoT, add self-consistency when single-shot is too noisy, then layer self-verification on top when you need both accuracy and an auditable reason for the chosen answer.

Structure and components

A self-verification setup has four moving parts:

• A CoT forward prompt — few-shot exemplars that elicit step-by-step reasoning. This produces the candidates.
• A sampler — temperature-based decoding to get K diverse chains rather than one greedy answer.
• A rewrite step — turns "question + candidate answer" into declarative conditions, masking one with a variable.
• A verification prompt — either CMV (solve for the masked number) or TFV (judge True/False), run P times and tallied into a score.

The two verification styles share a template skeleton:

# Condition Mask Verification (arithmetic)
Given: Jackie has X apples. Adam has 8 apples.
       Jackie has 2 more apples than Adam.
Question: What is the value of X?

# True-False Item Verification (commonsense / logic)
Given: [original conditions] and the conclusion [candidate answer],
Are all of these statements mutually consistent? Answer True or False.

The core algorithm

The selection logic is small — most of the work is in the prompts. In pseudocode:

def self_verify(question, model, K=5, P=10):
    # Phase 1: forward reasoning — sample K candidate answers
    candidates = [model.cot(question, temperature=0.7) for _ in range(K)]

    best, best_score = None, -1
    for cand in set(candidates):                 # dedupe identical answers
        score = 0
        for masked_q, true_value in mask_conditions(question, cand):
            # Phase 2: backward verification, P samples per masked slot
            preds = [model.solve(masked_q, temperature=0.7) for _ in range(P)]
            score += sum(1 for p in preds if p == true_value)
        if score > best_score:
            best, best_score = cand, score
    return best, best_score

mask_conditions rewrites the problem with each original numeric condition hidden in turn (CMV); for commonsense tasks you swap the inner loop for a True/False tally (TFV). The verification score is just a count of matches — interpretable and easy to log.

Configuration

Implementation workflow

1. Get a solid few-shot CoT prompt working first — self-verification re-ranks its output, it can't fix a bad forward prompt.
2. Sample K chains and extract the final answer from each.
3. Write the rewrite function: question + answer to declarative conditions, with one slot maskable.
4. Pick CMV or TFV based on whether the answer hinges on a number or a judgment.
5. Run verification P times per candidate, tally matches, select the top score.
6. Log the per-candidate scores — they're your audit trail when an answer looks wrong.

Do: dedupe identical candidates before verifying (saves calls), mask one condition at a time, and keep the original answer as a tiebreaker when scores tie.

Don't: verify with temperature 0 (you lose the noise-averaging), mask the answer itself instead of a premise, or apply CMV to tasks with no numeric condition to recover.

Debugging

Testing and proving it

To prove self-verification beats your baseline, hold the forward prompt fixed and compare answer-selection strategies on a labeled set:

def compare(dataset, model):
    metrics = {"greedy": 0, "self_consistency": 0, "self_verify": 0}
    for q, gold in dataset:
        chains = [model.cot(q, temperature=0.7) for _ in range(5)]
        metrics["greedy"]           += chains[0].answer == gold
        metrics["self_consistency"] += majority(chains) == gold
        metrics["self_verify"]      += self_verify(q, model)[0] == gold
    return {k: v / len(dataset) for k, v in metrics.items()}

Report exact-match accuracy per strategy. The interpretable score is a second signal: bucket questions by verification margin and you'll see low-margin answers are likelier to be wrong — useful for routing hard cases to a human.

Limitations

• It assumes a checkable answer. If the output can't be folded back into the problem as a condition, there's nothing to verify — so open-ended generation is out of scope.
• It inherits the model's weaknesses. A model that's bad at the forward task is usually bad at the backward one too; the +0.08 on weak GPT-3 shows verification doesn't manufacture ability.
• Commonsense gains are thin. Without a number to mask, TFV is a coarser signal (+0.41 on CommonsenseQA), so don't expect arithmetic-sized wins.
• It's expensive. The K×P call structure is the real tax — many tasks won't justify it over plain self-consistency.

Advanced techniques

Two refinements help. First, adaptive P: stop verifying early once one candidate's lead is statistically safe, instead of always running the full budget. Second, multi-condition masking: for problems with several numbers, verify against every maskable slot and average — a wrong chain usually fails to regenerate at least one of them, so more masks means a sharper score.

Ecosystem

Self-verification sits in a family of "check your reasoning" methods. They differ in what gets checked and when.

It composes cleanly with self-consistency (verify, then re-rank the votes) and slots into agent loops as a gate before an answer is committed downstream.

Future directions

The open questions are about cost and reach: can a lightweight verifier match the full K×P budget, can backward verification extend cleanly to non-numeric domains where masking is hard, and can the interpretable score drive selective human review at scale? As models get better at self-checking, expect verification to move from a bolt-on pass into the reasoning loop itself.

Summary

• Self-verification generates K candidate answers with CoT sampling, then verifies each backward by masking an original condition and checking if the candidate can reconstruct it.
• The candidate that best rebuilds its own premises wins — and you get an interpretable match score, not just a vote.
• Use Condition Mask Verification for arithmetic (mask a number) and True-False Item Verification for commonsense/logic (judge consistency).
• It lifts GSM8K from 60.81% to 65.14% (+4.33) on code-davinci-002, with consistent gains across AddSub, MultiArith, AQuA, SingleEq, and SVAMP.
• It needs a strong base model and costs K×P extra calls; weak models barely benefit (+0.08 on GPT-3) and open-ended outputs can't be verified at all.
• Reach for it when accuracy and an auditable rationale both matter; otherwise self-consistency is the cheaper default (Weng et al., EMNLP 2023 Findings).