Self-consistency prompting: a complete guide

Chain-of-thought gives a model room to reason, but it still commits to one path — and one wrong turn early sinks the whole answer. Self-consistency flips that: instead of trusting a single greedy chain, you sample several, then keep the answer most of them agree on. On GSM8K with PaLM-540B it pushed accuracy from 56.5% to 74.4% — a 17.9-point jump over plain chain-of-thought (Wang et al., 2022).

See it work

Take a small word problem and let a model reason step by step. With greedy decoding you get exactly one chain, and if its arithmetic slips, you're stuck with the wrong number.

Prompt (chain-of-thought):
Q: When I was 6, my sister was half my age. Now I'm 70. How old is my sister?
A: Let's think step by step.

Output (one greedy chain):
When you were 6, your sister was 3. So she is 70 / 2 = 35. The answer is 35.

That chain reasons fine until the last line, then fumbles. Now sample the same prompt a few times at non-zero temperature and collect the final answers:

Path 1: sister was 6 - 3 = 3 years younger; 70 - 3 = 67.  -> 67
Path 2: age gap is 3 years, so 70 - 3 = 67.               -> 67
Path 3: she is half my age, 70 / 2 = 35.                  -> 35

Tally: {67: 2, 35: 1}  ->  Final answer: 67

No single call became smarter. You just stopped betting everything on one chain and let the answers vote. The lone 35 gets outvoted, and the consensus 67 wins.

The mental model

Think of it like getting a second and third opinion before a big decision. One doctor might miss something; three doctors who land on the same diagnosis give you real confidence. The wrong calls tend to scatter in different directions, while the right call keeps showing up.

Self-consistency: reason many ways, then trust the answer that most paths converge on.

The key insight is that a hard problem usually has several valid routes to the same correct answer, but many distinct ways to be wrong. Correct reasoning concentrates; mistakes spread out. Majority voting exploits exactly that asymmetry.

How it works

You keep the chain-of-thought prompt unchanged. What changes is decoding: sample instead of taking the greedy path, do it several times, then aggregate the final answers.

1. Prompt with chain-of-thought. Use the same few-shot or zero-shot CoT prompt you'd use normally.
2. Sample diverse paths. Run the prompt N times with non-zero temperature so each generation explores a different chain.
3. Extract the final answer from each chain (the boxed number, the picked option, the label).
4. Marginalize over paths. Ignore how each chain got there; count only the final answers.
5. Take the majority. The most frequent answer wins. Ties break arbitrarily or by sampling more.

The paper frames this as marginalizing out the reasoning paths: many latent chains, one answer you actually care about.

Why it works

Ranked by how much each factor drives the gain:

Where it shines

Self-consistency helps most when a problem has one checkable answer reached by multi-step reasoning. The original results, all measured as gains over chain-of-thought prompting:

• Arithmetic reasoning — GSM8K +17.9% (56.5% to 74.4% on PaLM-540B), SVAMP +11.0%, AQuA +12.2%.
• Commonsense reasoning — StrategyQA +6.4%, ARC-challenge +3.9%.

The pattern holds across model families and scales — the paper reports gains on UL2-20B, LaMDA-137B, GPT-3, and PaLM-540B — with no extra training, no fine-tuning, and no auxiliary verifier. It's purely a decoding-time change, which is part of why it caught on.

When to use it (and when not)

Reach for it when:

• The final answer is a discrete value — a number, a multiple-choice letter, a class label.
• The task needs multi-step reasoning where early errors compound.
• A single CoT call gives unstable answers across runs.
• Reliability matters more than latency or per-call cost.

Skip it when:

• The output is free-form text (an essay, code, a translation) with no natural way to tally votes.
• The task is simple enough that greedy decoding is already correct.
• You're latency- or budget-bound and can't afford N calls per question.

Model fit. It needs a model that can already produce coherent CoT chains, so it pairs best with capable instruction-tuned models. On a model that can't reason step by step, voting just averages noise.

Escalation. If majority voting still misses, move to a weighted or verifier-scored vote, or to a search-style method like tree-of-thoughts that actively explores and prunes paths rather than sampling blindly.

Structure and components

There's no special prompt format — that's the appeal. You need three pieces around an ordinary CoT prompt:

• A chain-of-thought prompt (few-shot exemplars or a zero-shot "let's think step by step").
• A sampler configured for diversity (temperature above 0, optionally top-k or top-p).
• An aggregator that parses each final answer and counts votes.

[CoT prompt template]
Q: <question>
A: Let's think step by step. <reasoning> The answer is <answer>.

x N samples  ->  extract <answer> from each  ->  majority vote

The core algorithm

The whole method is a sampling loop plus a counter:

from collections import Counter

def self_consistency(model, prompt, n=10, temperature=0.7):
    answers = []
    for _ in range(n):
        chain = model.generate(prompt, temperature=temperature)
        answers.append(extract_answer(chain))   # parse the final value
    votes = Counter(answers)
    return votes.most_common(1)[0][0]            # most frequent answer

extract_answer is the part worth getting right — a flaky parser turns a correct chain into a spoiled vote. Anchor the answer with a fixed phrase ("The answer is X.") so extraction is a clean regex.

Configuration

Results in the paper were averaged over 10 runs, a reminder that sampled methods are themselves noisy — measure across several runs, not one.

Implementation workflow

1. Start from a working CoT prompt that ends each answer with a fixed marker phrase.
2. Write a robust extract_answer and unit-test it on real outputs.
3. Sample N chains at non-zero temperature (parallelize the calls — they're independent).
4. Tally answers and return the majority.
5. Sweep N (5, 10, 20, 40) on a dev set and stop where accuracy flattens.

Do

• Parallelize the N calls; they don't depend on each other.
• Log the full vote distribution — its spread is a free confidence signal.
• Cap N where the accuracy curve plateaus instead of paying for 40 by default.

Don't

• Use temperature 0 — identical greedy chains defeat the whole point.
• Vote on raw text; normalize answers first (trim, lowercase, canonicalize units).
• Apply it to open-ended generation without a real consistency metric.

Debugging

Testing and how to prove it

Measure it against the obvious baseline: greedy CoT on the same dev set. Then sweep the number of samples and watch the curve.

def compare(model, dataset, n=10):
    cot_hits = sc_hits = 0
    for q, gold in dataset:
        cot = extract_answer(model.generate(q, temperature=0))   # greedy CoT
        sc  = self_consistency(model, q, n=n)                    # voted
        cot_hits += (cot == gold)
        sc_hits  += (sc  == gold)
    return cot_hits / len(dataset), sc_hits / len(dataset)

Because sampling is stochastic, average the self-consistency accuracy over several runs before claiming a win, exactly as the paper did over 10 runs.

Limitations

• Fixed-answer only. The method needs a discrete answer set to vote over. Free-form outputs have no natural majority without a separate consistency metric.
• Linear cost. N samples mean roughly N× the tokens and latency of one CoT call.
• Saturating returns. Accuracy climbs then flattens; past a point you pay for samples that barely move the result.
• Garbage in, garbage out. If most chains share the same systematic error, the majority can be confidently wrong. Voting reduces random variance, not shared bias.
• Reasoning-dependent. It amplifies a model that can already reason; it won't bootstrap reasoning that isn't there.

Advanced variants

• Weighted voting. Weight each path by its likelihood or a confidence score instead of counting equally — useful when some chains are clearly stronger.
• Universal self-consistency. For free-form answers, feed all sampled responses back to the model and ask it to pick the most consistent one, replacing exact-match voting with an LLM judge.
• Early stopping. Sample incrementally and stop once one answer holds a decisive lead, saving calls on easy questions.

Ecosystem and related techniques

Most modern reasoning stacks treat self-consistency as a default reliability layer: cheap to bolt on, easy to reason about, and model-agnostic.

Summary

• Self-consistency samples several chain-of-thought paths and returns the answer the majority agree on, instead of trusting one greedy chain.
• It works because correct reasoning converges while mistakes scatter, so voting on the final answer filters out random errors.
• Headline result: GSM8K 56.5% to 74.4% on PaLM-540B (+17.9% over CoT), with gains across SVAMP, AQuA, StrategyQA, and ARC-challenge, and across UL2-20B, LaMDA-137B, GPT-3, and PaLM-540B (Wang et al., 2022).
• Use it for discrete-answer reasoning tasks; skip it for free-form text or when greedy CoT is already correct.
• Cost scales with the number of samples, but returns saturate — 5 to 10 paths usually capture most of the gain before you reach the paper's 40.
• It's a decoding-time wrapper: no training, model-agnostic, and stacks on top of any CoT prompt.