Self-calibration: a complete guide

A model will hand you a wrong answer with exactly the same swagger as a right one. Self-calibration fixes the swagger, not the answer: after the model responds, you ask it a second question — "is that answer correct?" — and read the confidence behind its reply. The trick comes from Anthropic's "Language Models (Mostly) Know What They Know" (Kadavath et al., 2022), which showed that larger models, asked in the right format, can estimate the probability that their own answer is true and stay surprisingly well-calibrated while doing it.

See it work

You don't change the first answer. You add a second pass that scores it.

PASS 1 — answer
Prompt:  Which planet has the most moons?
Output:  Jupiter.

PASS 2 — self-calibrate (new prompt, shows the Q and the A back to the model)
Prompt:
  Question: Which planet has the most moons?
  Proposed answer: Jupiter.
  Is the proposed answer correct?
  (A) Correct   (B) Incorrect
  The proposed answer is:

Output:  (B) Incorrect — confidence ~0.7. Recent counts put Saturn ahead of Jupiter.

Same model, same weights. The first pass produced a fluent-but-shaky fact; the second pass, framed as a true/false judgment, surfaced the doubt that pass one buried. Now you have a number to act on, not just a sentence.

The mental model

Think of a student who finishes an exam, then gets five minutes to mark each answer "sure / not sure." They can't see the answer key, but they often know which ones they guessed. That self-marking is what you're eliciting.

Self-calibration doesn't make the answer better. It tells you how much to trust the answer you already have.

That distinction is the whole technique. Self-refine rewrites the draft; self-calibration grades it.

How it works

1. Answer. Prompt the model normally and capture its answer.
2. Reframe. Build a fresh prompt that shows the question and the proposed answer, then asks whether that answer is correct — as a true/false or multiple-choice judgment.
3. Score. Read the model's confidence. Kadavath et al. call this P(True): the probability the model assigns to its own answer being right. You can read it from the probability of the "True" token, or, on chat APIs without logits, ask the model to state a confidence verbally.
4. Decide. Compare P(True) to a threshold you set. Above it, trust the answer; below it, flag it, resample, or hand off to a human or a stronger model.

Why it works

Ranked by how much each factor drives the result:

The core finding from Kadavath et al.: when questions are posed as well-formatted multiple-choice or true/false, large models are already well-calibrated, so you can bootstrap self-evaluation on open-ended tasks by having the model propose an answer and then score P(True).

Where it shines

Self-calibration earns its keep wherever a wrong answer is expensive and you'd rather abstain than bluff:

• High-stakes QA — medical, legal, or financial lookups where "I'm not sure" should route to a human.
• Selective prediction — answer only when confident, defer the rest. The threshold becomes a coverage/accuracy dial.
• RAG pipelines — score whether a retrieved-then-generated answer is actually supported before showing it. Calibrated confidence rises appropriately when relevant source material is in context.
• Agent guardrails — gate a tool call or a final action on the model's own confidence in the plan.
• Triaging ensembles — when self-consistency or multiple samples disagree, P(True) helps pick which candidate to trust.

The payoff isn't a higher accuracy number on its own answers — it's a confidence signal good enough to sort the trustworthy answers from the shaky ones.

When to use it (and when not)

Reach for it when:

• You need to abstain, defer, or escalate rather than always answer.
• The task has a checkable notion of "correct" (facts, classifications, math).
• You're on a frontier-scale model whose confidence means something.

Skip it when:

• The output is open-ended or creative — "correct" isn't well defined.
• You're on a small model; its self-judgment is poorly calibrated.
• You just want a better answer — use self-refine or self-consistency instead.

Model fit: this is a large-model technique. Kadavath et al. found calibration improves with scale; zero-shot true/false self-evaluation on smaller models is poorly calibrated. Verify on your own model before trusting the number.

Escalation: if P(True) sits below your threshold, the answer is a candidate for resampling, a retrieval step, or routing to a stronger model or a human.

The calibration vocabulary

A model is calibrated when its confidence matches its accuracy: of all the answers it calls 80% likely, about 80% should actually be right. Confidence and accuracy are different axes — a model can be accurate but overconfident, or cautious but right.

You measure the gap with expected calibration error (ECE). Bin predictions by stated confidence, then compare each bin's average confidence to its actual accuracy:

ECE = Σ_m (|B_m| / N) · |acc(B_m) − conf(B_m)|

B_m is bin m, N is the total number of predictions, acc is the fraction correct in the bin, and conf is the average confidence in the bin. ECE of 0 means perfectly calibrated; plotting accuracy against confidence gives a reliability diagram, where a perfectly calibrated model sits on the y = x line. ECE is what tells you a P(True) threshold is meaningful rather than wishful.

The calibration prompt

The format does the heavy lifting. Show the question, show the proposed answer, ask for a binary verdict, then read the probability of the affirmative token:

Question: {question}
Proposed answer: {proposed_answer}

Is the proposed answer correct?
(A) Correct
(B) Incorrect

The proposed answer is:

P(True) is the model's probability mass on "A" (Correct). On APIs that expose token logprobs, read it directly. On chat APIs that don't, ask for an explicit verbalized confidence — "Answer True or False, then a probability from 0 to 1" — and parse it, accepting that verbalized confidence is rougher than logit-based P(True).

Core mechanism

A minimal two-pass loop. The first call answers; the second judges and returns a confidence you threshold on.

def answer_then_calibrate(question, llm, threshold=0.6):
    # Pass 1: get the answer.
    answer = llm(f"Answer concisely.\n\nQuestion: {question}")

    # Pass 2: ask the model to judge its own answer.
    judge_prompt = (
        f"Question: {question}\n"
        f"Proposed answer: {answer}\n\n"
        "Is the proposed answer correct? Reply 'True' or 'False', "
        "then your confidence as a probability from 0 to 1.\n"
        "Format: <verdict> | <probability>"
    )
    verdict, prob = parse(llm(judge_prompt))      # e.g. "True | 0.82"
    p_true = prob if verdict == "True" else 1 - prob

    return {
        "answer": answer,
        "confidence": p_true,
        "trust": p_true >= threshold,             # gate the decision
    }

For the original logit-based variant, swap pass two for a call that returns token logprobs and take P(True) as the probability of the "Correct" token. Letting the model see several sampled answers before it judges one — as in the paper — tightens the estimate.

Measuring whether it's calibrated

Don't trust a confidence score you haven't checked. On a labeled holdout set, compute ECE so you know the threshold means something:

import numpy as np

def ece(confidences, correct, n_bins=10):
    confidences, correct = np.asarray(confidences), np.asarray(correct)
    bins = np.linspace(0, 1, n_bins + 1)
    score = 0.0
    for lo, hi in zip(bins[:-1], bins[1:]):
        in_bin = (confidences > lo) & (confidences <= hi)
        if in_bin.any():
            acc = correct[in_bin].mean()          # actual accuracy in bin
            conf = confidences[in_bin].mean()      # average stated confidence
            score += in_bin.mean() * abs(acc - conf)
    return score

A low ECE means your P(True) threshold reliably separates good answers from bad. A high ECE means the model is over- or under-confident, and you should recalibrate (for instance with temperature scaling) before relying on the gate.

Configuration

Implementation workflow

1. Pick the decisions worth gating — not every answer needs a confidence score.
2. Get the answer (pass one), then build the judge prompt with the question and that answer.
3. Read P(True) from logprobs, or parse a verbalized confidence.
4. On a labeled set, sweep the threshold and compute ECE; pick the operating point your product needs.
5. Wire the threshold into routing: trust, resample, retrieve, or escalate.
6. Re-check ECE whenever you change models — calibration is model-specific.

Do

• Show the proposed answer back to the model and ask a crisp true/false.
• Validate the score with ECE before you trust the threshold.
• Keep the judge prompt's format identical to whatever you calibrated on.

Don't

• Assume a small model's self-confidence means anything.
• Treat verbalized confidence as if it were a measured probability.
• Read a low P(True) as a fixed answer — it's a signal to act, not a correction.

Debugging

• Confidence is always high (or always 0.99): the model is overconfident; check ECE, lower judge temperature, and consider temperature scaling.
• Confidence doesn't track accuracy: the format is off, or the model is too small — switch to a larger model or the answer-visible true/false format.
• Threshold passes too many wrong answers: raise the threshold and re-measure coverage vs accuracy on your holdout.
• Verbalized confidence is erratic: constrain the output format tightly, or move to logit-based P(True) if your API exposes logprobs.

Limitations

• It scores, it doesn't fix. A low P(True) flags a shaky answer; producing a better one is a different technique's job.
• Scale-dependent. The whole effect leans on model size; small models are poorly calibrated and their self-judgment can mislead.
• Calibration drifts. P(IK) — the model's sense of whether it knows — generalizes only partially and degrades on new tasks, so confidence learned on one domain may not transfer.
• Open-ended tasks resist it. When there's no clean notion of "correct," there's nothing crisp to judge.
• Cost. Every gated answer costs at least one extra call.

Summary

• Self-calibration is a second pass: the model answers, then judges whether its own answer is correct, giving you a confidence score.
• It tells you how much to trust an answer — it doesn't rewrite it. That's what separates it from self-refine and self-verification.
• The mechanism is P(True) from Kadavath et al. (2022): show the question and answer back, ask true/false, read the probability of "correct."
• It's a large-model technique; calibration emerges with scale and clean formatting, and is poor on small models.
• Verify the signal with expected calibration error before trusting any threshold, and re-check it when you swap models.
• Use it to abstain, defer, or escalate on high-stakes answers; skip it for open-ended work and when you just want a better answer.