Re-reading (RE2): a complete guide

Most reasoning tricks work on the model's output. Re-reading works on the input. You just repeat the question with the line "Read the question again:" before letting the model answer, and that second pass lets a left-to-right model finally see the whole question at once. Across 14 datasets and 112 experiments, this one-line change consistently lifted reasoning accuracy (Xu et al., EMNLP 2024).

See it work

Take a word problem with a detail buried in the middle. A model that reads strictly left to right can latch onto the first numbers and miss the twist.

Prompt (vanilla):
Roger has 5 tennis balls. He buys 2 cans of balls. Each can has 3 balls,
but one ball in each can is already broken. How many usable balls does
Roger have now?

Output:
5 + 2 x 3 = 11 balls.   (misses the broken balls)

Now add the re-reading line. The model gets a second look at the full constraint set before it commits to a plan.

Prompt (RE2):
Roger has 5 tennis balls. He buys 2 cans of balls. Each can has 3 balls,
but one ball in each can is already broken. How many usable balls does
Roger have now?
Read the question again: Roger has 5 tennis balls. He buys 2 cans of
balls. Each can has 3 balls, but one ball in each can is already broken.
How many usable balls does Roger have now?
Let's think step by step.

Output:
Each can has 3 balls minus 1 broken = 2 usable. 2 cans x 2 = 4 usable.
5 + 4 = 9 usable balls.

Same model, same temperature, same question. The only change is that the model read the question twice. (The outputs above are illustrative, to show the failure mode RE2 fixes.)

The mental model

Think about how you read a tricky exam question. You skim it once, sense there's a catch, then read it again slowly before you pick up your pencil. The first pass builds a map; the second pass fills it in with the details you now know to look for.

RE2 gives a decoder-only model that same second pass. The first reading becomes context for the second, so words at the start of the question can finally "see" the words at the end.

How it works

1. Take the question exactly as written. No rephrasing, no summarizing.
2. Repeat it verbatim after the line Read the question again:.
3. Add a reasoning trigger if you want it, such as the chain-of-thought line Let's think step by step.
4. Send one prompt. RE2 is single-pass at the API level; the "two readings" both live inside the same prompt, so there's no extra round trip.
5. Read the answer. The model has already absorbed every constraint before it starts planning.

Why it works

Decoder-only models read left to right with causal attention, so a token can only attend to tokens before it. The first time the model reads "5 tennis balls," it has no idea a "broken" twist is coming later. Repeating the question fixes that: during the second reading, every word can attend back to the complete first copy, which approximates the bidirectional encoding that encoder models get for free.

Where it shines

RE2 was tested on arithmetic, commonsense, and symbolic reasoning, and the gains held across both instruction-tuned and base models. On GSM8K (grade-school math):

• ChatGPT (gpt-3.5-turbo-0613): vanilla 77.79% rose to 79.45% with RE2; chain-of-thought 78.77% rose to 80.59% with CoT+RE2.
• text-davinci-003: vanilla 19.48% jumped to 24.79% with RE2 (+5.31 points); CoT 58.98% rose to 61.64% with CoT+RE2.

The pattern: the biggest lifts show up on weaker or non-instruction-tuned models and on detail-heavy questions, while strong instruction-tuned models still gain a smaller but real amount. RE2 was also shown to combine with different LLMs, with thought-eliciting prompts, and with ensemble strategies like self-consistency.

When to use it (and when not)

Reach for it when:

• The question is long, multi-clause, or hides a constraint in the middle.
• You're on a smaller or base model that drops details.
• You already use CoT and want a near-free accuracy bump on top.

Skip it when:

• The prompt is short and unambiguous; re-reading just burns tokens.
• The bottleneck is missing knowledge or a deep logical gap, not comprehension. Re-reading the question won't teach the model facts it never had.
• You're latency- or cost-bound on very high volume.

Model fit: works across model sizes, but the relative payoff is largest on weaker and non-instruction-tuned models. Escalation: if RE2+CoT still misses, add self-consistency (sample several reasoning paths and vote) before reaching for fine-tuning or retrieval.

Structure and components

An RE2 prompt has three parts, only the first two of which are required:

1. The question, written once.
2. The re-read trigger plus the question again. The exact phrasing matters: "Read the question again:" beat plain repetition in the paper (79.45% vs 78.09% on GSM8K with ChatGPT), so keep the instruction explicit.
3. An optional reasoning trigger such as Let's think step by step.

{question}
Read the question again: {question}
{optional reasoning trigger}

Building it is just string templating:

def re2_prompt(question: str, cot: bool = True) -> str:
    prompt = f"{question}\nRead the question again: {question}\n"
    if cot:
        prompt += "Let's think step by step."
    return prompt

Configuration

Implementation workflow

1. Start from your existing prompt and baseline accuracy.
2. Wrap the question with the re-read line; leave everything else untouched.
3. If the task needs reasoning, append your CoT trigger after the second copy.
4. Measure against the baseline on a held-out set; RE2 should help or stay flat.
5. If a long question makes token cost spike, decide whether the accuracy gain is worth it for that workload.

Do: keep the repeated question verbatim, place the re-read before any reasoning trigger, and A/B against your baseline. Don't: paraphrase the second copy, re-read four-plus times, or apply it blindly to short prompts where it only adds cost.

Debugging

• No improvement on a strong model: expected for short, clear questions; the gain concentrates on detail-heavy inputs and weaker models.
• Accuracy dropped: check the re-read count (cap at 2-3) and confirm the second copy is identical, not paraphrased.
• Token budget blew up: the question is long; re-reading doubles it. Consider trimming the question or skipping RE2 for that case.
• Still wrong after RE2+CoT: the failure is likely knowledge or logic, not reading. Move to self-consistency, retrieval, or a stronger model.

Testing and how to prove it

Run a paired comparison: the same questions, same model, with and without the re-read wrapper, then compare accuracy on a held-out set.

def evaluate(questions, answers, ask):
    base = sum(ask(q) == a for q, a in zip(questions, answers))
    re2 = sum(ask(re2_prompt(q)) == a for q, a in zip(questions, answers))
    return base / len(questions), re2 / len(questions)

Because the only change is the prompt wrapper, any accuracy delta is attributable to re-reading. Report exact-match accuracy and, for sampled outputs, average over a few runs to smooth variance.

Limitations

• Comprehension only. RE2 helps the model use what's in the question. It can't supply missing knowledge or repair a flawed reasoning chain.
• Diminishing and then negative returns. Beyond two or three reads, accuracy declines.
• Token and latency cost. It enlarges the input, which matters most for long questions at scale.
• Not universal. The paper found a few vanilla-ChatGPT scenarios where RE2 didn't help, so validate on your own task rather than assuming a guaranteed win.

Advanced techniques

The strongest documented combination is CoT+RE2: re-read the question, then trigger step-by-step reasoning. Layer self-consistency on top by sampling multiple RE2+CoT completions and voting on the answer. RE2 also slots into ensemble and thought-eliciting pipelines because it only touches the input, leaving every output-side method free to do its job.

Summary

• RE2 means appending "Read the question again:" plus a verbatim copy of the question before the model answers.
• It works on the input, not the output, recovering bidirectional context for left-to-right decoder-only models.
• It stacks cleanly with chain-of-thought and self-consistency, since it changes only the question.
• Gains are largest on detail-heavy questions and weaker or non-instruction-tuned models.
• Read twice, not four times: benefits peak at two or three passes, then fade.
• Reach for it on long, tricky questions; skip it on short prompts or when the gap is knowledge, not reading.