Least-to-most prompting: a complete guide

Chain-of-thought reasons in one pass, so it stumbles when the test problem is harder than the examples you showed it. Least-to-most prompting fixes that by splitting the work into two stages: first decompose the problem into a chain of easier subproblems, then solve them in order, feeding each answer into the next. On the SCAN compositional-generalization benchmark, this took code-davinci-002 from 16.2% with chain-of-thought to 99.7% accuracy using just 14 exemplars (Zhou et al., 2022, ICLR 2023).

See it work

Here's a problem where one-pass reasoning quietly drops a detail.

Problem: It takes Amy 4 minutes to climb to the top of a slide.
It takes her 1 minute to slide down. The slide closes in 15 minutes.
How many times can she slide before it closes?

Chain-of-thought (one pass):
  "15 / 4 = 3.75, so she can slide 3 times."   (forgets the slide-down time)

Least-to-most (decompose, then solve in order):
  Subproblem 1 - How long does each trip take?
    4 + 1 = 5 minutes.
  Subproblem 2 - How many times can she slide in 15 minutes?
    15 / 5 = 3 times.   (correct, and for the right reason)

The model outputs above are illustrative, but the shape is the point. The single pass tries to leap straight to the answer. Least-to-most forces an easy first question whose answer (each trip is 5 minutes) is exactly what the hard question needs.

The mental model

Think of a tutor who never hands you the final answer. They ask the small question you can answer right now, then use your answer to ask the next slightly harder one, until the original problem has quietly already been solved.

Don't ask the model to leap. Build it a staircase of subproblems, each step standing on the answer below it.

How it works

Least-to-most runs in two distinct stages, each with its own prompt. The first prompt only decomposes; the second only solves. Crucially, the solving stage is sequential, so each subproblem is answered with all prior answers already in context.

1. Decompose. A few-shot prompt teaches the model to list the subproblems needed to reach the goal, ordered from least to most complex. The last subproblem is usually the original question.
2. Solve the first subproblem. Answer the easiest one in isolation. It needs no prior context.
3. Carry the answer forward. Append the subproblem and its answer to the working context.
4. Solve the next subproblem with everything so far. The model sees prior questions and answers, so it can reuse them instead of recomputing.
5. Repeat to the final subproblem. Its answer is the answer to the original problem.

Why it works

The gains come from a few reinforcing effects, roughly in order of impact.

Where it shines

Least-to-most pays off most when a task gets harder as inputs grow, and the difficulty is composed of repeatable smaller steps. The published results on code-davinci-002 make the pattern concrete.

• Compositional generalization (SCAN). Mapping commands like "run left and walk twice" to action sequences. Least-to-most hit 99.7% with 14 exemplars, versus 16.2% for chain-of-thought and 16.7% for standard prompting on the length split. Specialized neural-symbolic models reached comparable accuracy only after training on more than 15,000 examples.
• Symbolic manipulation (last-letter concatenation). Take the last letter of each word and concatenate. The gap widens with list length. At 4 words, least-to-most scored 94.0% versus 84.2% for chain-of-thought; at 12 words, 74.0% versus 31.8%. Standard prompting scored 0.0% throughout.
• Numerical reasoning (DROP). On non-football questions, 82.45% versus 74.77% for chain-of-thought; on football questions, 73.42% versus 59.56%.
• Math word problems (GSM8K). A smaller but real lift: 62.39% versus 60.87% overall, widening to 45.23% versus 39.07% on problems needing at least five reasoning steps.

The throughline: the harder the instance relative to the exemplars, the bigger the win.

When to use it (and when not)

Reach for it when:

• Test inputs are longer or harder than what fits in your exemplars.
• The problem has a natural easy-to-hard ordering of steps.
• Later steps genuinely depend on earlier results.
• You need compositional generalization, not just a single chain.

Skip it when:

• The task is one atomic step; decomposition adds latency for nothing.
• Subproblems are independent and could run in parallel.
• You can't articulate a sensible decomposition, even by hand.

Model fit. The published results lean on strong base models (code-davinci-002, text-davinci-002). Smaller models often decompose poorly, which poisons the whole chain. Verify decomposition quality before trusting the pipeline on a weaker model.

Escalation. If chain-of-thought already clears your bar on in-distribution problems, stay there. Move to least-to-most when accuracy collapses on the long or hard tail.

Structure and components

A least-to-most setup is two prompts, not one.

Stage 1 - Decomposition prompt
Q: <complex problem>
A: To solve this, we first need to solve:
   1. <easiest subproblem>
   2. <next subproblem>
   ...
   N. <the original problem>

Stage 2 - Sequential solving prompt
Q: <subproblem 1>
A: <answer 1>
Q: <subproblem 2>          # answer 1 is now in context
A: <answer 2>
...
Q: <original problem>      # all prior answers available
A: <final answer>

The required pieces: a decomposition exemplar set that demonstrates ordering, a solving exemplar set that demonstrates reusing prior answers, and a mechanism to append each answer back into context. Optional pieces include a fixed subproblem template for highly regular tasks (like SCAN's reduction-then-mapping split).

The core mechanism

The whole technique is a decomposition call followed by a fold over subproblems, accumulating context.

def least_to_most(problem, decompose_prompt, solve_prompt, llm):
    # Stage 1: produce an ordered list of subproblems, easy -> hard
    subproblems = parse_list(llm(decompose_prompt + problem))

    # Stage 2: solve in order, each step sees all prior answers
    context, answer = "", None
    for sub in subproblems:
        answer = llm(solve_prompt + context + f"\nQ: {sub}\nA:")
        context += f"\nQ: {sub}\nA: {answer}"
    return answer  # the answer to the final (hardest) subproblem

Configuration

Implementation workflow

1. Pick a handful of representative problems, including some hard ones.
2. Hand-write the ideal decomposition for each. If you can't, the task may not suit this technique.
3. Build the decomposition prompt from those exemplars and test it alone. Bad decompositions sink everything downstream.
4. Build the solving prompt, ensuring exemplars visibly reuse earlier answers.
5. Wire the loop that appends each answer to context.
6. Evaluate on a difficulty-stratified test set and compare against chain-of-thought.

Do

• Keep subproblems genuinely easy at the start.
• Make the final subproblem restate the original question.
• Show answer reuse explicitly in solving exemplars.

Don't

• Cram decomposition and solving into one prompt.
• Let decompositions balloon into dozens of trivial micro-steps.
• Assume a small model decomposes as well as a large one.

Debugging

Testing and how to prove it

The honest test is whether accuracy holds as problems get harder. Stratify your test set by difficulty (word count, step count, command length) and watch the gap against chain-of-thought widen on the hard buckets, exactly as the paper's last-letter results do.

# Prove it generalizes: bucket the test set by difficulty, compare per bucket
for length in [4, 6, 8, 10, 12]:
    bucket = [x for x in testset if x.length == length]
    cot = accuracy(run_chain_of_thought, bucket)
    ltm = accuracy(run_least_to_most, bucket)
    print(length, cot, ltm)  # the ltm - cot gap should grow with length

Limitations

• Domain-specific decomposition. The paper notes that a decomposition prompt that works for one domain often fails to generalize to others; you frequently rewrite it per task family.
• Error propagation. A wrong early answer is treated as fact by every later step. The sequential structure has no built-in recovery.
• Not every problem decomposes. Tasks without a natural easy-to-hard ordering gain little, and the overhead is pure cost.
• Modest on already-easy tasks. On GSM8K overall the lift was small (62.39% versus 60.87%); the payoff concentrates on the hard tail.

Advanced techniques

For highly regular tasks, fix the decomposition shape instead of generating it freely. SCAN used a two-part split: a reduction step that rewrites the command into explicit intermediate language, then a mapping step that emits the action sequence. You can also self-verify each subproblem answer before carrying it forward, which blunts error propagation at the cost of more calls. And least-to-most composes with self-consistency: sample the final step several times and vote.

Ecosystem and transitions

Coming from chain-of-thought, you already have solving exemplars; the new work is writing the decomposition prompt and the loop that carries answers forward. Coming from decomposed prompting, the difference is sequencing: least-to-most insists later steps consume earlier answers, where decomposed prompting often runs modular subtasks independently. Moving onward, if independent subproblems dominate, a parallel or agentic decomposition will be cheaper than a strict chain.

Future directions

Open threads include automatic decomposition (letting the model decide where to split without hand-written exemplars), recovery mechanisms so a wrong subproblem answer doesn't doom the chain, and tighter integration with tool use, where each subproblem dispatches to a calculator, retriever, or code runner.

Summary

• Least-to-most splits reasoning into two stages: decompose into ordered subproblems, then solve them sequentially with answer reuse.
• Its signature strength is generalizing to problems harder than the exemplars, not just matching them.
• Headline result: 99.7% on SCAN with code-davinci-002 and 14 exemplars, versus 16.2% for chain-of-thought (Zhou et al., 2022, ICLR 2023).
• Gains grow with difficulty: last-letter concatenation went from 31.8% to 74.0% at 12 words; DROP and GSM8K improved most on the hard tail.
• Costs scale with call count, and a wrong early answer propagates, so verify decomposition quality and budget for multi-stage latency.
• Reach for it when test inputs outgrow your examples and steps depend on each other; stay with chain-of-thought when problems sit near the exemplars.