Program of thoughts (PoT): a complete guide

Language models are great at reasoning but bad at arithmetic. Chain-of-thought makes the model do both in its head, so a single botched multiplication sinks the whole answer. Program of thoughts (PoT) flips that: the model writes the reasoning as a Python program and hands the actual math to an interpreter. On eight math and finance benchmarks it beat chain-of-thought by roughly 12% on average (Chen et al., 2022).

See it work

Ask a model for the 50th Fibonacci number. Chain-of-thought tries to track the running sum in prose, and it almost always drifts somewhere around the tenth term.

Prompt (CoT):
Q: What is the 50th Fibonacci number (starting 0, 1)? Let's think step by step.

Output:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, ... (the model keeps adding by hand)
... I think it's around 12586269025? (often wrong by a few terms)

Now ask the same question, but tell the model to write a program. It stops doing mental math and describes the procedure instead.

Prompt (PoT):
Q: What is the 50th Fibonacci number (starting 0, 1)?
# Write Python. Store the result in `ans`.

Output:

fib = [0, 1]
for i in range(2, 50):
    fib.append(fib[i-1] + fib[i-2])
ans = fib[49]   # 7778742049

The interpreter runs the loop and returns 7778742049 — exact, every time. The model never had to be good at addition; it only had to be good at describing addition.

The mental model

Think of a smart colleague with a calculator. You wouldn't ask them to multiply six-digit numbers in their head — you'd ask them to set up the equation and let the calculator finish. PoT splits the job along the same seam.

Reasoning is the model's job. Computation is the computer's job. Don't make one do the other.

Chain-of-thought blurs that line: the model reasons and calculates in the same token stream, where it's strong at the first and shaky at the second. PoT keeps the model in its lane.

How it works

1. Prompt for code, not prose. Give the model the question and ask it to express the solution as a program (Python by default), storing the result in a variable like ans.
2. Bind variables to meaning. The model names variables after the quantities they represent — interest_rate, num_months — instead of x and y. This semantic binding is what keeps the generated code aligned with the word problem.
3. Offload the math. The model writes the relationships; it does not evaluate them. Multiplication, iteration, equation-solving (via libraries like SymPy) all go to the interpreter.
4. Execute and read off the answer. A Python interpreter runs the program and returns the value of ans. That value is the answer — no second LLM pass needed.

Why it works

Where it shines

PoT pays off whenever the answer hinges on exact numbers rather than fluent text.

• Math word problems — arithmetic, algebra, and multi-step quantitative reasoning.
• Financial QA — multi-row table math, percentages, and ratios where CoT slips most.
• Tabular reasoning — pulling values from a table and combining them.
• Any task with a clean numeric answer that a program can compute and verify.

The headline numbers, all using Codex (code-davinci-002) in the original paper:

Finance is the standout: FinQA jumps over 24 points and ConvFinQA 19, because those tasks stack many small calculations where CoT compounds rounding and slips. Layering self-consistency on top (PoT-SC) pushes GSM8K to 80.0% and FinQA to 68.1%. PoT also wins in zero-shot: GSM8K climbs from 40.5% (CoT) to 57.0%, and MultiArith from 79.3% to 92.2%.

When to use it (and when not)

Reach for PoT when:

• The answer is a number, table, or anything a program can compute exactly.
• The problem needs multi-step arithmetic, iteration, or equation-solving.
• You can run the generated code in a sandbox and trust the interpreter.

Skip it when:

• The task is open-ended language — summaries, classification, creative writing. There's nothing to compute.
• Questions are wildly diverse with no representative exemplars; PoT only reached 54% on AQuA's varied algebra, its weakest result.
• You can't safely execute generated code (see the security note below).

Model fit. PoT needs a model fluent in code. Code-tuned or modern general models (Codex, GPT-4-class, Claude) handle it; weak or tiny models that can't write correct Python will fail at the new task instead of the old one.

Escalation. Start with CoT. If accuracy stalls on quantitative tasks because of arithmetic errors, switch to PoT. If a few programs still disagree, add self-consistency voting over PoT samples.

Structure and components

A PoT prompt has three moving parts:

1. The question — the problem statement, often with a table or context.
2. A code instruction — "write Python," "store the answer in ans," sometimes a hint to use SymPy for symbolic work.
3. Few-shot exemplars (optional) — a handful of question-to-program pairs that demonstrate the style: meaningful variable names, no inline answer, result assigned to ans.

A standard exemplar looks like this:

Q: Carla downloads a 200 GB file at 2 GB/min, but 40% in, the
   download restarts. How long total, in minutes?
# Answer in Python, store result in `ans`.

total_gb = 200
rate = 2
gb_before_restart = total_gb * 0.40
time_before = gb_before_restart / rate
time_full = total_gb / rate
ans = time_before + time_full

Note what's not there: no "the answer is 220 minutes" line. The program is the reasoning, and the interpreter produces the number.

The core mechanism

The loop around the model is small. Generate a program, execute it, return ans; resample if it crashes.

import subprocess, textwrap

def pot_solve(question, generate, max_tries=3):
    for _ in range(max_tries):
        program = generate(POT_PROMPT + question)  # LLM writes Python
        try:
            scope = {}
            exec(program, {"__builtins__": SAFE_BUILTINS}, scope)
            return scope["ans"]                     # interpreter computes
        except Exception:
            continue                                # resample on error
    return None

For self-consistency, run pot_solve N times and return the most common ans. Because execution is deterministic, disagreement comes from the programs, not the math — exactly the signal you want to vote on.

Configuration

Implementation workflow

1. Write 2–8 few-shot exemplars in your domain: question, code instruction, program ending in ans.
2. Set temperature to 0 and a stop sequence so the model emits exactly one program.
3. Parse the generated code and run it in a sandboxed interpreter with a tight import whitelist.
4. Read ans; on a crash or missing variable, resample (and log the failing program).
5. If accuracy plateaus, add self-consistency: sample several programs and vote.

Do and don't

• Do name variables after real quantities — semantic binding is half the technique.
• Do sandbox execution; you're running model-written code.
• Do keep exemplars in the same style and domain as your real questions.
• Don't ask the model for the final number — let the interpreter compute it.
• Don't use PoT for tasks with no computable answer.
• Don't run generated code with unrestricted system access.

Debugging

Testing and how to prove it

Run PoT and CoT on the same held-out set and compare exact-match accuracy. Because PoT execution is deterministic, you can also separate reasoning errors (wrong program) from computation errors (none — the interpreter is exact), which CoT can't do.

def compare(dataset, cot_solve, pot_solve):
    cot = sum(cot_solve(q) == a for q, a in dataset)
    pot = sum(pot_solve(q) == a for q, a in dataset)
    n = len(dataset)
    return {"CoT": cot / n, "PoT": pot / n}

If PoT doesn't beat CoT on a quantitative set, your exemplars or sandbox are likely the problem — the technique itself reliably wins on numeric tasks.

Limitations

• Only as good as the program. PoT removes arithmetic errors but not reasoning errors; a wrongly-set-up equation still gives a wrong answer.
• Needs a computable answer. Tasks without a numeric or programmatic target gain nothing.
• Diverse problem sets hurt. Without representative exemplars, accuracy drops — AQuA's varied algebra capped PoT at about 54%.
• Requires a code-fluent model. Models that can't write correct Python fail outright.
• Execution overhead. Running code (and voting, for PoT-SC) adds latency and cost.

Risk and ethics

Ecosystem

PoT sits in the "reasoning-as-code" family alongside its close sibling PAL.

PoT composes well with self-consistency (vote over programs) and slots into agents and RAG pipelines as the "do the math reliably" step: retrieve the numbers, then let PoT compute on them. Tools like LangChain and DSPy ship code-execution and Python-REPL components that wire up the generate-execute loop.

Summary

• PoT separates reasoning from computation: the model writes a program, an interpreter runs it.
• It beats chain-of-thought by about 12% on average across eight math and finance benchmarks.
• Semantic variable names anchor the code to the problem and cut logic slips.
• It shines on math word problems, financial QA, and tabular reasoning — anything with an exact numeric answer.
• Skip it for open-ended language tasks or when you can't safely execute code.
• Always sandbox the generated program; you're running model-written code.
• Start with single-sample PoT; add self-consistency voting only for the hardest problems.