Generated knowledge prompting: a complete guide

Models often know the answer but don't surface what they know when you ask directly. Generated Knowledge Prompting (GKP) fixes that by splitting the work into two steps: first ask the model to write out relevant facts, then answer the question with those facts in the prompt. Liu et al. (2022) showed this beats direct prompting by 7-10% zero-shot and 14-20% over few-shot on commonsense benchmarks, hitting state-of-the-art on NumerSense, CommonsenseQA 2.0, and QASC.

The core flip: instead of P(answer | question), you condition on P(answer | question, generated_knowledge). The model is its own knowledge base. No retrieval system, no external database, no fine-tuning.

See it work

Take a question where the wording is a trap. Direct prompting takes the bait; generating knowledge first does not.

Direct prompt:
Is golf about getting a higher score than your opponents?
Output: Yes. [Wrong — the framing nudges toward "higher = better."]

GKP — Stage 1: generate knowledge
Question: Is golf about getting a higher score than your opponents?
Knowledge:
- Golf is played on a course with a series of holes.
- The objective is to complete each hole in the fewest strokes.
- A lower total score is better than a higher one.

GKP — Stage 2: answer with knowledge
Knowledge: [the three facts above]
Question: Is golf about getting a higher score than your opponents?
Answer: No. In golf, the player with the lowest score wins.

The facts are illustrative, but the mechanism is real: surfacing "lower is better" before answering shifts the model off the trap.

The mental model

Think of GKP as thinking out loud before you commit. Asked a tricky question, a person recalls a few relevant facts first, then the answer becomes obvious. You're forcing the model to do the same — pull knowledge out of its parameters and put it where attention can use it.

Direct prompting reads the question. GKP makes the model brief itself first, then read the question.

How it works

1. Build a knowledge prompt. An instruction ("generate facts about the topic") plus 3-5 few-shot demonstrations of question-to-knowledge pairs.
2. Generate knowledge. The model emits M statements, typically 5-20. Use temperature above 0 if you want diverse samples for an ensemble.
3. Integrate. Concatenate the knowledge with the original question, clearly separated: Knowledge: [...] Question: [...].
4. Answer. Generate an answer conditioned on question plus knowledge. With multiple knowledge sets, answer each one.
5. Select. Pick the answer with the highest prediction probability, or take a majority vote across knowledge-augmented predictions.

The win comes from four mechanisms: knowledge generation casts a wider net than a direct question (surface-area expansion); the statements extend the model's effective working memory; attention can reference explicit facts instead of reconstructing them; and surfacing correct facts corrects the most common error mode — missing or misremembered knowledge.

Why it works

Ranked by impact on the improvement:

Where it shines

GKP helps most when the answer depends on world knowledge the question doesn't supply. In the original paper it set state-of-the-art on three benchmarks: NumerSense (numerical commonsense, e.g. "a person has ___ legs"), QASC (multi-hop scientific reasoning), and CommonsenseQA 2.0 (everyday reasoning). Generated knowledge even beat knowledge loosely retrieved from Wikipedia or Google by roughly 9% — though gold-standard, domain-specific knowledge bases still win when you have them.

Strong fits by task type:

• Commonsense reasoning — physical, social, temporal, and causal questions.
• Factual QA — trivia, scientific facts, history, geography, definitions.
• Numerical reasoning — typical quantities, order-of-magnitude, statistical common knowledge.
• Classification with world knowledge — sentiment, intent, and topic where context decides the label.
• Text generation — grounding blog posts, reports, and educational content in accurate background.

Domain results: QASC showed significant gains on multi-hop scientific reasoning (biology, chemistry, physics, earth science). Healthcare, legal, business, and finance work only for educational context — generated knowledge should never replace verified sources in those domains.

Performance plateaus once you include any knowledge: even a single statement helps, and gains flatten somewhere between 1 and 50 statements per question.

When to use it (and when not)

Reach for GKP when:

• The task needs commonsense or world knowledge.
• Direct prompting gives factually wrong answers.
• The model has the knowledge but doesn't activate it.
• Quality gains justify the extra latency and cost.

Skip it when:

• The question is a simple single-fact lookup (use direct prompting).
• The task needs multi-step logic (use Chain-of-Thought).
• You need recent or highly specialized info (use RAG).
• Latency must stay under 2 seconds, or the answer must be from verified facts.
• The model lacks knowledge of the domain.

Model fit: you need a model with substantial world knowledge. Minimum is GPT-3.5 or Claude Haiku tier; recommended is GPT-4, Claude 3+, Gemini Pro, or Llama 70B+. Small models (below 7B) without broad general knowledge aren't suitable.

Escalate when GKP isn't the right tool: to Chain-of-Thought for multi-step reasoning or math; to RAG for post-cutoff or authoritative-source needs; to a GKP + CoT hybrid for problems that need both knowledge and reasoning.

Structure and components

A GKP pipeline has two prompts. The knowledge generation prompt needs an instruction, few-shot demonstrations (optional but they help a lot), a format spec, and a question slot. The integration prompt needs a clearly-marked knowledge section, the original question, and an answer instruction. Required: the generation instruction, the question, and the knowledge-question integration. Optional: few-shot examples, answer-format spec, multiple samples, and probability-based selection.

Design principles that carry the technique: generate knowledge specifically relevant to the question, prioritize accuracy over quantity, keep statements clear and self-contained, cover different aspects across statements, and keep a clean separation between knowledge and question. Favor declarative, factual, definitional, and relational phrasings.

The prompt-format template, two-stage:

# Stage 1 — knowledge generation
Generate knowledge that would help answer the question.

Input: How many legs does a spider have?
Knowledge: Spiders are arachnids, not insects. Arachnids have 8 legs.
Spiders use their legs for walking, building webs, and catching prey.

Input: {new question}
Knowledge:

# Stage 2 — answer with knowledge
Use the following knowledge to answer the question.

Knowledge: {generated knowledge}
Question: {original question}
Answer:

Modify for scenarios: raise temperature and add definitional knowledge for ambiguous questions; add domain examples and terminology for specialized fields; for complex multi-part questions, generate knowledge per part and synthesize before answering; for time-sensitive questions, focus on stable general principles and flag possible staleness.

The core algorithm

The full pipeline with ensemble selection — generate several knowledge samples, answer each, keep the highest-confidence answer:

def generated_knowledge_prompting(question, num_samples=5):
    """Generate multiple knowledge sets, answer each, select best."""
    candidates = []
    for _ in range(num_samples):
        # Stage 1: generate knowledge (temperature > 0 for diversity)
        knowledge = generate_knowledge(question, temperature=0.7)

        # Stage 2: answer conditioned on question + knowledge,
        # returning an average-logprob confidence score
        answer, confidence = answer_with_knowledge(
            question, knowledge, temperature=0.3
        )
        candidates.append((answer, confidence, knowledge))

    # Stage 3: pick the answer with highest prediction probability
    best = max(candidates, key=lambda c: c[1])
    return {"answer": best[0], "knowledge": best[2], "candidates": candidates}

On the Anthropic API, the two stages are two messages:

import anthropic

client = anthropic.Anthropic()

def claude_gkp(question: str) -> dict:
    # Stage 1: knowledge generation
    knowledge = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=400,
        messages=[{"role": "user",
                   "content": f"Generate 3-5 relevant facts that would help "
                              f"answer this question.\n\nQuestion: {question}\n\nFacts:"}],
    ).content[0].text

    # Stage 2: answer with knowledge
    answer = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=300,
        messages=[{"role": "user",
                   "content": f"Based on the following knowledge, answer the question.\n\n"
                              f"Knowledge:\n{knowledge}\n\nQuestion: {question}\n\nAnswer:"}],
    ).content[0].text

    return {"knowledge": knowledge, "answer": answer}

A single-prompt variant folds both stages into one call ("first list 3-4 facts, then answer") — cheaper and faster, but the model can skip its own knowledge. Use it only when latency matters more than accuracy.

Configuration

Token budget for a full request runs about 500-1500 tokens across both stages: 200-500 for few-shot examples plus 100-300 output in generation, then 100-500 of knowledge plus the question and answer in integration. Each request is a minimum of 2 API calls, or M+1 for an ensemble.

Model-specific notes: GPT-4 and Gemini handle structured examples well (knowledge temp 0.6-0.7); Claude responds to conversational instructions and wants clear knowledge-question separation; open-source models (Llama, Mistral, 70B+) need more examples (5-7) and simpler knowledge formats with lower answer temperature (0.1-0.2).

Implementation workflow

1. Analyze the task (5-10 min) — does it benefit from knowledge, and does the model likely have it?
2. Design prompts (30-60 min) — write the generation prompt with 3-5 diverse examples and the integration prompt.
3. Test (30 min) — run 5-10 examples, check knowledge quality and accuracy versus baseline.
4. Iterate (30-60 min) — refine examples and instructions on failures.
5. Validate (30-60 min) — run 20-30 held-out examples, measure improvement, analyze failure modes.
6. Deploy — add monitoring for knowledge quality and a fallback to direct prompting.

Do: use clear instructions; include diverse few-shot examples; separate knowledge from question; validate knowledge quality; use an ensemble for important applications; monitor for hallucinations; always measure against a baseline first.

Don't: trust generated knowledge for high-stakes tasks without verification; use GKP when verified external sources exist; apply it to simple questions; assume knowledge is always correct; over-sample (diminishing returns); ignore latency and cost.

Debugging

Common mistakes: generating too much knowledge, never testing against a baseline, using GKP for reasoning tasks, ignoring hallucinations, and applying one knowledge format to every task.

Testing and optimization

Build a test set of 30-50 examples: ~50% common cases, ~30% edge cases, ~20% questions direct prompting gets wrong. Always compare GKP against a direct-prompting baseline, hold out the test set from development, and judge both answer accuracy and knowledge quality (accuracy, relevance, coverage, hallucination rate). Standard task metrics apply: exact match and F1 for QA, accuracy for classification, BLEU/ROUGE plus human eval for generation.

A baseline comparison is the one snippet worth keeping:

def evaluate_gkp(test_set, baseline_fn, gkp_fn):
    baseline_correct = sum(
        evaluate_answer(baseline_fn(ex["question"]), ex["answer"])
        for ex in test_set
    )
    gkp_correct = sum(
        evaluate_answer(gkp_fn(ex["question"])["answer"], ex["answer"])
        for ex in test_set
    )
    n = len(test_set)
    return {
        "baseline": baseline_correct / n,
        "gkp": gkp_correct / n,
        "improvement": (gkp_correct - baseline_correct) / n,
    }

Optimize tokens by asking for concise, bullet-point knowledge (saves 20-30%) and trimming examples (saves 10-20%). Cache knowledge for repeated similar queries. The cost-performance trade-off:

When experimenting, run each configuration 3-5 times, use paired comparisons on the same questions, and test for statistical significance. Stop iterating when accuracy hits target, when improvements fall below 2% for two iterations, or at five iterations — whichever comes first.

Limitations

• Hallucination propagation (the primary risk). A wrong fact in Stage 1 is treated as true in Stage 2, producing confidently wrong answers that are harder to catch than direct errors. Inherent to using parametric knowledge without external validation. Mitigate with lower temperature, uncertainty prompts, knowledge filtering, and external verification for critical facts.
• Recency. Knowledge is frozen at the training cutoff — wrong for recent events and evolving topics. Use RAG for time-sensitive queries.
• Domain gaps. Knowledge is uneven; specialized domains see more hallucination. Use domain retrieval or fine-tuned models there.
• Computational overhead. The two-stage design roughly doubles cost and adds 1.5-2x latency. Inherent; mitigate with the single-prompt variant, caching, and batching.
• No reasoning. GKP surfaces facts, not logic chains — it won't help with math or multi-step deduction. Pair with CoT.
• Quality variability. Some queries benefit greatly, others not at all, and it's hard to predict which. Use ensembles, filtering, and A/B testing.

Edge cases worth handling: questions with no relevant knowledge (fall back to direct prompting), contradictory generated facts (vote or filter), unfamiliar-topic hallucination (request confidence indicators), very long questions (decompose), and recent-info questions (detect and route to retrieval). Degrade gracefully — check knowledge relevance before integrating, and fall back to a direct answer on empty, low-quality, or below-50-character knowledge or on an API error.

Advanced techniques

Beyond flat fact lists, you can structure the generation: multi-perspective (scientific, historical, environmental, economic views of one question), hierarchical (general to technical levels of detail), contrastive (distinguishing similar concepts like viruses vs bacteria), and conditional (knowledge for different scenarios). JSON or categorized output (definitions, facts, relationships, context) makes integration cleaner.

For quality control, add a verification step that reviews each generated fact for accuracy and relevance before answering, ask the model for per-fact confidence levels, or run a self-consistency check that keeps only facts appearing across multiple generations. Interaction patterns extend the idea: conversational GKP accumulates knowledge across turns, iterative GKP refines knowledge when an answer looks incomplete, and chained GKP generates knowledge per domain for cross-domain questions.

Risk and ethics

Failure modes to watch: hallucinated knowledge leading to a confidently wrong answer (verify against sources), irrelevant knowledge giving no improvement (measure against baseline), and biased knowledge propagating bias (audit, balance few-shot examples). A single hallucinated fact can become the premise for otherwise-sound reasoning, which is why cascading failures are the real danger. Validate input against prompt injection and filter output for high-stakes use.

The capability cuts both ways: GKP shows models can leverage their own knowledge to improve, which opens self-improving knowledge systems, hybrid generation-plus-retrieval pipelines, and compositional knowledge reuse.

Ecosystem and integration

GKP is supported by LangChain (prompt templates, chain composition for the two-stage pipeline), DSPy (signature-based prompts, automated few-shot optimization), and LlamaIndex (hybrid GKP-plus-retrieval over document stores). The original paper's code is at github.com/liujch1998/GKP, with tutorials on the Prompt Engineering Guide and Learn Prompting.

How it relates to neighbors:

Versus Chain-of-Thought: GKP generates facts, CoT generates reasoning — knowledge-dependent versus logic-dependent tasks. Self-Ask is a related approach generating intermediate questions, better for multi-hop reasoning. Analogical Prompting extends GKP by generating examples and analogies. GKP itself built on earlier knowledge-enhanced work (ConceptNet, WordNet, custom retrieval, task-specific fine-tuning) but dropped those dependencies; follow-ups include Knowledge-Augmented Chain-of-Thought (2023) and Recitation-Augmented Generation.

Useful hybrids: GKP + RAG combines generated and retrieved knowledge (or uses RAG as a fallback when generated knowledge looks unreliable); GKP + CoT generates knowledge, then reasons through it step by step. To adopt GKP, find where direct prompting fails on knowledge-dependent questions, test on that subset, measure the gain, and expand gradually while keeping direct prompting for simple queries. Transition to RAG for the queries where generated knowledge proves unreliable.

Future directions

Open frontiers: automated fact-checking of generated knowledge against trusted sources; adaptive generation that scales knowledge to question complexity; multi-modal knowledge from images, tables, and code; knowledge-graph integration producing graph triples; and theoretical work on why and when self-generated knowledge helps. The trajectory points toward hybrid generation-plus-retrieval systems, built-in verification, and adaptive approaches that apply GKP only when it pays off.

Summary

• GKP is two stages: generate relevant knowledge, then answer with it in the prompt — conditioning the answer on (question, generated_knowledge).
• The numbers: 7-10% over zero-shot, 14-20% over few-shot, ~9% over loosely retrieved knowledge; state-of-the-art on NumerSense, CommonsenseQA 2.0, and QASC (Liu et al., 2022).
• Use it for commonsense, factual, and numerical questions where the model knows but doesn't activate the knowledge.
• Skip it for simple lookups (direct), multi-step logic (CoT), recent or specialized facts (RAG), and latency-critical paths.
• Costs roughly 2x tokens and 1.5-2x latency; ensembles of 3-5 samples trade more cost for more accuracy.
• The main risk is hallucination propagation — a wrong fact becomes a confident wrong answer, so verify critical facts and never replace authoritative sources.