Self-generated in-context learning (SG-ICL): a complete guide

Few-shot prompting works, but it has a tax: you need a labeled dataset to draw the examples from. SG-ICL pays that tax differently. It asks the model to write its own demonstrations first, then feeds them back as the few-shot examples. In the original paper, those self-written examples were worth roughly 0.6 of a real gold training sample each, and they beat zero-shot with lower variance (Kim et al., 2022).

See it work

Say you're classifying a borderline movie review and you have no labeled examples on hand.

Zero-shot, the model gets one shot with no pattern to anchor on:

Review: "It tries hard, and you can tell, but the payoff never lands."
Sentiment:
→ Positive        (wrong — it read "tries hard" and stopped there)

SG-ICL first asks the model to invent one example per label, then classifies using them:

Step 1 — generate (one prompt per label):
  "Write a negative movie review: ..."  → "Flat pacing and a story that goes nowhere."
  "Write a positive movie review: ..."  → "Sharp, funny, and over before you want it to end."

Step 2 — classify with the generated demos as the few-shot context:
  Review: "Flat pacing and a story that goes nowhere."   Sentiment: negative
  Review: "Sharp, funny, and over before you want it to end."  Sentiment: positive
  Review: "It tries hard, and you can tell, but the payoff never lands."  Sentiment:
  → Negative        (right — the demos gave it the label boundary)

Same model, same test input, no dataset. The only thing that changed is that the model saw two examples of what each label looks like — examples it wrote itself.

The mental model

Think of a student walking into an exam with no textbook. Instead of guessing cold, they first scribble two worked examples from memory, one for each answer type, then solve the real question by analogy to their own scribbles.

SG-ICL turns the model into its own example bank: it manufactures the demonstrations it wishes it had, then learns in-context from them.

That's the whole flip. Standard few-shot says "find good examples." SG-ICL says "the model already knows what a good example looks like — ask it."

How it works

SG-ICL is a two-pass process. The first pass generates demonstrations; the second pass does the actual prediction, conditioned on those demonstrations.

The steps:

1. Take the test input and the label set. For a sentiment task that's the review plus {positive, negative}. The label set has to be known up front — SG-ICL is built for classification.
2. Generate one demonstration per label. For each label, prompt the model to produce a synthetic input that would carry that label. The generation can be conditioned on the test input's domain, so the demos look like the kind of text you're actually classifying.
3. Pair each generated input with its target label. You now have a small set of (generated_text, label) pairs — a synthetic, balanced demonstration set with no dataset behind it.
4. Build the few-shot prompt. Lay the generated pairs out as in-context examples, then append the real test input with an empty label slot.
5. Predict. The model classifies the test input the same way it would with hand-picked few-shot examples — except the examples came from itself.

Why it works

The gains come from a few stacked effects, roughly in order of impact:

Where it shines

SG-ICL was built and tested on text classification, and that's where it's strongest. The paper ran it on four tasks: sentiment classification (SST-2 and SST-5) and natural language inference (CB and RTE), using GPT-J (6B) as the demonstration generator.

The headline findings:

• It beats zero-shot, clearly. Across the four tasks, self-generated demonstrations lifted accuracy over the no-example baseline.
• Each generated example is worth about 0.6 gold samples. That's the paper's framing: a self-written demo carries roughly 60% of the signal of a real, human-labeled training example — for free.
• Lower variance than random demos. Pulling random examples from a training set makes accuracy jump around; the self-generated set is more consistent run to run.

Good fits beyond the benchmarks: intent detection, topic tagging, toxicity or spam flags, and any classification where you have a fixed, known label set but no curated example bank to draw from.

When to use it (and when not)

Reach for SG-ICL when:

• You have a classification task with a small, fixed label set and no labeled data.
• Bootstrapping a new task fast, before any annotation effort exists.
• You want more stability than randomly sampled few-shot examples give you.
• The subject matter sits comfortably inside the model's training knowledge.

Skip it when:

• You already have good labeled examples — real demos beat synthetic ones, and retrieval methods like KNN or vote-k will win.
• The task is generative, open-ended, or has an unbounded output space rather than discrete labels.
• The domain is niche enough that the model can't write a faithful example (it'll hallucinate a wrong-looking demo and anchor on it).

Model fit. SG-ICL leans on the model being a competent demonstration generator, so it favors larger, capable models — the original work used GPT-J (6B), and stronger modern models generate cleaner demos. A small model that writes a mislabeled or off-style demo will poison its own context.

Escalation path. If SG-ICL plateaus, the next move is to introduce real data: even a handful of gold examples to mix in, or a retrieval step (KNN over a labeled pool) to pick relevant demos. SG-ICL is the bridge you use until you have that data.

Structure and components

A SG-ICL setup has three moving parts:

• A label set. The discrete set of classes. This drives the generation loop — one demo per label.
• A generation template. The prompt that asks the model to write a synthetic example for a given label, optionally conditioned on the test input's style.
• An inference template. The standard few-shot layout that holds the generated demos plus the real query.

The generation template, in text form:

Write a {label} example for the following task.
Task: classify the sentiment of a movie review.
{label} review:

The inference template, once the demos are filled in:

Review: {generated_demo_for_label_1}
Sentiment: {label_1}

Review: {generated_demo_for_label_2}
Sentiment: {label_2}

Review: {test_input}
Sentiment:

The core algorithm

The whole technique is a generate-then-classify loop. In pseudo-Python:

def sg_icl(test_input, labels, generate, classify):
    # Pass 1: model writes one demonstration per label.
    demos = []
    for label in labels:
        gen_prompt = (
            f"Write a {label} example for this task.\n"
            f"Style it like: {test_input}\n"
            f"{label} example:"
        )
        synthetic_text = generate(gen_prompt)
        demos.append((synthetic_text, label))

    # Pass 2: classify the real input using the self-made demos.
    few_shot = "".join(
        f"Input: {text}\nLabel: {label}\n\n" for text, label in demos
    )
    final_prompt = f"{few_shot}Input: {test_input}\nLabel:"
    return classify(final_prompt, labels)

Two knobs matter. Conditioning generation on the test input (the Style it like: line) keeps demos on-domain. And you can generate more than one demo per label and keep the most distinct ones, trading cost for a richer context.

Configuration

Implementation workflow

1. Define the label set and write a generation template per task, not per label — the label slots in.
2. Loop the labels, generate one demo each, and sanity-check that each demo actually reads like its label.
3. Assemble the inference prompt with demos balanced and shuffled.
4. Run the prediction at temperature 0.
5. Spot-check on a small held-out set against a zero-shot baseline to confirm the lift is real.

Do:

• Condition generation on the test input so demos match domain and length.
• Keep the demo set label-balanced — one per class is the safe default.
• Filter or regenerate any demo that comes back blank, off-topic, or self-contradictory.

Don't:

• Don't use SG-ICL when you already have good real examples; they're strictly better.
• Don't trust a single demo per label on an ambiguous task without spot-checking.
• Don't let generation temperature run so high the demos stop matching their labels.

Debugging

Testing and proving it works

The honest test is a head-to-head against zero-shot on the same inputs:

def compare(test_set, labels, generate, classify):
    zero = sum(
        classify(f"Input: {x}\nLabel:", labels) == y
        for x, y in test_set
    )
    sg = sum(
        sg_icl(x, labels, generate, classify) == y
        for x, y in test_set
    )
    n = len(test_set)
    return {"zero_shot_acc": zero / n, "sg_icl_acc": sg / n}

Run it a few times. SG-ICL should not only score higher on average but also show a tighter spread across runs than randomly sampled few-shot — that low variance is one of the technique's selling points.

Limitations

• Classification-shaped only. SG-ICL needs a discrete, known label set to loop over. It doesn't map cleanly onto open-ended generation or free-form reasoning.
• It doesn't beat real data. Self-generated demos are worth about 0.6 of a gold sample, not 1.0. Methods that select real examples (KNN, vote-k) outperform SG-ICL when a labeled pool exists.
• Generator quality is a ceiling. If the model can't write a faithful example for a class, the demo is wrong and the model anchors on its own mistake. Weak models or niche domains break this.
• Cost scales with labels. Every label adds a generation call. Fine-grained classification gets expensive fast.

Advanced variations

The same generate-then-use idea spawned a small family of methods. Self-ICL (Chen et al., 2023) generates pseudo-inputs and pseudo-labels for zero-shot tasks beyond classification. Auto-ICL pushes toward fully unsupervised in-context learning. You can also generate several demos per label and keep the most diverse, or seed generation with a single real example to pin down the format — a cheap hybrid that often closes the gap to fully supervised few-shot.

How it relates to its neighbors

SG-ICL is easy to confuse with two siblings, so keep the line sharp:

• Versus standard few-shot ICL: few-shot pulls demonstrations from an external dataset; SG-ICL has the model write them. Both then prompt identically. SG-ICL is what you use when the dataset doesn't exist.
• Versus analogical prompting: analogical prompting asks the model to recall related solved problems to guide multi-step reasoning, mainly for math and code. SG-ICL fabricates labeled classification examples to set a decision boundary. Different goal, different task shape.

Summary

• SG-ICL has the model write its own few-shot demonstrations — one per label — then classifies the real input using them, so you need no labeled dataset.
• It's a two-pass process: generate demos per label, then predict in-context.
• In the original paper (Kim et al., 2022), self-generated demos beat zero-shot, ran with lower variance than random samples, and were each worth roughly 0.6 gold training samples on SST-2, SST-5, CB, and RTE with GPT-J (6B).
• Reach for it on classification tasks with a fixed label set and no data; skip it when you already have good real examples or the task is open-ended.
• It doesn't beat real labeled data and its cost scales with the number of labels — treat it as the bridge you use until you can retrieve or curate actual examples.