Choose few-shot examples that teach the boundary
Choose few-shot examples that explain a decision boundary, include near misses, and preserve a separate holdout for evaluating the prompt.
Few-shot prompting supplies examples of the task inside the model’s input. For a developer building a classifier or extractor, the useful question is which examples clarify the behavior that instructions alone leave ambiguous. Ten nearly identical successes can teach less than one carefully chosen boundary case.
Suppose a hypothetical support assistant routes messages to billing, account access, or clarification. The model can usually recognize an obvious invoice question. The harder cases mention two topics or use a keyword in an unrelated sense. Examples should make the intended routing policy visible in those cases.
Define the boundary before selecting examples
Assume the product routes by the user’s immediate blocker. “I cannot sign in to update my payment card” should go to account access because the user must regain access before changing the card. Another product could use a multi-label policy, but that would be a different task.
A compact example set can show the rule:
| Message | Label | Why this example belongs |
|---|---|---|
| I need a copy of last month’s invoice | billing | Clear positive case |
| I cannot sign in to change my card | account_access | Immediate blocker overrides a billing mention |
| The demo will be charged with showing the new dashboard | clarify | A keyword does not establish a support intent |
| Something is wrong with my account | clarify | Insufficient detail for a specific route |
The sentences are synthetic. The third is intentionally awkward but grammatically possible, illustrating that keyword presence alone is not the desired rule. A production set should include realistic near misses drawn from permitted, appropriately handled data rather than relying only on contrived examples.
The figure shows why the card message receives the access label even though it mentions a billing goal. The example’s value comes from the stated priority rule. If the rule changes, the example must change with it.
Keep examples consistent with the instructions
The GPT-3 paper evaluates language-model behavior with examples supplied in context. It establishes a research setting for few-shot use, not a guarantee that a particular number of examples will solve your application. The examples still need to express a coherent task.
If the instruction says to choose the immediate blocker but an example routes the card message to billing, the prompt contains a contradiction. A model may follow either signal depending on wording and context. Review examples as part of the specification, with the same care you would give test fixtures.
Use a stable output format across examples. If one label is account_access and another is login-help, decide whether they are synonyms or distinct classes. A machine consumer should not have to guess. Keep display text separate from the values expected by the application.
Choose coverage instead of repetition
Select examples that differ in the decision they teach. Include a straightforward positive, an ambiguity, a near miss, and a missing-information case when those distinctions matter. You do not need exactly four; the natural number follows the task’s boundaries and the context budget.
Long examples cost input tokens on every request and can crowd out the current user’s evidence. A shorter example that preserves the decisive fact may be preferable to a full conversation transcript. Remove names, timestamps, and narrative details that do not change the label, especially when they introduce unnecessary personal data.
The prompt engineering guide, retrieved on September 20, 2026, discusses using examples and clear context. For a multilingual feature, test whether examples in one language support the other languages you intend to serve. Do not assume translation preserves every ambiguity or cultural convention in the source message.
Protect the holdout set
Separate the examples shown to the model from the cases used to estimate performance. If a test message appears verbatim in the prompt, the evaluation does not measure how the system handles an unseen case. Near-duplicate paraphrases can create a similar problem when they preserve the same distinctive wording and answer.
Group related messages when splitting data. Variants of one source conversation should remain in the same partition. Otherwise, a prompt tuned on one variant can appear to generalize to another that is effectively the same case.
Keep a record of which evaluation failures influenced the example set. Those cases become development material after you use them to revise the prompt. They remain useful regression tests, but a separate untouched set is needed for the next estimate of generalization.
Test whether each example earns its cost
Run an ablation that removes one example or one type of example while keeping the model, task set, and output contract fixed. Compare paired results. If removing the near-miss example increases keyword-based misrouting, it may be teaching a useful boundary. If removing a redundant success changes nothing on the evaluation set, the shorter prompt may be sufficient for that workload.
Repeat runs when the generation setup is stochastic, and avoid treating one changed answer as a stable effect. Keep cost and latency alongside correctness if the examples materially increase input size. The relevant decision is whether the additional context improves the task enough to justify its operational cost.
Read a paired comparison rather than two totals
Suppose a synthetic evaluation compares prompt A with prompt B, which adds a near-miss example. Both prompts answer the same 100 held-out messages. Seventy messages are correct with either prompt, fifteen are correct only with B, five only with A, and ten remain wrong with both. These numbers illustrate the calculation; they are not observations from a deployed model.
Prompt A gets 75 messages right and B gets 85. Reporting only that difference conceals five regressions. Review the twenty disagreements by label and failure severity before deciding whether the new example helps the product. A regression that routes an access-blocked user to billing can matter even when several easy billing cases improve. The aggregate counts alone cannot tell you which kind of error changed.
def paired_summary(both_right, b_only, a_only, both_wrong):
counts = (both_right, b_only, a_only, both_wrong)
if any(type(n) is not int or n < 0 for n in counts):
raise ValueError("counts must be nonnegative integers")
n = sum(counts)
if n == 0:
raise ValueError("at least one paired case is required")
return {
"cases": n,
"a_correct": both_right + a_only,
"b_correct": both_right + b_only,
"net_correct": b_only - a_only,
"disagreements": b_only + a_only,
}
assert paired_summary(70, 15, 5, 10) == {
"cases": 100, "a_correct": 75, "b_correct": 85,
"net_correct": 10, "disagreements": 20,
}This helper checks arithmetic on already labeled pairs. It does not label responses, estimate a confidence interval or establish that the added example caused a stable improvement. The hypothetical accuracy difference is ten percentage points. Generalizing it requires a representative task sample and a treatment of sampling and generation variability. Retain case identifiers so that multiple runs and related conversations are not accidentally counted as independent messages.
Lilian Weng’s Prompt Engineering review, published March 15, 2023, discusses example selection and sensitivity to example ordering. Use that research overview to design an experiment for your model rather than assume an order that worked elsewhere will transfer. On the development set, compare a small predeclared collection of example orders while keeping the examples themselves fixed. Freeze the chosen order before running the untouched holdout; choosing it after looking at holdout results would turn that holdout into development data.
A useful exercise is to annotate the twenty synthetic disagreements with the three routing labels and a reason for failure. Then ask what additional information is needed to choose between A and B. You need the direction and cost of each change, not just a larger correct count. For a Vietnamese deployment, build a separate near-miss case around a real Vietnamese ambiguity: translating the English phrase “charged with” literally does not preserve its contrast with a billing charge.
For the support router, the first acceptance check is specific: the card message must follow the immediate-blocker policy, while the vague account message must request clarification. That is a stronger foundation than adding more examples simply because the prompt looks more complete.