Chapter 2. Examples and structure

Learning objectives

• Explain how examples teach both intended rules and accidental patterns, and identify and remove those patterns.
• Assemble few-shot examples based on relevance, diversity and format consistency, keeping examples separate from evaluation data.
• Use XML tags to mark boundaries between instructions, context, examples and input.
• Choose a format for the recipient—human, system or voice—and define a data dictionary and missing-value conventions.
• Choose instructions, examples or structured outputs for machine-readable output, and check format compliance separately from content accuracy.
• Migrate prompts that relied on prefill so they work with newer models.

Introduction

A mail-order retailer built a prompt to classify product inquiries into four categories: shipping, exchanges, payment and other. The coordinator added four examples, all of which happened to be polite sentences beginning with “Hello.” Once it was in use, brief inquiries without greetings were disproportionately classified as “other,” and dozens of urgent inquiries went unattended for more than a day. The coordinator found the cause a week after the person handling “other” complained about the increased workload. Along with the intended classification criteria, the model had learned an unintended rule: only polite sentences count as proper inquiries. The bias came from the selected examples, rather than the written instructions. Examples are among the strongest instructions and the hardest to control.

This chapter starts with the criteria for good examples, diversity design, boundary cases, a process for reducing bias, and separating examples from evaluation data. It then covers tags separating instructions and material, recipient-appropriate formats, data dictionaries and missing values, the guarantees and limits of structured outputs, alternatives to prefill, and evidence-field design. The deeper discussion covers examples that show methods to models with thinking enabled and formatting issues across display environments.

Key concepts

Examples are the sample garments you hand a tailor

Handing a tailor a few sample shirts is far more precise than saying “a neat shirt.” But if every sample is a white short-sleeved shirt, the tailor may conclude that the customer only wants white short sleeves. Few-shot examples—showing a few pairs of inputs and desired outputs—work the same way. Anthropic’s guide identifies examples as one of the most reliable ways to steer output format, tone and structure. Yet the model also learns unintended patterns hidden in them.

Before adding examples, divide “what this set teaches” into two columns to expose the risks.

If items in the right column are not well mixed across examples, the model may treat them as rules.

What makes a good example

Vendors emphasize different points about quantity. Anthropic recommends three to five examples, adding that you can ask the model to evaluate their relevance and diversity or generate more. Gemini recommends always including examples but warns that too many can cause responses to overfit them. OpenAI’s reasoning-model best practices recommend trying zero-shot first, without examples. Google’s documentation also describes tasks where hundreds of examples in a long context, called many-shot prompting, performed comparably to fine-tuned models. The practical rule is “varied content, identical format, quantity determined by experiment.” Add examples gradually and measure on the same evaluation set. Stop when scores plateau or outputs start resembling example sentences.

A table for designing diversity

“Diverse” is another uncheckable expression of the kind discussed in Chapter 1. Break diversity into dimensions and fill in a table when selecting examples.

Empty cells identify the next examples to find. Search real logs for matching inputs, removing personal information before including them.

Boundary cases and contrastive pairs

The model already gets easy examples right. Ambiguous examples teach decision criteria. Is “The box is dented, but I have not opened the product yet” a shipping problem or an exchange request? Labeling such a boundary case shows the model where the categories divide.

Contrastive pairs make the boundary clearer: two nearly identical inputs receive different labels because of one small difference.

<examples>
  <example>
    <input>The heel has come off one of the shoes I received. Please exchange them for the same size.</input>
    <label>Exchange</label>
  </example>
  <example>
    <input>The heel has come off one of the shoes I received. I would just like my money back.</input>
    <label>Refund</label>
  </example>
</examples>

Use incorrect outputs as examples cautiously. Showing a wrong format at length leaves it strongly represented in the context and may encourage imitation. If a negative example is necessary, keep it short, explicitly mark it as something not to do, state why, and immediately follow it with a correct example. The default is to show the desired result.

A process for reducing example bias

Example bias occurs when an unbalanced example set skews model judgments. Three common forms are overpredicting a label overrepresented in examples, being drawn toward example order or the final example, and copying example wording. Diagnose and reduce it as follows.

1. Measure the symptom. Compare accuracy per label and the output-label distribution with the real distribution. A label appearing too often indicates bias.
2. Check examples against the diversity table. Look for overrepresentation of the favored label and imbalance in style or length.
3. Change one thing at a time. Balance labels, shuffle order or vary style, then measure again on the same evaluation set.
4. Check copying. For generation tasks, count sentence overlap between outputs and examples. If there is overlap, vary example wording and explain that examples demonstrate structure and tone, not sentences to copy.
5. Do not compensate with more rules. As Anthropic’s context engineering article advises, select a representative set of examples before adding long exception lists.

Keep examples separate from evaluation data

Examples and evaluation sets often come from the same logs. If an input used as an example remains in the evaluation set, the model is answering a question whose answer it has already seen, inflating the score. First set aside and lock evaluation inputs. Then choose examples from the remaining inputs to fill the diversity table. Whenever examples change, check that they do not overlap evaluation inputs. Chapter 5 revisits this as separating improvement and confirmation sets.

Draw boundaries with XML tags

When instructions, reference material, examples and user input are mixed in a prompt, the model may mistake reference sentences for instructions. Anthropic’s guide explains that wrapping content types in tags such as <instructions>, <context> and <input> reduces confusion. Tags act like colored index tabs in a binder.

<instructions>
Read the quotation-request email and extract the requested items and desired delivery date.
</instructions>
<price_policy>
Quotations remain valid for 30 days from the date of issue.
</price_policy>
<email>
{{CUSTOMER_EMAIL}}
</email>

The rules for using tags are simple.

• There are no fixed tag names. Choose names that describe their contents and use them consistently throughout.
• Nest hierarchical content. For multiple documents, put <document index="1"> inside <documents>. As in OpenAI’s guide, XML attributes can carry metadata referenced by instructions.
• Tags only mark boundaries. Specify output format separately, and do not treat tags as a security barrier. If external material could contain a string such as </email> that breaks the boundary, wrapping it in JSON is safer, as Chapter 3 explains.
• Both OpenAI and Anthropic also mention Markdown headings as separators. Use one consistent approach within a prompt.

Choose a format for the recipient

The first question when choosing a format is “Who receives this output?”

If you need human-readable explanations and machine-readable data together, place explanations in defined tags and data in a separate field, or separate the calls. Explanation inserted among data can break a parser.

Data dictionaries and missing values

Field names without defined meanings get filled differently by different people and on different requests. Create a data dictionary before a schema.

Avoid treating every kind of absence as the same. Information missing from the source, an inapplicable item, and a judgment unsupported by sufficient evidence have different consequences for subsequent work. Define different values for all three, in both instructions and schema descriptions. A required string field encourages the model to fill in a plausible value even when information is absent. Allow null in the schema for fields that may have no value.

Structured outputs: enforcing format

There are three levels of output-format control.

OpenAI’s documentation distinguishes JSON mode from structured outputs. Both produce valid JSON, but only structured outputs guarantee schema compliance, so the recommendation is to use them when possible. Requirements vary somewhat by vendor.

For classification, define allowed labels with enum. Anthropic recommends tools or structured outputs with enum fields, but notes that enum-value capitalization is not guaranteed. Compare case-insensitively and avoid values distinguished only by capitalization.

Format compliance is not content accuracy

Structured outputs guarantee shape, not correctness. OpenAI explicitly notes that structured outputs can contain mistakes and recommends refining instructions, adding examples or splitting tasks into simpler subtasks. Check machine-read output in four stages.

1. Syntax: Does it parse as JSON?
2. Schema: Are fields, types and enums correct? Use code here to check unsupported constraints, such as maximum values and length limits, that could not be enforced.
3. Business rules: Is the delivery date after today? Does the item code exist in the code table?
4. Source comparison: Does the value actually appear in the source? Have a person check a sample or search the source strings.

How prefill works and what replaces it

Prefill supplies the beginning of an assistant message and asks the model to continue. For example, filling the last assistant message with { makes the model continue the rest of the JSON. This fixed the opening by exploiting the model’s next-character prediction behavior.

According to Anthropic’s documentation, Claude 4.6 and later models, and Claude Mythos Preview, no longer support prefill on the last assistant turn. Such requests return a 400 error. Earlier models still support it, and assistant messages in the middle of a conversation are unaffected. The documentation explains that improved instruction following makes prefill unnecessary for most uses and gives migration paths by purpose.

Prefill is the first item to check when changing models. The resulting errors are easy to locate, but each former use needs its own replacement, as the table shows.

Request short explanations in evidence fields

Often you want evidence alongside a judgment. Design two things. The first is field order. Because the model writes sequentially, putting evidence before judgment encourages judgment based on that evidence. OpenAI structured outputs follow schema key order; Anthropic emits required properties first. To preserve order, make all properties required or account for reordering when parsing.

The second is what the evidence field requests. On newer Claude models with thinking enabled, asking for step-by-step internal reasoning in fields such as reasoning, thinking or trace may produce a reasoning_extraction refusal. The official documentation advises requesting a short explanation or evidence supporting the result instead. Use descriptive field names such as evidence and explanation. Chapter 4 returns to this topic.

Going deeper

Show methods to models with thinking enabled

Examples also help models with thinking enabled. Anthropic explains that worked examples shape how a model approaches similar problems within its thinking block, and recommends presenting a problem, the method to apply and the expected answer in each example. Input-answer pairs only teach answer shape; examples including a method teach procedures the model cannot know from general knowledge, such as company-specific calculation rules.

Even when showing a method, instruct the system to extract the final answer only from specified fields or tags. This prevents explanations mixed with answers from breaking parsing.

The display environment determines formatting

When default model formatting does not suit the display, users see the literal symbols: asterisks and hash signs on a screen that cannot render Markdown, for example. Anthropic states that newer models use LaTeX by default for mathematical expressions. Its example instruction for plain text requests formulas using only symbols such as /, * and ^ instead of LaTeX or MathJax.

Chapter 1’s principles apply to formatting problems too.

1. State the desired format instead of only a prohibition: “smooth paragraphs” rather than “no Markdown.”
2. Add the reason: “This screen cannot render formatting, so asterisks appear as literal characters.”
3. Match prompt formatting to the desired output. Remove headings and bullets from the prompt if you want plain text.
4. Use post-processing as a last resort. Regex-based symbol removal can also remove intended symbols, such as multiplication signs in formulas.

At work

Sunrise Clinic: turning intake notes into record fields

Sunrise Clinic, a local clinic, wants to extract symptoms, time of onset and current medications from nurses’ intake notes and pre-fill electronic medical record fields. The first prompt is “Extract the information from the following note as JSON,” plus two examples. In live use, onset sometimes turns into start_date, and when the patient takes no medication the field disappears. Notes saying only “a few days ago” also produce invented specific dates.

Seo, a staff member on the information team, identifies three causes. First, both examples contain medications and clear onset times, so they show neither absent nor ambiguous information. Second, instructions and examples alone cannot guarantee field names or presence. Third, onset is a required date field, leaving no way to omit its value.

Seo first builds a data dictionary. onset means the symptom start time reported by the patient. If a date appears, use it. If only a relative expression such as “three days ago” appears, copy it unchanged into onset_text and set onset to null. Medication distinguishes “explicitly none” (empty array) from “not mentioned in the note” (null), because the latter requires the nurse to ask in the next interview.

Next, Seo expands the examples to four using the diversity table: explicitly no medication, medication unmentioned, relative onset time, and multiple symptoms. All use the same tags and field order. A <memo> tag separates the original note from instructions. Because a system reads record input directly, structured outputs enforce the schema, with onset allowing either a date or null.

A teammate suggests using the intake date when onset cannot be extracted. Seo rejects this: intake date is not symptom onset, and a plausible filled-in value might go unchecked by the doctor. Finally, the system checks stop reasons first and fills records only after normal completion. A nurse compares ten daily samples against the source. Field changes and omissions disappear, and samples contain no invented dates. The key is dividing responsibilities: examples teach decision boundaries, the schema enforces shape, and the data dictionary defines meaning.

Corner Office: correcting bias in facilities-complaint classification

Fictional coworking operator Corner Office classifies app-submitted tenant complaints into five labels—equipment failure, cleanliness, noise, access/security and other—and automatically routes them to teams. After two pilot weeks, team leads are unhappy. The equipment team receives unrelated complaints, and noise complaints repeatedly end up in “other.”

Cha, an operations planner, compares model classifications with human reviewers’ revised labels for 400 complaints. The model chooses equipment failure much more often than warranted. The diversity table exposes the cause: three of five examples are equipment failures, all five are long, polite sentences, and the final example is also equipment failure. More than half the real complaints, however, are short messages such as “Too noisy next to the third-floor meeting room.”

Before revising, Cha locks 150 of the 400 complaints for evaluation. From the remaining 250, Cha selects examples to fill diversity gaps: at least one per label, short and long text, and two boundary cases. “The air conditioner is so loud I cannot make calls” resembles noise but is equipment failure because the cause is equipment. “Someone keeps talking loudly on the phone in the corridor” is noise. Both include a label and a short explanation: an equipment cause means equipment failure. Example order is shuffled so identical labels are not consecutive.

Because the routing system reads output directly, structured outputs enforce label as an enum. An evidence field briefly quotes the complaint wording supporting the decision for the team lead. No field requests internal reasoning.

At review, someone proposes exception rules for each label. Cha chooses to evaluate the example-only revision first: changing rules and examples together would obscure which change drove improvement. On the locked 150 complaints, noise accuracy rises substantially and equipment-failure bias falls. Remaining errors mostly involve the boundary between “other” and another label. Cha records that category as the next revision target.

Common mistakes

• The more examples, the better. More of the same type strengthens bias; too many can overfit. Determine quantity by experiment.
• Good examples make evaluation trustworthy. Evaluation inputs reused as examples inflate scores. Set evaluation inputs aside first.
• Tagging input structures the output. Specify output format separately; enforce machine-read output with a schema.
• JSON mode guarantees fields too. It guarantees syntax only. Structured outputs guarantee fields and types.
• Schema compliance means values are correct. Structured outputs guarantee shape, not content. Check business rules and source correspondence separately.
• Required fields eliminate missing information. They encourage plausible inventions when information is absent. Allow null for potentially absent values.
• Prefill works everywhere. Newer Claude models return a 400 error. Migrate each use to a suitable alternative.

Summary at a glance

Check the official docs

• Anthropic — Prompting best practices: Read “Use examples effectively,” “Structure prompts with XML tags,” “LaTeX output” and “Migrating away from prefilled responses.”
• Google — Prompt design strategies: Read about the number and formatting consistency of few-shot examples.
• OpenAI — Structured model outputs: Check differences from JSON mode, required fields, additionalProperties rules and refusal handling.
• Anthropic — Structured outputs: Check output_config.format, strict tools, unsupported schema features and property order.
• Anthropic — Refusals and fallback: Read about reasoning_extraction refusals and appropriate ways to request evidence.

Self-check

2-1. Give two examples each of what examples “intend to teach” and what they “might accidentally teach.”

2-2. You have only one log as the source of both examples and evaluation data. Describe the correct work sequence and explain the problem caused by reversing it.

2-3. Rewrite “Draft a reply to the event inquiry below. In rain, the event moves to the indoor gym. Inquiry: …” with tags separating instructions, policy and input.

2-4. You export the same sales data for (a) an executive reporting screen, (b) an in-store announcement and (c) inventory-system loading. Give a suitable format and one caution for each.

2-5. A travel-expense field extracts a string array of accompanying travelers’ names. Define and justify values for three cases: the document explicitly says no one accompanied the traveler, it says nothing about companions, and the writing is smudged beyond reading.

2-6. Give two cases where a response may not match the schema despite structured outputs, and state what to check before parsing.

2-7. Three prompts prefill the final assistant message to (a) output only one label, (b) remove opening greetings and (c) continue interrupted long text. Give a replacement for each when migrating to newer Claude models.

Hands-on tasks