Chapter 2. Requirements and Success Criteria

Learning objectives

• Turn vague requirements like "it works well" into verifiable success criteria with data, metrics, threshold values, and measurement methods.
• Set criteria across multiple dimensions together, including consistency, tone, personal information handling, latency, and cost, in addition to accuracy.
• Define the inputs accepted and the outputs returned by a feature through input and output contracts, and specify handling rules for out-of-scope inputs and format errors.
• Select design options (model, output length, streaming, batch processing) that balance quality against latency and cost.
• Write user-facing failure experiences as requirements, and distinguish launch-blocking criteria from improvement targets.

Introduction

In the requirements section of a product specification, it was written: "AI summaries must be accurate, natural, and respond quickly." One month before launch, the development team asked, "Is 91% accuracy considered accurate?", the sales team said, "What the client wants is 99%", and the designer asked, "How many seconds is 'quickly'?" Nobody was wrong. The requirements were simply written in unverifiable terms.

Because AI features operate probabilistically, they cannot promise to "never make mistakes." Instead, they promise "at least this proportion when measured with this method on this data." Claude documentation explains that successful LLM applications begin with clearly defining success criteria, and that the core is an iterative loop of designing evaluations to measure those criteria. This chapter addresses the front end of that loop: defining what counts as success.

First, you will learn the conditions and multiple dimensions of verifiable criteria, and practice the procedure for turning vague requirements into numbers. Next, you will cover input and output contracts and the trade-offs between quality, latency, and cost, followed by how to write user-facing failure experiences as requirements. Finally, you will organize all criteria into launch-blocking criteria and improvement targets, assembling them into a single requirements document.

Key concepts

How requirements for AI features are different

Requirements for traditional features have clear truth values, such as "When the button is clicked, navigate to the payment screen." Requirements for AI features have the characteristic of being mostly correct and partly incorrect, such as "Correctly classify inquiry types." Therefore, AI requirements need three additional things:

1. Criteria written as proportions. Not "classifies correctly," but "achieves at least 95% accuracy across an evaluation set of 1,000 cases."
2. Measurement methods. Who judges, compared against what, and how.
3. Commitments when incorrect. How the incorrect 5% appears to the user and how it is handled.

By analogy, traditional requirements are like blueprints, while AI requirements are closer to quality inspection specifications. Since a factory cannot reduce defects to zero, it defines an upper bound on defect rates, inspection methods, and handling procedures for defective products together.

Verifiable success criteria

Claude documentation lists four conditions for good success criteria: they must be specific, measurable, achievable, and relevant. Instead of "good performance," write specifically such as "accurate sentiment classification," measure with numbers or well-defined scales, set targets based on industry standards, prior experimental results, or expert knowledge, and align with user needs.

Examples from the same documentation clearly show the difference:

A good criterion must include four elements: what (metric), where (data), how much (threshold value), and how (measurement method). If even one is missing, teams will debate the numbers again in meetings. In addition, noting what the number is compared against (baseline) explains why that threshold value was chosen. The sentiment classification example in the table above includes this baseline (a 5% improvement over the current baseline).

Multiple dimensions of success criteria

Claude documentation states that most features must be evaluated against multiple success criteria, listing the following common dimensions:

Setting only accuracy as a criterion causes teams to miss failures across other dimensions. Using a larger model to improve accuracy degrades latency and cost, and writing longer answers can reduce relevance. Setting separate criteria for each dimension makes the trade-offs visible.

Precision and recall

For features that "catch" something (such as detecting suspected fraud, spam, or urgent requests), precision and recall are more important than accuracy. By the definition in the Google PAIR Guide, precision is the proportion of actual positives among those predicted as positive, and recall is the proportion of actual positives caught as positive.

The PAIR Guide states that weighing the costs of false positives and false negatives is a decision that governs the user experience, warning against mechanically treating both with equal weight. Because it is difficult to improve both metrics simultaneously, the requirements must specify which one to prioritize for each feature. The F1 score (the harmonic mean of precision and recall) is the metric used when evaluating both together.

Procedure for turning vague requirements into criteria

When given a requirement such as "The summary must capture the intent well," convert it in the following order:

1. Break down the meaning. Break down with the requester whether "intent" means a resolution commitment, next steps, or tone.
2. Translate into observable behavior. "Next steps (who, when, what) are included in the summary without omission."
3. Determine measurement methods. If there is a ground truth, comparison (code); if judgment is required, a rubric (a scoring guide listing criteria and score scales by item) and evaluators (humans or an LLM judge calibrated against humans).
4. Specify the data. An evaluation set drawn from real records, including its size and composition.
5. Set the baseline and threshold value. Measure the current level (baseline), review its supporting rationale, and set an achievable threshold value.

Claude documentation explains that even "fuzzy" topics like ethics or safety can be quantified, and that qualitative scales are valuable when used consistently alongside quantitative metrics. In this translation process, reaching alignment with the requester is more important than the numbers themselves.

Input contracts

An input contract is a commitment defining what a feature accepts. It specifies language, file types, length, required information, and information that will not be accepted. Without an input contract, a feature will process even unvalidated inputs, leaving users unaware of why results are poor.

While some boundaries, such as format and length, can be checked before sending to the model, others—such as "questions regarding unsupported tasks"—can only be recognized after reading the content. The former is handled via pre-validation, while the latter is filtered through intent classification or model judgment and routed to a designated guidance path, with that determination also verified against an evaluation set. The input scope must match the scope validated through evaluation. To support an unverified language or length, add that input to the evaluation set, verify that it exceeds the criteria, and then expand the scope. The Google PAIR Guide recommends using error states to inform users about the inputs that the AI requires.

Output contracts

An output contract is a commitment defining what a feature returns and in what format. For human-facing screens, it specifies length and structure; for results consumed by downstream systems, it defines fields, formats, and empty value handling methods.

Outputs passed to downstream systems must be particularly strict. Claude documentation explains that structured outputs guarantee responses adhering to a JSON schema in most cases through constrained decoding (a mechanism preventing tokens that do not conform to the schema from being generated). However, it notes that outputs may not match the schema when a refusal occurs (stop_reason is refusal) or when the output length limit (max_tokens) is reached, and that letter case for string enumeration values (enum, const) is not guaranteed, meaning values differing only by case can appear even in responses that complete normally without error. Therefore, enumeration values should be compared case-insensitively, and distinct values differing only by case should not be defined. As a result, output contracts should specify, alongside the schema, the rule: "Results that fail validation are not consumed, but are reprocessed or routed to a human."

[Example output contract]
Format: JSON {"item": string, "quantity": integer, "unit_price": integer}
Empty values: Unreadable fields are left as null and not filled by guessing
Validation: Schema validation failure, refusal, length limit reached -> routed to reprocessing queue rather than sent to downstream systems
Failure alerts: Alert the person in charge if the reprocessing queue exceeds 20 cases per hour

Even if accuracy is high, a downstream system will break if the format is invalid. Accuracy and format are distinct requirements.

Balancing quality, latency, and cost

Quality, latency, and cost exist in trade-off relationships, but multiple control levers are available. Claude documentation recommends first finding a configuration that works well without constraints before reducing latency. Reducing latency too early prevents teams from discovering the ceiling of maximum performance.

When measuring latency, distinguish total response completion time from Time To First Token (TTFT). For features where users wait in front of a screen, TTFT is critical; for features passing results to downstream systems, total time is critical. Cost should be written as "cost per transaction × monthly throughput" and compared against the budget.

User-facing failure experiences

AI features fail at a certain rate. What the user sees at that moment is a requirement. The Google PAIR Guide notes that what users perceive as errors is deeply tied to their expectations of the system, dividing errors into system limitations, context errors, and background errors. It advises focusing on what users can do after a failure and enabling users to take over the task when a failure occurs.

When using streaming, failures can occur even mid-response. Claude documentation explains that in streaming responses, errors can occur even after a normal response (200) has begun. Therefore, how to display a "half-generated answer" must also be included in the requirements. A screen that displays only "Cannot answer" is not captured in the incorrect answer rate, but it blocks the user. Examining post-failure screen behaviors (drop-off, retries, phone inquiries) in usage logs reveals the cost of failure experiences.

Launch-blocking criteria and improvement targets

Not all criteria carry equal weight. Launch-blocking criteria are standards that prevent release if unmet, no matter how good other metrics are; improvement targets represent performance levels to be elevated post-launch.

Having too many blocking criteria delays launches indefinitely, while having too few leads to incidents. The NIST AI RMF advises organizations to determine and document risk tolerance levels (MAP 1.5). Blocking criteria translate that tolerance level into numbers. There is another option: if only specific types fall short of criteria and a routing path to humans exists for those types, a conditional launch that excludes those types from scope offers a solution between blocking and improvement.

Rationale for threshold values

Threshold values are not numbers determined by negotiation. Claude documentation advises grounding them in industry standards, prior experiments, AI research, and expert knowledge, keeping targets realistic given state-of-the-art model capabilities. In practice, three rationales are used:

1. Current performance level (baseline). The performance level currently achieved by humans or legacy systems. If human agents have a classification accuracy of 93%, the justification for setting an AI target at 99% is weak.
2. Cost of errors. The harm incurred when a single case is incorrect. The higher the potential harm, the higher the criterion must be set, or human verification must be added.
3. Small-scale experiments. Results obtained from running 50 to 100 sample cases. This checks in advance whether criteria are achievable.

Promotional figures from competing services are difficult to use as justification because their measurement methods are unknown.

Structure of a requirements document

Requirements documents for AI features facilitate review and determination when written using the following structure:

1. Purpose and users: Who uses it and for what
2. Input contract: Accepted inputs, excluded inputs, out-of-scope handling
3. Output contract: Format, empty values, validation failure handling
4. Success criteria: Metric, data, threshold value, and measurement method by dimension (including baseline)
   4-1. Launch-blocking criteria
   4-2. Improvement targets
5. Failure experiences: Screen display and next action by failure moment
6. Latency and cost: TTFT/total time, cost per transaction × throughput
7. Legal requirements: Notices/disclosures, personal information handling conditions
8. Unresolved items: Requirements requiring definition, assignees, and deadlines

Going deeper

The temptation to reduce everything to a single metric

Executives usually want a single number: "So what is the score?" An overall score calculated as a weighted average across multiple dimensions is convenient for reporting, but dangerous. This is because blocker criteria like personal information exposure get buried under high scores from other dimensions. If the weighted average of accuracy 95, tone 90, and personal information 40 is 80, that 80 looks like a signal that it is acceptable to launch.

Blocker criteria are evaluated separately instead of being included in the average. If an overall score is necessary, it is used solely to show trends across improvement targets for features that have passed all blocker criteria. The first line of the report must be "Passed all five blocker criteria," with the overall score following after.

Error magnitude also serves as a criterion

Not all errors carry the same weight. In the multidimensional criteria examples in the Claude documentation, there is an item stating that "90% of errors are inconveniences rather than critical errors." Even between two features with identical accuracy, the launch decision differs depending on whether an incorrect answer is "at a level where asking again with different phrasing suffices" or "at a level that promises an incorrect amount."

Therefore, mature requirements categorize errors by severity and set criteria accordingly. For example, critical errors (incorrect guidance on amounts, deadlines, or qualifications) are set to 0.5% or lower, while minor errors (awkward phrasing) are set as improvement targets. Because the definition of severity itself is also a requirement, it is documented alongside examples.

When is consistency a requirement?

If a different answer is produced every time for the same question, users lose trust. Consistency is as important as accuracy, especially in areas where "the same conditions must yield the same answer" is taken for granted, such as policy guidance, price explanations, or eligibility evaluations. The Claude documentation cites consistency as one dimension of success criteria and advises asking how important it is to receive semantically similar answers when asking the same question twice.

Conversely, in features where diversity is valuable, such as brainstorming ideas or generating candidate copy, demanding consistency actually diminishes usefulness. Consistency requirements must stem from the purpose of the feature, and remember that high consistency does not imply accuracy. A model can consistently produce the same incorrect answer.

At work

Report classification at Moon Halo Games

The game company 'Moon Halo Games' sought to classify 20,000 player reports per day (profanity, cheating software, payment issues, etc.) using AI and route them to the responsible departments. The initial requirement was "Classify reports accurately."

The planner first divided the dimensions: classification accuracy, emergency report detection (cash fraud, personal information exposure), latency, and cost. For accuracy, taking 3,000 reports classified by humans last month as an evaluation set, the planner recorded "Exact match across 11 categories of 92% or higher." Because the human reviewers' current agreement rate was 94%, there was no basis to demand 100%.

Emergency reports were handled separately. Emergency reports accounted for only 0.3% of the total, but missing them led to actual financial damage. Because human reviewers re-examined all classified emergency reports, false positives were manageable. Therefore, "Emergency report recall of 98% or higher" was recorded as a launch blocker criterion. Precision was calculated backward from the human reviewers' daily capacity to review 300 cases. Because actual emergency reports amounted to about 60 cases per day, precision had to be 20% or higher so that cases classified as emergency would not exceed 300. This "Precision of 20% or higher" was recorded as an operational condition. The discarded proposal was "Setting overall accuracy of 95% or higher as the sole blocker criterion." This was because the 0.3% of emergency reports would be buried within overall accuracy and remain invisible.

An output contract was also included. The classification output permitted only one of the 11 category codes, while any other value or formatting error was routed to an "unclassified queue" for human handling. Finally, "Naturalness of classification reasoning sentences" was designated as an improvement target. Because the responsible department read the original text rather than the reasoning, there was no reason to block the launch.

Card messages at Wildflower Letters

The flower delivery service 'Wildflower Letters' planned a feature where AI suggested three message card drafts when an ordering customer selected the recipient and occasion. The initial requirement was "Generate touching messages quickly."

The planner broke down "touching." What emerged from customer interviews was: "Appropriate for the occasion (no congratulatory expressions in condolences)," "Appropriate form of address for the recipient," and "Not too long (fits on the card)." The first two became blocker criteria. Because even a single error where congratulatory expressions were mixed into a condolence context led to major complaints, the planner recorded "Zero congratulatory expressions in condolence contexts (programmatic code checks on all text using a forbidden expressions list), and other context mismatches of 0.5% or lower." Card length was handled via an output contract (80 characters or fewer per message), while the warmth of the phrasing was designated as an improvement target scored by a rubric.

"Quickly" was converted into a TTFT criterion. Because customers waited in front of the screen, streaming was adopted so that the first message appeared within 2 seconds. In the evaluation, because a smaller model exceeded all blocker criteria, the smaller model was chosen instead of a larger model, reducing cost per request as well. The discarded proposal was "Controlling length by setting a low max_tokens." This was rejected because sentences were cut off midway, making them unusable on cards.

Common mistakes

1. "Accuracy alone is enough." Latency, cost, tone, and personal information handling move independently of accuracy. Set criteria for each dimension.
2. "If there is a number, it is a good criterion." If the measurement method cannot be trusted (such as self-evaluation by a generative model) or is irrelevant to the objective (such as "shorter is better"), it remains a poor criterion even if it includes numbers.
3. "Setting every criterion as a launch blocker makes things safe." If you block a launch over quality aspects that do not hinder usage, the release will be delayed indefinitely. Only criteria that lead to harm should be blocker criteria.
4. "If accuracy is high, format validation is unnecessary." Format is a requirement distinct from accuracy. Even structured outputs can deviate due to refusals or length limits.
5. "Reducing max_tokens makes answers short and clean." Hitting a length limit cuts off sentences midway. Determine length through prompts and output contracts.
6. "If speed is an issue, switch to a smaller model." If only one model exceeds the quality criteria, reduce perceived latency using streaming or output length.
7. "Simply refusing when unable to answer is safe." A refusal without a next action blocks the user and transfers costs to other channels.

Summary at a glance

• Good criteria = Metric + Data + Target threshold + Measurement method, plus a baseline providing the rationale for the threshold. Another person must be able to evaluate them.
• Success criteria are established across multiple dimensions (accuracy, consistency, relevance, tone, personal information, context utilization, latency, and cost).
• For detection features, prioritize whichever has the higher cost of error between precision and recall.
• Align the input contract with the scope verified through evaluation; provide guidance for out-of-scope inputs prior to processing.
• In the output contract, record handling rules for validation failures, refusals, and length limits alongside the schema.
• For latency, distinguish between TTFT and total duration, adjusting via streaming, output length, model selection, and batch processing.
• Write the user's next action as a requirement for every moment of failure.
• Blocker criteria are strictly limited to those leading to legal violations, contractual breaches, or direct harm; treat the rest as improvement targets. If there is no definition, begin with the definition.

Check the official docs

• Claude Docs — Define success criteria and build evaluations: Conditions for good success criteria, examples of bad and good criteria, common success criteria dimensions.
• Claude Docs — Structured outputs: Guarantees of schema adherence and exceptions (refusals, max_tokens).
• Claude Docs — Reducing latency: Baseline latency and TTFT, model selection, output length, streaming.
• Claude Docs — Batch processing: Costs and processing time for non-real-time batch operations.
• Google PAIR Guidebook — User Needs + Defining Success: Precision and recall, costs of false positives and false negatives.
• Google PAIR Guidebook — Errors + Graceful Failure: User-perceived errors, error states, next actions following failure.
• NIST AI RMF — Core: MAP 1.5 (Determining and documenting risk tolerance levels).

Self-check

2-1. State the four elements that must be included in a good success criterion, and state what is helpful to document alongside them as the rationale for the target threshold.

2-2. Convert "Answers must be fast" into an evaluable criterion, and explain which latency metric to use if the feature involves a user waiting in front of the screen.

2-3. Define precision and recall in one sentence each.

2-4. There is a feature that detects "cancellation requests" in hospital appointment text messages and automatically cancels the appointments. Detected cases are executed without human verification. Which metric must be safeguarded as a launch blocker criterion, and why?

2-5. Explain using two scenarios why handling rules for validation failures must still be recorded in the output contract even when using structured outputs.

2-6. Divide the following items into launch blocker criteria and improvement targets: (a) The rate of customer bank account numbers remaining in consultation summaries (b) The conciseness score of summary sentences (c) The rate of stating refund deadlines incorrectly.

2-7. (Design) Choose one frequently used AI feature, and create a table describing three failure moments along with the next action to display to the user for each moment.

Hands-on tasks