Appendix B. Self-check answers and explanations

1-1. HTTP 200 means only that a request was processed; refusals, output-limit stops and tool requests also arrive as 200. Check HTTP success through the status code, generation completion through the stop reason (Anthropic stop_reason, OpenAI status/incomplete_details), and business success through schema/semantic validation and prohibited-expression checks based on the response contract. Mixing them allows truncated answers or refusals to be published as normal results.

1-2. Truncation loses early conditions and facts; summarization loses omitted details. Server-side compaction can also lose details, and readability differs by provider (OpenAI items are opaque; Anthropic blocks contain readable summaries). When summarizing yourself, define information that must survive, such as customer identifiers, agreed conditions and unresolved questions, and check with the same evaluation conversations before and after compaction.

1-3. temperature sharpens (low values) or flattens (high values) the candidate probability distribution; top_p keeps candidates in descending probability order up to cumulative probability p; top_k keeps only the top k. Three rules apply. Check support in provider documentation first (Anthropic rejects temperature other than 1.0 and all top_k on models released after Claude Opus 4.6; Gemini 4 and 3.6 and later ignore these values). Remove unsupported values from requests. Achieve consistency with structured output and evaluation sets.

1-4. Message construction: place rules only in system (or developer) positions, send page content in a separate sourced block or tool result, and explicitly state not to follow instructions inside material. Application code: remove or minimize tools and permissions available to the summarizer, and add output checks and input-length limits. Delimiters only mark structure, so code-level controls are essential.

1-5. Task quality, latency, cost, limits, features and lifetime. Limits and feature support differ by model and change frequently, making code checks safer. Anthropic's Models API returns model-specific max_input_tokens, max_tokens and capabilities, including whether thinking can be disabled and server tools are supported. Use a token-counting API to check in advance whether input fits the window.

1-6. Example answer. Classification: small tier, low reasoning effort and a small ceiling based on observed maximum output plus headroom; judge by p95 latency and accuracy. Reports: large tier or high effort, a generous ceiling including reasoning, and streaming; judge by evaluation pass rate and per-request cost ceiling. For both, choose the cheapest combination meeting evaluation-set criteria and record settings.

2-1. Meaning of absence, units and representation, model-generated versus server-supplied values, and version. For example, make receipt total_krw a nullable integer and capture the reason separately with total_status (read, not_printed, illegible). Then missing printed totals and unreadable totals can follow different handling.

2-2. Any two of the following. OpenAI requires optional fields in required with nullable types; Anthropic permits omitting them from required. OpenAI supports minimum, maximum, pattern and recursive schemas (except fine-tuned models); Anthropic lacks numeric/string-length constraints and recursion (using them returns 400). Key ordering and complexity limits also differ. With both providers, use a shared subset or provider-specific conversion layer, move unsupported constraints into descriptions and application validation, and run the same evaluation set for each.

2-3. Syntax validation checks JSON parsing; schema validation checks fields, types and enum values against the schema; semantic validation checks business rules involving totals, date ordering, state and similar relationships. Check response state before all three. Refusals, output-limit stops and incomplete responses may not match the schema and must be filtered before parsing.

2-4. For example, four fields: status (ok, error, needs_info), data, error, hint, plus result-size limits and personal-information removal rules. Explaining what went wrong and what to try next lets the model recover or choose alternatives without guessing. Anthropic recommends actionable messages instead of generic “failed” errors; after invalid-argument error results, Claude usually corrects and retries two or three times.

2-5. States include generating, call requested, validating, executing, returning results, complete and budget exceeded, plus abnormal exits such as refusal/truncation. Budgets cover steps, elapsed time and token cost. When exceeded, preserve partial results, hand off to a person or inform the user of progress, and record which budget was exceeded.

2-6. Examples include user ID, organization (tenant) ID, permission scope, current time/time zone and payment limit. Model output can be manipulated by instructions hidden in material (LLM01 Prompt Injection). Executing with model-chosen identifiers or permissions can cause harmful actions such as accessing another person's data (LLM06 Excessive Agency); passing unvalidated model output downstream is itself LLM05 Improper Output Handling. The server supplies session-known values and rechecks immediately before execution.

2-7. Example answer.

3-1. Time to first token (TTFT), output speed in tokens generated per second, and total response time to the last token. Also consider the first result the user can use. Complaints arise in the slow tail rather than the average, which hides a few very slow requests; use p95.

3-2. Any two: gateway/reverse-proxy buffering, compression accumulating chunks, server accumulation before sending, or a screen waiting for structured completion/postprocessing. Ensure buffering is disabled on SSE routes, verify immediate chunk forwarding with reasonable screen-update units, check upstream propagation of user cancellation and review disclosure timing for content requiring moderation.

3-3. If prefix-processing cost is 1 and the read multiplier is r, one-hour caching costs 2 + r × (N − 1), five-minute caching 1.25 + r × (N − 1), and no cache N. At r = 0.1, the one-hour cache costs 2.10 at N = 2, still a loss; at N = 3 it costs 2.20, below 3, so it saves from the third use (N > approximately 2.11). The five-minute cache saves from the second. Lower read multipliers, such as 0.05, increase savings. One-hour caching needs more reuse because of higher write cost but lasts longer.

3-4. Prompt caches reuse input-prefix processing and generate output anew. Response caches return earlier answers themselves. Caching answers containing user-specific information risks exposing it to others or returning answers with incorrect permissions or freshness. Avoid personalized response caching or include user, organization, permissions and document version in keys, invalidating on revision.

3-5. Any three verifiable signals such as schema validation, inclusion of required facts/elements, absence of prohibited expressions or business-rule validation. Escalated requests pay for both the small and larger model and take longer. High escalation rates offset small-model savings; judge total call, escalation and review costs divided by tasks passing quality criteria.

3-6. Example answer.

4-1. The server may have finished before disconnection, so treating the outcome as failure and re-executing can duplicate the action. Sequence: look up actual state with the same business key → if applied, use that result as success → if confirmed unapplied, repeat with the same idempotency key → if lookup is impossible, hold and notify a person.

4-2. Error codes in HTTP response bodies and response headers. For example, an OpenAI 429 caused by request rate is retried following Retry-After; exhausted credits or spend limits require stopping and alerting because retries cannot help. When Retry-After is present, wait at least that long; for a temporary rate limit without the header, use exponential backoff, jitter and retry caps. Do not infer the cause from header presence alone; inspect the body code. Side effects, idempotency-key availability and remaining deadline are also valid answers.

4-3. Token-bucket capacity continuously refills toward its maximum rather than resetting all at once at a fixed time, so per-minute limits can be enforced over shorter intervals. Anthropic's example notes that 60 per minute may be enforced as one per second; a momentary burst can be rejected even within the minute's total.

4-4. Connection timeout limits waiting for TCP/TLS establishment; idle timeout limits waiting for the next stream event; the overall deadline is the absolute time by which the user needs an answer. Long generation may keep emitting events, so idle timeouts terminate stalled connections promptly while preserving healthy long requests. A short single timeout kills healthy requests; a long one holds stalled connections too long.

4-5. Store the key, processing state (processing/complete/failed), request fingerprint (hash) and result, with a unique constraint on the key. Retain it beyond the possible retry period. For the same key and content, return the stored result; for the same key with different content, reject as a conflict. If the key is still processing, do not duplicate execution; let the caller look up the result.

4-6. Example answer. Real-time support: one retry within three seconds, dedicated concurrency pool, ten-second idle timeout and 15-second deadline, preevaluated fallback model or guidance during incidents. Document summarization: three retries with backoff/jitter, two-minute deadline, queue on failure. Nightly analysis: generous retry budget, low concurrency, defer to the next run during incidents. All use one retry layer, per-feature limits and circuit breakers.

5-1. Specific, measurable, achievable and relevant. Examples: “At least 95% of summaries contain all five required items—inquiry type, request, deadline, action and unresolved question (code grading)”; “At least 90% score four or above on the tone rubric (calibrated LLM grading)”; “p95 latency no more than five seconds”.

5-2. Make the rubric specific enough for decisions → restrict output to pass/fail or 1–5 → reason before scoring → use a different model from the generator → measure agreement with human-graded samples (also chance-adjusted measures for imbalanced sets), revising the rubric if below target → check length bias favoring long answers and order bias favoring the first answer → pin model and rubric versions.

5-3. From generation 4.6 onward, every Anthropic Claude model ID, including undated ones, is a fixed snapshot. Earlier aliases are convenience names pointing to dated IDs. Pinned models still have retirement dates, and cheaper or better models keep arriving, so prepare an evaluation-based replacement procedure.

5-4. Article 2, subparagraph 1-2 defines pseudonymization as processing personal information by deleting parts or replacing some or all of it so a specific person cannot be identified without additional information. For evaluation sets, replace names, contact details and order numbers with consistent placeholders, keep mapping tables separately with access controls, and preserve evaluation-relevant structure such as inquiry types and date relationships. Anonymization takes priority if it serves the purpose (Article 3, paragraph 7). Confirm implementation with privacy staff.

5-5. Compare with predetermined segment thresholds → inspect paired losses and classify causes → check whether the difference exceeds between-run variation. Options include holding the switch entirely, splitting by segment (keeping the old model for high-risk work), retuning prompts/effort and reevaluating, or adding human review. Record the decision and rationale and add losing cases to evaluations.

5-6. Example answer. Cost (tokens per task/cache-read ratio; inspect recent deployments after sharp drops); truncation (shares by stop reason; check ceiling/format when limit stops rise); format (validation failure rate by stage; inspect configuration history on failure); limits (429 rates by cause; page immediately on one spend-limit error); business success (automation/human correction rates; inspect losing cases and expand evaluations when corrections rise). Add an owner to every row.