Appendix B. Self-check answers and explanations

Chapter 1

1-1. It is a workflow if code determines the next action along predefined paths, and an agent if the model determines it dynamically during execution. For example, a support system can use code for inquiry classification and permission checks, while letting the model choose tools to investigate only inquiries requiring cause analysis. Such a system combines both approaches. Document who determines the next action at every stage.

1-2. Sectioning runs independent subtasks concurrently to gain speed and focus per call. Voting runs the same task repeatedly to gain reliability through several perspectives and attempts. Voting increases cost with call count. Using the same evidence can correlate errors, so accuracy may improve less than expected. Restrict voting to ambiguous stages where a single error is costly.

1-3. The four elements are objective, output format, guidance on tools and sources, and clear task boundaries. Without an objective, workers investigate the wrong scope. Without boundaries, several workers duplicate research. Without an output format, synthesis becomes difficult. Without source guidance, low-quality sources may be used. An explanation of any one of these symptoms is sufficient.

1-4. A maximum iteration count, and a stop when scores no longer rise or improvement falls below a minimum. Both belong in the code running the loop, rather than the model. It is also useful to hand the final result and evaluation record to a person when the loop ends without convergence.

1-5. A handoff transfers control to the chosen specialist, which handles the remaining execution for the turn and produces the final answer. With agent-as-tool, the manager retains control, calls the specialist as a tool, receives its result and produces the final answer itself. Choose agent-as-tool when you need the result to continue; choose handoff when the specialist should interact directly with the user.

1-6. If much of the multi-agent benefit comes from spending more tokens, check whether success increases enough to justify higher total costs. Compare cost per successful task. Doubling cost may be justified by a substantial success-rate gain; even unchanged cost is a loss if success remains unchanged.

1-7. Example answer: the simplest structure is a fixed workflow using one LLM call to summarize each announcement, with code grouping and sending summaries by department tags. Signals of insufficiency include department classification errors exceeding a defined threshold, such as 5%, or a significant increase in requests requiring several announcements to be read together. Then separate department classification into a routing stage, sending only cross-announcement requests to a tool-using agent. The key is specifying numeric signals.

Chapter 2

2-1. Human developers read documentation once and remember it. Unless lazy loading such as tool search is used, agents must read definitions and results as context repeatedly, so more tools and larger results increase context cost. Wrapping endpoints directly also creates similar tools, increasing selection errors and making the model compose multiple calls. A small number of workflow-level tools is therefore preferable.

2-2. Always returning detailed results wastes tokens; always returning concise results may omit identifiers needed for the next call. A detail parameter lets the agent choose according to circumstances, reducing both problems. Use detailed responses only when the next call needs IDs or detailed fields.

2-3. State that truncation occurred and the total extent, such as 20 of 1,240 results, and explain how to narrow the range through filters or a next-page cursor. Without this information, the agent assumes the subset is complete, draws conclusions from it or incorrectly decides a needed item does not exist.

2-4. Protocol version, client capabilities and client identity. These are carried in each request's _meta. Version (io.modelcontextprotocol/protocolVersion) and capabilities (io.modelcontextprotocol/clientCapabilities) are mandatory. Client information is recommended, but is self-reported and must not be used for security decisions.

2-5. Defaults mean the tool is not read-only (readOnlyHint: false), may be destructive (destructiveHint: true), is not idempotent (idempotentHint: false), and may interact with the outside world (openWorldHint: true). These conservative defaults place unannotated tools in the most cautious category. Even provided annotations are not trusted when from an untrusted server.

2-6. Audit trails: downstream logs appear to show the original token owner rather than the MCP server making calls, while the server cannot distinguish which client called, complicating incident investigation. Rate limits: direct use of a token can bypass server or downstream API controls relying on its audience and restrictions.

2-7. Example answer: use a name combining a service prefix and action, such as loan_extend. Accept loan_number, a string such as LN-2026-00123, and new_due_date, an ISO date within 14 days of the current due date. Repeating the same request then yields the same result. A business error should return reasons and facts, such as “Books with waiting reservations cannot be renewed. The scheduled return date is 2026-10-27.” Require human confirmation when exceeding the renewal limit or incurring overdue fees. An answer is correct if it explains how each choice reduces misuse.

Chapter 3

3-1. The model divides attention among relationships between all context tokens, leaving less attention for each piece of information as tokens increase. This is the attention budget; the resulting decline in accurate recall is context rot. A larger window permits more information, but does not ensure equal use of everything inside. Management remains necessary.

3-2. Old raw tool results deep in a conversation have already served their purpose and are rarely needed again. Removing them can preserve continuity and decisions. However, removal may invalidate the prompt cache from that point, so clear enough at once to justify invalidation rather than clearing small amounts frequently. Exclude results that are repeatedly referenced.

3-3. Task state—progress files and checkpoints—should determine the next action. Conversation history is summarized and truncated, while long-term memory is only reference material. When task state conflicts with the system of record, that system is authoritative. Correct task state to match it before choosing the next action.

3-4. It prevents later sessions from prematurely declaring completion after seeing progress, or editing feature lists and deleting tests to create an appearance of success. JSON was less likely than Markdown to be edited or overwritten inappropriately. Restricting changes to the pass-status field prevents reducing scope or lowering standards.

3-5. If the process dies just before recording the result, records alone cannot show whether execution occurred. Recording intent first lets resumption code find entries with intent but no result, then query the external system by idempotency key before re-executing. If processed, record only the result; otherwise execute with the same key to avoid duplicates.

3-6. First, source and time let you judge staleness and apply expiry or reverification. Second, they resolve conflicts with new information through source authority and recency. A source also makes incorrect memories easier to trace, correct or delete.

3-7. Example answer: ① Goal and completion criteria: what must be delivered, by when and in what format. ② Completed items and verification: what is finished and how it was checked. ③ Remaining work and priorities: the order of next actions. ④ Facts and assumptions: distinguish sourced, timestamped facts from unverified assumptions. ⑤ Prerequisites for the next action: external changes to check before resuming, approvals needed, and locations of artifacts and external operation identifiers. The key is verified artifacts rather than self-assessments such as “almost done.”

Chapter 4

4-1. The transcript records the agent's statements, tool calls and intermediate results during the trial. The outcome is the environment state after the trial: whether exactly one reservation for that date actually exists in the booking system. The transcript shows what was attempted, but does not prove what remained in the environment. A claim that a booking was made is a failure if no reservation exists. Grade state-changing agents by outcomes.

4-2. Apply trajectory checks when mandatory or prohibited rules such as user confirmation before payment cannot be verified from the outcome alone. Do not require them when several valid lookup sequences reach the same answer: exact path matching would fail more efficient correct solutions too.

4-3. pass@3 = 1 − (1 − 0.9)³ = 1 − 0.001 = 0.999, or about 99.9%. pass^3 = 0.9³ = 0.729, or about 72.9%. At least one of three attempts is almost certain to succeed, but all three succeed only about 73% of the time. The first value reflects operations with retries or candidate selection; the second is closer when each attempt must succeed on the first try. In practice, failures across trials of the same task are often correlated, so measured values may differ from the independent-trial assumption.

4-4. ① Separate evaluation dimensions, define observable criteria and penalty examples for each, and allow Unknown when evidence is insufficient. ② Create expert-graded samples and measure agreement with grader judgments. ③ Read disagreements, revise the rubric and measure again. Continue periodic sampling to ensure agreement holds.

4-5. A trace covers one workflow from beginning to end. Spans are its constituent work units with start and end times, linked by parent-child relationships. One agent run can include agent execution, LLM generation, function tool call, guardrail and handoff spans, based on the OpenAI Agents SDK defaults.

4-6. Failures usually begin with an incorrect decision at an earlier stage and surface later. Fixing only the visible symptom leaves the cause to recur in a different form and can lead to changing the wrong element and misjudging its effect. Knowing the first incorrect stage helps decide whether to change tool definitions, prompts or the environment.

4-7. Example answer: mandatory passes—zero prohibited actions and regression pass rate above a defined threshold. Comparative metrics—outcome-based success and pass^3 at least as high as the old version, with no regression by task type. Operating constraints—cost per successful task at most 1.2 times the old version and first-response latency within contractual limits. Defer judgment—if the difference is smaller than repeated-run variation or the sample below a minimum, increase trials and measure again. The key is keeping mandatory conditions separate from averages.

Chapter 5

5-1. First, multiple expensive calls or large results within a turn: stop them with per-run token and cost limits. Second, repeated searches or lookups without progress: use repeated-argument and no-progress detection. Continually growing subagent counts, controlled by limits on subtasks and redelegation depth, is also an acceptable answer.

5-2. Independent retries at each layer multiply attempt counts, turning one failure into dozens of calls and increasing overload. Retry at only one layer, only for transient errors, with exponential backoff, jitter, attempt limits and respect for server retry-after. Other layers should pass recovery instructions rather than retry.

5-3. Side-effecting calls, such as payment, reservation and dispatch requests, may have been processed while only the response was lost. Treating them as failures and resending can duplicate effects. First query status through an idempotency key or request identifier. Resend with the same key only if not processed.

5-4. Excessive requests create approval fatigue: approvers click without reading, allowing dangerous requests through as a formality. Select approval targets by reversibility—how difficult the action is to undo—and impact on customers, external parties or money. Automate reversible, low-impact actions within limits.

5-5. A PreToolUse hook exiting with code 2 stops a call before permission rules are evaluated. The call is blocked even with an allow rule, and the hook's standard error supplies the reason. Conversely, a hook returning allow cannot bypass deny or ask rules. Permission rules themselves run deny → ask → allow, with the first match determining the result. However, an installed mod's tool.check can override this decision where managed settings or Team/Enterprise login are absent.

5-6. The sandbox covers shell commands and their child processes. It does not cover built-in file and web tools such as Read, Edit and WebFetch, or other processes Claude Code launches, such as hooks and local MCP servers. Permission rules control built-in tools. Use only trusted hooks and local MCP servers, which run with full user privileges, and isolate them separately when needed. Direct ! shell commands in most sessions, excluded commands and unsandboxed retries are also valid examples.

5-7. Example answer: define a daily per-user cost limit, such as tokens equivalent to KRW 3,000, and reserve estimated cost before runs begin. At 80%, tell users the remaining allowance and move expensive features such as long research or multi-agent work to smaller models or narrower paths. At 100%, refuse new runs, save partial results and stop active runs, and explain the next day's reset time and how to request a higher limit. The key is enforcing both thresholds in code.