Chapter 4. Reasoning and multi-step tasks

Learning objectives

• Explain how reasoning models, reasoning tokens and reasoning effort affect quality, cost and latency.
• Choose effort to suit task type and latency requirements, and validate the choice through evaluation.
• Adjust instruction detail for reasoning models and request verifiable evidence instead of internal thought processes.
• Split multi-step work into prompts and design input/output contracts and gates between stages.
• Recognize when splitting is counterproductive and distinguish parallel from sequential work.

Introduction

A marketing agency added “Think carefully, step by step” to its advertising-copy prompt and selected the highest reasoning effort. Results improved slightly, but each response took tens of seconds and the bill tripled. Meanwhile, its contract-review feature used low effort in a single call for speed, frequently misjudging clauses involving three or more conditions. The advertising team missed deadlines waiting for responses; the contract team trusted an incorrect judgment and overlooked a risky clause. The teams appeared to have opposite problems but made the same mistake: they did not match the cost of thinking to the nature of the task.

This chapter starts with what model “thinking” means and where its costs arise. It covers reasoning versus general models, the effort control and how to calibrate it, the relationship between output limits and reasoning tokens, prompts for reasoning models, and why internal thought processes should not be requested in output. The second half covers task decomposition: workflow patterns, stage input/output contracts, making self-review concrete, parallel and sequential work, and subagents. The deeper discussion addresses excessive decomposition and work spanning multiple context windows.

Key concepts

Reasoning is scratch paper

In a math exam, a student working on scratch paper handles difficult problems better than one using mental arithmetic, but takes longer. Reasoning models similarly think on scratch paper before answering. Anthropic explains that with thinking enabled, a model reformulates the problem, tries different approaches, checks intermediate results and abandons unsuccessful paths.

This scratch paper is not free. Both companies describe the same structure.

Deciding when and how much reasoning to use is therefore part of design. Remember that reasoning works more deeply with available information; it does not create new facts absent from context.

Reasoning models and general models

OpenAI compares reasoning models to senior coworkers and general-purpose GPT models to junior coworkers: seniors can work from higher-level guidance, while juniors need explicit instructions. Its reasoning best practices also describe reasoning models as planners for complex, multi-step problems and GPT models as workers handling well-defined tasks quickly and cheaply. Most workflows use both.

Anthropic uses adaptive thinking in newer models. The model decides whether and how much to think from request complexity and the effort setting, answering easy questions directly without thinking. Thinking is always on and cannot be disabled on some newer models. The budget_tokens parameter from earlier extended thinking still works on Claude Opus 4.6 and Sonnet 4.6 but is deprecated; Claude 4.7 and later return a 400 error. Effort is now the main control.

Reasoning effort as a dial

Both companies explain that low effort favors speed and cost, while high effort favors thorough reasoning. Anthropic says effort affects all tokens, including explanations and tool calls, so lower effort produces fewer and shorter tool calls. “Behavioral signal” means the model can still think at low effort on a sufficiently hard problem, but thinks less than it would on the same problem at high effort. Use max_tokens, rather than effort, for an exact cap.

Effort is not a length control either. Anthropic notes that on some models, changing effort does not reliably change visible response length. Set length through instructions.

Match reasoning effort to cost and latency

Choose effort through measurement, not intuition. Anthropic’s best practices recommend explicit settings, low effort for simple or speed-sensitive work, testing on your own tasks, and varying effort with complexity. This becomes the following process.

1. Separate task types. Simple lookups, structured extraction and multi-condition judgment differ even within one feature.
2. Set allowed latency and minimum quality per type. Use numbers, such as within two seconds and at least 95% accuracy.
3. Sweep effort levels on the same evaluation set. Record accuracy, average and maximum response time, and output tokens per level.
4. Choose the lowest effort meeting minimum quality. If it cannot meet latency requirements, reconsider the model or task structure.
5. Add targeted instructions if only certain types become shallow at low effort. When latency requires keeping effort low, Anthropic gives an instruction such as “This task requires multi-step reasoning, so think carefully before responding.”
6. Repeat the sweep when changing models. Identically named levels can involve different amounts of thinking across models.

Output limits and reasoning tokens

Output limits include reasoning tokens. OpenAI’s max_output_tokens caps everything generated: reasoning, visible output and hidden formatting tokens. If set too low, the request may stop during reasoning with status incomplete and no visible answer. OpenAI recommends initially reserving at least 25,000 tokens for reasoning and output combined, then adjusting based on usage. Anthropic likewise describes max_tokens as a strict cap on thinking plus response text and recommends ample headroom at high effort.

Example: the output limit is 4,000 tokens, and the model spends 3,600 on thinking.
→ At most 400 tokens remain for the visible answer. It may be truncated or absent.
→ Tokens used are still billed.

Control answer length through instructions, not the cap. Use the cap as a safeguard against unbounded thinking.

Prompts for reasoning models

Give reasoning models goals rather than procedures. OpenAI recommends clear goals, strong constraints and explicit output formats without prescribing every step. Its reasoning best practices say reasoning occurs internally, making “think step by step” unnecessary and sometimes harmful. Anthropic likewise says general guidance such as “think thoroughly” often produces better reasoning than a manually written step-by-step plan.

OpenAI emphasizes defining completion and self-verification for agentic or research tasks. Anthropic likewise recommends clear research success criteria and verification across sources. The more open-ended the task, the more necessary explicit stopping conditions become.

Do not request internal thinking in the output

Before built-in thinking, “write your thoughts, then answer” was common. Anthropic says it remains an alternative when thinking is off, but newer thinking-enabled models are different. Asking them to reproduce internal reasoning in response text may return a stop_reason of refusal, with category reasoning_extraction. The documentation lists these common causes.

• <thinking>, <reasoning> or scratchpad sections to fill before answering
• reasoning, thinking or trace fields in JSON output or tool inputs
• Instructions to keep writing private notes or reasoning records
• Requests for verbatim or complete reasoning

Such wording can hide in tool descriptions as well as system prompts. No fallback model is recommended for this category, so revise the prompt instead of retrying. Evidence is still available: request a short explanation, supporting evidence or a summary of work performed. To inspect a thinking summary, set display to summarized and read the thinking block. Reviewers need evidence supporting conclusions, assumptions and checkable calculations, rather than traces of private thinking.

Task decomposition and workflow patterns

Prompt chaining splits a long task into multiple calls. Anthropic’s agent-design article recommends starting with the simplest solution and adding complexity only when needed. It describes these common patterns.

Adaptive thinking and subagent orchestration can handle much multi-step reasoning within the model, so chaining is not always needed. Anthropic identifies two remaining uses: inspecting intermediate results and enforcing a specific pipeline structure. Separate calls allow stage-level logging, evaluation and branching, and gates such as human approval.

Input/output contracts for each stage

Once work is split, define what passes between stages as a contract. Without one, a slight format change upstream can break downstream stages or lose evidence along the way.

Three principles apply. First, pass only what the next stage needs while carrying evidence locations, such as document and paragraph numbers, all the way through. Second, check gates with code where possible. Third, identify Chapter 1’s task type at every stage to prevent polishing during extraction. Contracts especially prevent error propagation. One misread number can support smooth downstream logic that final-document reviewers struggle to detect. Intermediate artifacts with source locations let you restart at the first failed stage instead of rerunning everything.

Make self-review a concrete check

Self-review is the most common chaining pattern. Anthropic describes three separate calls: draft, review against criteria, and revise from the review.

Call 1: Generate a draft with instructions containing a checklist
Call 2: Review each checklist item against the draft: pass/fail and location
Call 3: Produce a revision changing only failed items

“Check again” is too vague. Specify missing requirements, agreement between source and numbers, exceptions and counterexamples, and conditions that do not require changes. Without the last item, review may alter sound content. If only one call is possible, add “Before finishing, check the answer against [test criteria].” Anthropic warns, however, that some models already verify well without instructions; older verification prompts can cause excessive checking. Set a maximum revision count and hand remaining issues to a person when it is reached.

Parallel, sequential and subagent work

Run independent work together and dependent work in order. Anthropic explains that newer models parallelize independent tool calls and that instructions can adjust this. Its example says to run calls concurrently when independent, sequentially when a prior result supplies parameters, and never guess missing parameters or fill them with placeholders. Parallel work reduces latency; sequential work preserves correctness. Dependency is the dividing criterion.

Subagents parallelize larger units. Focused work in clean contexts with summary-only returns helps broad research, but is costly. Anthropic’s multi-agent research article reports agents using roughly four times as many tokens as standard chat, and multi-agent systems roughly fifteen times as many. It describes rules under which one agent and three to ten tool calls suffice for a simple fact lookup. Anthropic also warns that some models spawn too many subagents and provides instructions reserving them for independent work that can run in parallel or needs context isolation, while handling simple or sequential work directly.

Going deeper

The cost of over-splitting a chain

Assuming more stages always help is risky. Every stage adds round-trip latency, fragments context across stages and increases the number of prompts to maintain. OpenAI’s latency optimization guide explains that merging sequential steps into one prompt and response removes round trips. Splitting may still win when a small model can do most of the work, so it recommends testing both on production examples. The guide also notes that reducing generated tokens affects latency much more than reducing input tokens.

Ask these questions when deciding whether to split.

Long work across multiple context windows

Work spanning days will not fit one context window. Anthropic recommends building the framework—tests and setup scripts—in the first window and iterating through a task list in later ones. Keep structured state such as tests in formal files such as JSON, progress notes in free text, and work history in Git. Start each new window by reading those records. Restoring state from files in a fresh window can sometimes be better than compaction. It also recommends an explicit rule from the outset against deleting or modifying tests merely to obtain a pass.

At work

Solid Metals: splitting quotation comparisons into stages

The purchasing team at fictional precision-parts manufacturer Solid Metals uses a feature comparing raw-material supplier quotations and drafting purchase-approval requests. Formats differ by supplier, with four or five quotations per case. Currently, all are supplied at high effort with “Compare and write the request” in one call. The prose is smooth, but twice last month it compared per-kg and per-tonne quotations as though the units matched, recommending the wrong supplier. The team lead, reviewing only the request, missed the error.

Purchasing specialist Jung divides the work into three stages with contracts.

1. Extraction: low effort. Use structured outputs to extract supplier, item, unit price, unit, delivery date and validity period per quotation, retaining page numbers per item and units exactly as written.
2. Gate: code. Normalize units with a conversion table. Return incomplete quotations to the coordinator and stop on any unit absent from the table.
3. Comparison and writing: high effort. Supply the normalized table and internal purchasing criteria—weights for price, delivery and transaction history. Obtain supplier judgments and evidence page numbers, then draft the approval request.

A final self-review call uses the same checklist: Is the recommended supplier cheapest in the table, and if not, does the criteria table justify it? Does every number have a page reference? Is any expired quotation recommended? Items needing no changes must remain untouched.

Jung rejects two alternatives. Adding a bolded warning to “pay attention to units” to the single call leaves extraction errors hidden in final prose. Splitting further into five stages creates uninspected intermediate steps and extra latency, so the workflow stops at three. Effort differs by stage too: structured extraction achieves the same evaluation accuracy at low and high effort, while multi-criteria comparison meets the quality floor only at high effort.

Comparing 40 quotation sets from the previous three months catches every unit mismatch at the gate and reduces cost per case because extraction is cheaper. Page references also reduce the lead’s review time.

Garam City Youth Employment Center: different amounts of thinking for different questions

The fictional Garam City Youth Employment Center operates a counseling bot for young jobseekers. Questions fall into three broad types: simple guidance such as hours and location, training-program schedule lookups, and subsidy eligibility judgments. Eligibility combines age, residency duration, income and overlapping subsidies. Initially, everything uses maximum effort, making even opening-hours answers so slow that users close the window.

Coordinator Ha introduces routing. A lightweight classifier chooses the question type and corresponding path. Simple guidance selects from prepared answers to minimize generation. Schedule lookups briefly summarize scheduling-tool results. Eligibility judgments receive a criteria table and user conditions, returning a decision, supporting clauses and missing-information list.

Effort is chosen by a sweep. Ha creates 60 evaluation questions per type and first defines latency targets and quality floors.

Medium effort meets the overall eligibility accuracy floor, but errors cluster on questions combining four or more conditions. Ha chooses high effort: the latency allowance permits it, and wrong eligibility guidance can cost users an opportunity to apply. Ha also removes the old instruction to write all thinking inside <생각> tags, replacing it with a request for brief criteria-table clause numbers and user conditions supporting the decision. Counselors need evidence, not private thinking traces.

At review, someone asks what happens if classification is wrong. Ha defines a default: route uncertain classifications to eligibility judgment. A slower answer is preferable to incorrect guidance. Simple responses become noticeably faster while eligibility accuracy is maintained. Operating rules require a fresh sweep on the same questions whenever the model changes.

Common mistakes

• The highest effort is always best. For simple, high-volume work it adds cost and latency. Choose by task and confirm through evaluation.
• “Think step by step” is enough. It can be unnecessary or harmful for reasoning models and does not meet task-specific latency and accuracy needs.
• Writing thoughts in the answer makes it verifiable. Visible thinking is a summary, and newer thinking-enabled models may refuse reproduction requests. Ask for evidence, assumptions and calculations.
• A lower output cap makes answers shorter. Reasoning may be cut off with an empty answer. Set length through instructions.
• More stages improve quality. Uninspected intermediate steps only add latency and maintenance. Compare with a single call.
• Subagents always make work faster. They greatly increase tokens and may slow simple lookups.
• Dependent tool calls are faster in parallel. Calling before required earlier results exist forces parameter guessing.

Summary at a glance

Check the official docs

• OpenAI — Reasoning models: Read about effort values, reasoning-token billing, max_output_tokens and the 25,000-token recommendation.
• OpenAI — Reasoning best practices: Read about dividing roles between reasoning and GPT models and prompting advice.
• Anthropic — Effort: Read effort-level descriptions, tool-use effects and best practices.
• Anthropic — Refusals and fallback: Read causes and alternatives for reasoning_extraction refusals.
• Anthropic — Prompting best practices: Read “Thinking and reasoning,” “Chain complex prompts,” “Subagent orchestration” and “Optimize parallel tool calling.”
• Anthropic — Building effective agents: Read workflow patterns and when to use them.

Self-check

4-1. Give two effects of reasoning tokens on cost and context, and explain how visible thinking-block text differs from the original thinking.

4-2. Explain why Anthropic’s effort is a behavioral signal rather than a strict token budget. What should be used for an exact output cap?

4-3. You are choosing effort separately for simple lookups and comparative analysis in an internal-document assistant. Write the selection process in order.

4-4. A reasoning request with a small output cap returns no visible answer. Explain the cause and give two remedies.

4-5. A general-model prompt has a seven-step procedure from reading the problem to writing the answer, plus “think step by step.” What should you keep and change when moving to a reasoning model?

4-6. A proposed evaluation JSON includes a thinking field for lengthy judgment reasons. Explain a potential problem on newer thinking-enabled models and propose another field design preserving the purpose.

4-7. A product-launch press release is produced in three stages: (a) extract facts from the specification, (b) compare competitor products and (c) write the release. Write stage input/output contracts and propose one gate between stages.

Hands-on tasks