Chapter 5. Operations, cost and safety

Learning objectives

• Prevent runaway execution and infinite loops through maximum turns, budget limits and detection of loops without progress.
• Design timeouts and retries according to error type and idempotency, preventing retry storms.
• Choose human approval points by reversibility and impact, binding approvals to execution content.
• Place hooks, callbacks and guardrails on the execution path and understand each mechanism's coverage.
• Contain damage through sandboxes and least privilege, and establish an incident response sequence.

Introduction

One Friday night, a research agent repeated the same search hundreds of times. Search results had changed format, so it could not read them; unable to read them, it searched again. Turn limits were disabled, and the cost alert was found in the inbox only on Monday morning. Another company had the opposite experience. Requiring human approval for every action generated hundreds of requests daily. Approvers pressed buttons within seconds, until a dangerous request passed without anyone reading it.

Both incidents relied on model judgment or human attention for safety. This chapter explains how to put safety into the system. Stopping conditions, budgets, timeouts, retries and overload controls prevent runaway execution. Approvals go only where risk requires them. Hooks, callbacks, guardrails and sandboxes enforce boundaries on the execution path. The chapter then establishes an incident response sequence.

Key concepts

Put stopping conditions in the system

“Stop when the task is done” is only advice. The code running the loop must enforce stopping. The OpenAI Agents SDK runner loops until final output, continuing on handoffs or tool calls. max_turns limits turns; exceeding it raises MaxTurnsExceeded. Passing max_turns=None disables the limit, so a shared execution wrapper should enforce a production maximum. A "max_turns" handler in error_handlers can return controlled final output, such as “Please narrow your request and try again,” instead of an exception.

result = await Runner.run(agent, "Check this week's settlement differences", max_turns=15)

A turn limit is insufficient. Several expensive tools may run within one turn, subagents use their own turns, and identical searches may repeat.

When stopping, record the reason for stopping, partial results and available next steps so users and operators can decide what to do next.

Layers of budget limits

Overlap cost limits at several layers so another layer catches failures in one.

Use provider-side limits too. Claude API lets organizations and workspaces set their own spending limits. Lower workspace spending and rate limits prevent one workspace's overuse from blocking others (these workspace-specific limits exclude the default workspace). Reaching a self-set spending limit usually returns a 400 error; Claude Code workspaces may receive a 429 with retry-after. Reaching the usage tier's monthly spending cap returns a 429 without retry-after. In the latter case, all retries, including SDK automatic retries, fail until access resumes. Distinguish errors by their content and notify operators instead of retrying.

The OWASP Top 10 for LLM Applications 2025 classifies this risk as Unbounded Consumption (LLM10), including denial of wallet attacks that drive up costs through excessive calls. Mitigations include input size validation, per-user rate limits and quotas, timeouts and throttling, usage monitoring, and graceful degradation retaining some functionality.

With concurrent workers, accounting only after completion makes limits hard to enforce. Reserve estimated costs before launching workers and reconcile afterward to reduce the risk of concurrent workers collectively exceeding the limit.

Detecting loops without progress

Even with few calls, repeating the same failure creates no value. Progress signals include the following.

1. The same tool is called with the same arguments N consecutive times.
2. The same error message returns consecutively.
3. No new evidence, files or state changes appear in the last M turns.
4. After failure, the same tool and argument combination is tried again.

When detected, stop and hand over records of repeated calls, error details and existing artifacts to a person. Implement detection in tool wrappers or loop code so it works regardless of whether the model follows instructions.

Timeouts and deadlines

A timeout is the maximum time a client waits for a request to finish. An Amazon Builders' Library article recommends setting timeouts from evidence rather than intuition: first choose an acceptable false-timeout rate—abandoning a request that would soon finish—such as 0.1%, then use the corresponding downstream latency percentile, such as p99.9. Set both connection and request timeouts for remote calls.

Agents also need an overall task deadline. Even with reasonable stage timeouts, accumulated stages and retries can prolong work indefinitely. Allocate remaining time among stages and do not start another if insufficient time remains. MCP likewise recommends timeouts on all requests and enforcing a maximum even if progress notifications reset the clock. For model calls, Claude API documentation recommends streaming or the batch API for long requests that could exceed ten minutes.

Retries

Retries recover from transient failures, but are “selfish,” in Amazon's wording: they consume more server capacity to improve their chance of success. During overload, retries worsen the overload. Follow these principles.

Include SDK defaults in calculations. Official Claude SDKs retry transient failures such as connection errors, rate limits and 5xx errors twice by default, with exponential backoff and respect for retry-after. Adding three tool-wrapper attempts and three more agent attempts can turn one failure into dozens of calls. Choose one layer to own retries. When execution status is unknown (Chapter 2), check the processing result before retrying.

Concurrency and overload control

One runaway task must not block other users' normal work. Four mechanisms help.

• Concurrency limits: Limit simultaneous runs and external calls by organization, user and tool.
• Queues and admission control: Slow admission or lower priority as queues grow. Give queued work an expiry too.
• Circuit breakers: When downstream failure rates exceed a threshold, block calls temporarily to avoid wasting resources on calls likely to fail and give the service time to recover. Like an electrical breaker interrupting excess current, the circuit is normally closed, passing calls; open while blocking; and half-open for trial calls after a delay. Successful trials close it again.
• Graceful degradation: Retain limited functionality when processing is unavailable, such as accepting requests and promising notification after processing.

For internal limits, a continuously replenished token bucket—also used for Claude API rate limits—handles bursts more easily than counters that reset at fixed intervals.

Human approval points

If every action needs approval, approvers soon stop reading: approval fatigue. OpenAI's guide recommends assigning low, medium and high risk to tools using read/write behavior, reversibility, account privileges and financial impact, then automatically associating high-risk execution with safeguards such as guardrail checks or human handoff. It names two triggers for human intervention: exceeding failure thresholds, such as repeatedly failing to understand intent, and high-risk actions such as canceling orders, issuing large refunds and making payments. Keep human supervision for high-risk actions until trust develops.

The two judgment axes are reversibility and scope of impact.

On approval screens, show what will actually be executed—targets, amounts, recipients and counts—along with its justification and a trace link. Fewer approval requests allow each one to receive serious attention.

Binding approval to execution content

There is a gap between approval and execution. The amount shown may differ from actual arguments, targets may change afterward, or one approval may be used twice. Binding approval to execution content closes that gap.

1. Include all execution arguments in the approval request. Store a hash of the canonically serialized arguments with fixed key order, or a preview ID (Chapter 2), in the approval record.
2. Immediately before execution, compare actual arguments with the approved content on the server. If different, do not execute; request approval again.
3. Give approvals an expiry and consume them after one use.
4. Recheck the approver's permissions at execution time. Approval itself grants no new business permissions.

In OpenAI Agents SDK, setting needs_approval on a tool pauses execution and returns pending items in interruptions. Convert the result to RunState, approve or reject, then resume with that state. Since serialized state includes pending tool arguments, follow documentation advice to store it under server control and restore only trusted snapshots. Also remember that approval is optional for local ShellTool and ApplyPatchTool, where needs_approval defaults to False.

Blocking mechanisms on the execution path

Place prohibitions on the execution path, rather than only in prompts. Code must be able to check and block immediately before a tool actually executes.

ADK plugin short-circuiting makes order important. Python plugins run in registration order and stop at the first returned value. A plugin returning a result early prevents later checks from running. Verify registration order for mandatory checks.

What guardrails cover

Know exactly where OpenAI Agents SDK guardrails apply.

• Input guardrails run only on the first agent in the chain; output guardrails run only on the agent producing the final output.
• Input guardrails run concurrently with the agent by default (run_in_parallel=True). Before a guardrail trips, the agent may already spend tokens and execute tools. For agents using tools with side effects, set run_in_parallel=False so the agent starts after the guardrail finishes. Documentation recommends this for reducing costs and preventing tool side effects.
• Tool guardrails wrap function tools (FunctionTool). They do not cover handoff calls themselves. Hosted tools and built-in execution tools such as ShellTool and ApplyPatchTool bypass this pipeline. Agent.as_tool() currently does not directly expose tool guardrail options.

Sandboxes and least privilege

Agents handling untrusted input or executing code rely on their environment for safety rather than model judgment. A sandbox is an isolated environment that restricts a program's file, process and network access at operating system level.

Claude Code's sandbox is a useful example. Disabled by default, it applies OS-enforced boundaries to shell commands and child processes when enabled: Seatbelt on macOS, bubblewrap on Linux and WSL2. Native Windows runs without the sandbox, so use WSL2. Documentation describes these boundaries.

• Writes are restricted to locations such as the working and temporary directories, but reads are open to most of the device by default, including credential files such as ~/.ssh. Configure denyRead to block them.
• Networking passes through a proxy checking an allowed-domain list, initially empty. Documentation warns that allowing broad domains such as github.com can create data exfiltration routes.
• Built-in file and web tools run outside the sandbox under permission rules. Command hooks and local MCP servers run with the user's full privileges, so their internal actions require separate isolation.
• A command failing in the sandbox may request an unsandboxed retry. Set allowUnsandboxedCommands: false to close this route. Enable failIfUnavailable to prevent execution entirely if the sandbox cannot start.

Documentation describes permission rules and sandboxes as complementary layers: sandbox limits remain even when prompt injection bypasses model judgment. It warns that bypassPermissions, which skips confirmations, should be used only in isolated containers or VMs where damage cannot spread.

OWASP groups the risk of agents taking harmful actions as Excessive Agency (LLM06), rooted in excessive functionality, permissions and autonomy. Mitigations include minimizing extensions and their scope, avoiding open-ended extensions, least privilege, execution within the user's authorization context, human approval for high-impact actions, and complete mediation, where downstream systems rather than the LLM decide authorization.

Prompt injection and separation of privileges

Prompt injection is a vulnerability where input changes model behavior away from its intended purpose (OWASP LLM01). Direct injection comes from users themselves; indirect injection hides in external content such as web pages or files. Indirect injection is especially dangerous for tool-equipped agents: a document intended only for reading starts instructing actions.

OWASP mitigations include constraining model behavior, defining and validating expected output formats, input/output filtering, access control and least privilege, human approval for high-risk actions, separating and labeling external content, and adversarial testing including attack simulations. The key design assumption is that injection cannot be detected perfectly. Place a permission boundary between reading untrusted content and taking irreversible actions. For example, require human confirmation for external sending tools after reading external content, or let a read-only subagent pass back only a summary.

Incident response

If an agent performs an incorrect batch operation, follow this sequence.

1. Contain: Immediately cut off write tools or the agent itself with a kill switch. Prepare feature flags changeable without deployment to disable tools, tenants or the whole system. Preserve traces and logs when stopping.
2. Scope: Determine affected targets and times through traces and audit logs. Preserve evidence.
3. Recover: Reverse or compensate only the confirmed scope. Rolling back program versions is separate from recovering effects already produced externally.
4. Prevent recurrence, then resume: Fix the cause, add the case to regression evaluations and confirm passing, then reopen permissions gradually.

This requires audit logs, tracing, kill switches and reversible write designs in advance. At organizational level, NIST's AI Risk Management Framework (AI RMF) is a starting point. Designed for voluntary use, it addresses risk through Govern, Map, Measure and Manage, with a separate Generative AI Profile (NIST AI 600-1). Legal reporting and recordkeeping obligations vary by sector and jurisdiction and need separate checking.

Going deeper

Partial success and compensating actions

Work changing several external systems sequentially leaves partial changes if it fails midway. These cannot be canceled together like one database transaction. Define compensating actions—opposite actions offsetting completed effects—for each step in advance. Distributed systems call this the saga pattern. Canceling compensates for creating a reservation; reclaiming credit compensates for granting it.

Compensation has limits. It can fail, and actions such as sending notifications or transferring money can be effectively irretrievable. Design stage order accordingly: easily reversible steps first, irreversible ones last. Send external notifications only after all internal changes are committed. Recording stage commit points and partial-success state speeds incident scoping and recovery.

Deploying without breaking active tasks

Long-running agents change the nature of deployment. Changing tool argument formats or prompts during hours-long work can break active tasks under new code. Anthropic's research system uses rainbow deployment, keeping old and new versions running together and gradually shifting traffic to avoid disrupting active agents.

There are three principles. First, version the model, prompt, tool definitions and policy together as one run configuration, recording the version at task start. Second, let active tasks finish on that version and send only new tasks to the new version. Third, accept both old and new tool argument formats for a transition period. Before a change, shadow execution—running the new version alongside the old without real side effects—can reduce risk by revealing behavioral differences.

Expanding autonomy gradually

Do not enable irreversible tools all at once. Observe behavior in simulated runs, then begin real execution with human approval for every case. Expand automation first in low-risk areas where human correction rates and incident counts stay below defined thresholds. Set numeric conditions for both expansion and reversal. As OpenAI's guide explains, human supervision for high-risk actions remains until trust develops.

At work

Outing Tickets: a retry storm on sale day

The performance booking platform Outing Tickets uses a support agent for reservation lookup, seat changes and cancellation requests. On a popular show's sale day, external booking system latency rose from a normal p99 of 0.8 seconds to 12 seconds. The model SDK retried by default, the tool wrapper tried three more times, and the agent called the same tool again after seeing failure. During one hour, booking system calls reached nine times normal and cost per support case five times normal. Queues delayed ordinary inquiries too.

The team rejected three alternatives. A 60-second timeout would hold connections longer and reduce concurrent capacity. More workers would increase load on the already slow booking system. Lowering max_turns would reduce cost while preserving multiplicative retries.

The fix centralized retries in the tool wrapper, using exponential backoff and jitter with at most two retries. The agent received a recovery instruction: “The booking system is delayed. Do not call again; acknowledge receipt of the request.” Booking calls gained per-tenant concurrency caps. A failure rate above 50% over 30 seconds opened a circuit breaker for two minutes, then a successful trial call closed it. While open, the agent only accepted seat changes and cancellations, sending notifications after processing. Each support case had a 90-second overall deadline and token and tool call limits. Seat changes used idempotency keys derived from reservation numbers and change request numbers, ensuring one accepted request despite retries.

The same delay returned on the next sale day, but booking calls stayed at 1.4 times normal and ordinary inquiry response times barely changed.

Rapids Settlement: fewer approvals, better review

The settlement agency Rapids Settlement piloted an agent identifying and correcting merchant settlement errors. Approval for every correction generated more than 400 requests daily, and average review time fell to four seconds.

The team divided tools by risk. Lookups and report writing were automatic. Corrections at or below KRW 100,000 that would be offset in the next settlement were automatic within a daily limit. Larger corrections and external merchant notices required approval. The execution stage before the correction tool enforced approval, displaying the amount, rationale and trace link on the approval screen. Approval records stored a hash of the correction payload. Different arguments immediately before execution blocked the call and required new approval. Approvals expired after 24 hours. Requests fell to about 30 daily, and actual review lasting over a minute per request became normal.

Per-run turn and cost limits were added. More than five lookups with identical arguments stopped execution and alerted operators. A month later, a source format change caused repeated lookups, but detection stopped the run within three minutes. The operator used a kill switch to disable only corrections, kept lookups running, fixed the parser, added the case to regression tests and reopened corrections. The rejected alternative was adding approvers: with 400 requests, more people would not increase attention per request.

Common mistakes

1. “Approval for every action is safest.” More controls appear safer, but approval fatigue makes review a formality and lets dangerous requests through. Select approval targets by reversibility and scope of impact.
2. “Prohibitions in the system prompt block actions.” Models usually follow instructions, which encourages trust, but instructions are advice. Block irreversible actions on the execution path.
3. “Input guardrails make tools with side effects safe.” A checkpoint at the entrance feels reassuring, but with default concurrent execution, tools may already have run before a guardrail trips.
4. “More retries increase success.” One more attempt seems likely to work, but retries add load during overload, and retries at each layer multiply. Use one layer, backoff and jitter, and a cap.
5. “Turn limits prevent cost overruns.” Knowing the loop ends is reassuring, but turn counts do not catch expensive calls within one turn, subagent proliferation or repeated searches.
6. “A sandbox prevents secrets leaking.” Isolation sounds reassuring, but open reads or broadly allowed domains leave exfiltration routes. Some tools run outside the sandbox too.
7. “Audit logs remove the need for approval.” Having records can feel like control, but logs are for after-the-fact investigation. They cannot restore deleted data.
8. “Once approved, just execute.” Human confirmation may seem to finish the matter, but arguments can change between approval and execution. Bind approvals to execution content, with expiry and single use.

Summary at a glance

Check the official docs

• OpenAI Agents SDK: Running agents — agent loop, max_turns and error handlers
• OpenAI Agents SDK: Guardrails — input, output and tool guardrail coverage and execution
• OpenAI Agents SDK: Human-in-the-loop — needs_approval, interruption and resumption
• Claude Code: Hooks reference — PreToolUse blocking and exit codes
• Claude Code: Configure permissions — rule evaluation order, hook precedence and bypass warnings
• Claude Code: Sandboxing — boundaries, allowed network domains and limitations
• Google ADK: Plugins — global policy, precedence over callbacks and short-circuit rules
• OWASP: LLM06 Excessive Agency — excessive functionality, permissions and autonomy, and mitigations

Self-check

5-1. Give two cost overrun scenarios maximum turns cannot prevent, and a mechanism to stop each.

5-2. What happens when the model SDK, tool wrapper and agent each retry independently, and what design is recommended?

5-3. When should a timed-out tool call not immediately be recorded as failed and resent, and what should happen instead?

5-4. Explain why approval for every action can increase risk, and name the two axes for selecting approval targets.

5-5. In Claude Code, how does PreToolUse blocking with exit code 2 relate to allow and deny permission rules?

5-6. Name two things Claude Code's sandbox does not cover, and state how to control them.

5-7. Set a per-user daily agent cost limit, and design the system's actions at 80% and 100% of that limit.

Hands-on tasks