| Guardrail |
A mechanism that inspects input, tool results, tool calls or output to block or modify them |
| Redaction |
Removing sensitive values or replacing them with asterisks or similar marks |
| Pseudonymization |
Processing that prevents identification of a specific person without additional information |
| Indirect injection |
Instructions in documents, web pages, emails or tool results the model reads for the user change its behavior |
| Audit log |
Records of who did what, when and where, suitable for incident investigation |
| Retrieval-augmented generation (RAG) |
Answering with retrieved document chunks added to context |
| Boundary case |
An evaluation case testing judgment criteria, such as a legitimate request that resembles an attack |
| Attack success rate |
The proportion of attack attempts that pass defenses and achieve their goal |
| Attack surface |
All entry points where untrusted external content can reach the model or system |
| Supply chain |
The path of externally acquired components: models, weight files, adapters, packages, plugins, MCP servers and more |
| Excessive agency |
A vulnerability where excess functionality, permissions or autonomy turn wrong output into harmful actions. OWASP LLM06:2025 |
| Rules of engagement |
Agreement on red-team targets, environment, data, allowed/prohibited actions, stop conditions and reporting |
| Cross-context leakage |
One group's data reaches another through a store shared by multiple user groups |
| Permission-aware retrieval |
Returning only documents authorized for the requester at retrieval time |
| Answer contamination |
Recommendations, summaries or judgments are distorted to align with the attacker's intent without data leakage |
| Data flow diagram (DFD) |
A system diagram of external entities, processes, stores, flows and trust boundaries |
| Red teaming |
Structured testing that deliberately attacks one's own system within authorized scope to find flaws |
| Memory poisoning |
Poisoning stored summaries, memory or indexes to distort later decisions. OWASP ASI06 |
| Model extraction |
Attempts to replicate model functionality by collecting output through bulk queries |
| Goal hijack |
External content or forged messages change an agent's goal itself and redirect subsequent stages. OWASP ASI01 |
| Harmlessness screening |
A pattern using a lightweight model to classify input before it reaches the main model |
| Unbounded consumption |
Uncontrolled inference requests cause denial of service, financial loss or model extraction. OWASP LLM10:2025 |
| Improper output handling |
Failure to validate or encode model output before a screen, DB, shell or other downstream component interprets it. OWASP LLM05:2025 |
| Classifier |
A guardrail's decision engine: rules, dedicated classification models or LLM screening |
| Human confirmation (HITL) |
A control requiring human approval before high-impact actions execute |
| State handle |
An identifier issued by an MCP server to reference state across calls; not an authentication method |
| Server-Side Request Forgery (SSRF) |
An attack causing a server to request an attacker-chosen address and reach internal resources |
| System prompt leakage |
Exposure of secrets or control logic in a system prompt. OWASP LLM07:2025 (broadened to Hidden Context Exposure in 2026) |
| Trust boundary |
A point where data or components with different trust levels meet |
| Cascading failures |
One flaw spreads and grows across agents and pipelines. OWASP ASI08 |
| False-positive rate |
The proportion of legitimate requests incorrectly blocked by defenses |
| Complete mediation |
Downstream systems, rather than the model, authorize every request to them |
| Risk register |
A table recording threats, paths, classifications, decisions, residual risks and owners, and review triggers |
| Residual risk |
Risk remaining after controls are applied |
| Embedding inversion |
An attack reconstructing significant portions of source text from embedding vectors |
| Asset |
Data, credentials, ability to act or resources whose damage would cause problems for the organization |
| Denial of Wallet |
An attack inflating metered AI charges to impose financial loss on the operator |
| Direct injection |
Application user input changes model behavior |
| Containment switch |
A mechanism stopping execution per tool, connection, feature or task during incidents |
| Least-Agency |
The principle of avoiding agent autonomy where it is unnecessary |
| Canary |
A unique fake secret planted in an evaluation set to detect leakage |
| Jailbreak |
A form of prompt injection aimed at making a model ignore safety protocols |
| Token passthrough |
Passing received access tokens to downstream APIs without verifying that the service itself is the intended audience; an antipattern prohibited by MCP authorization specifications |
| Tokenization |
Replacing sensitive values with meaningless substitute strings and storing the mapping separately |
| Tripwire |
A guardrail signal that stops execution on detecting a violation (OpenAI Agents SDK term) |
| Judge model |
A model used to grade another model's output (LLM grader) |
| Payload splitting |
Splitting instructions into pieces that each appear harmless |
| Handoff |
A call transferring conversation and work from one agent to another (OpenAI Agents SDK term) |
| Confused deputy |
A privileged intermediary is tricked into exercising authority for another party |
| Regression evaluation |
Rerunning test cases from past incidents and security findings after changes to check that the same flaws are absent |
| Circuit breaker |
Temporarily blocking calls after repeated identical errors to break a chain |
| AI BOM |
An inventory of AI components such as models, datasets, adapters and configurations; CycloneDX ML-BOM is one format |
| MCP |
Model Context Protocol, an open protocol through which applications connect tools and data to models |
| MITRE ATLAS |
MITRE's public resource cataloging adversary tactics, techniques and real cases targeting AI systems |
| NIST AI RMF |
NIST's AI risk management framework released in January 2023, with four functions: Govern, Map, Measure and Manage |
| NIST AI 600-1 |
AI RMF's Generative AI Profile, covering generative-AI-specific risks and recommended actions |
| OWASP Agentic Top 10 |
Ten agent-specific risks (ASI01–ASI10), 2026 edition |
| OWASP LLM Top 10 |
Ten major risks for LLM applications; this book and exam use 2025 numbers, with a 2026 crosswalk in Chapter 1 |
| Prompt Shields |
Microsoft's classification service detecting user prompt attacks and document attacks |
| safety_identifier |
A hashed user identification string attached to requests for per-user abuse tracking (OpenAI) |
| safetensors |
Hugging Face's format designed for safe tensor storage, unlike pickle |
| SAIF |
Google's Secure AI Framework: six core elements and a risk map |
| STRIDE |
Microsoft's threat classification: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege |
| TOCTOU |
A mismatch between check and use times that leaves intervening condition changes unaccounted for |