Chapter 4. Organizational policy and controls

Learning objectives

• Create an AI use inventory, and use it to design separate layers of policy, standards, procedures and guidance.
• Design allowed, conditional and prohibited categories by combining data level, tool, impact of use and action permissions.
• Create risk-proportionate approval procedures and exceptions with expiry dates.
• Implement rule conditions through admin controls (accounts, retention, feedback, organizational units, web search and agents), and verify their behavior.
• Separate permissions for agents and connected features, and reduce shadow AI through training and alternative tools.

Introduction

Suppose a security team discovers in proxy logs that many employees visit unapproved AI sites. Blocking is the fastest response. A week later, however, employees start accessing the same sites on personal phones, and the company can no longer even see the usage. Employees wanted to meet deadlines, not break rules, and the company provided no approved route meeting that need. The problem lay in the design. This chapter addresses that design.

An organization's AI rules should explain how to work, rather than merely list prohibitions. The goal is to clearly enable usable tools and methods, raise barriers only where risk is greater, and enforce conditions through system settings rather than human memory.

This chapter starts with understanding current use, the first step in governance. It then covers rule layers and structure, category design, approvals, exceptions and accountability, before moving to admin controls, permission cleanup, agents, training and supply chain management. Vendor terms from Chapter 1 and legal requirements from Chapters 2 and 3 become the material for these rules. Adapt the examples to your data levels and work rather than copying them unchanged.

Key concepts

The first step in governance: an AI use inventory

You cannot make rules without knowing what is used. NIST AI RMF calls for mechanisms to inventory AI systems and allocate resources according to organizational risk priorities (GOVERN 1.6). An AI use inventory brings the organization's AI use into one table.

Look for AI features added to existing SaaS, not only newly purchased tools. Summarization and generation quietly enabled through updates are common blind spots. The inventory need not be perfect initially. Build it by combining department surveys, purchase records and network logs, and connect tool request procedures to inventory updates so it stays current.

Policy, standards, procedures and guidance

Putting everything in one document makes it hard to read and revise. Separate layers let principles remain stable while responding quickly to product changes.

For example: "Personal information may be processed only in approved tools" is policy; "Approved tools must have no training use, fixed retention and feedback restrictions" is a standard; "Review new tool requests within five business days" is a procedure; and "Remove customer names before pasting" is guidance.

The structure of an AI use policy

An AI use policy needs at least five parts.

1. Purpose and scope: Who is covered (including whether temporary and partner staff are covered), and which tools and tasks does it apply to?
2. Data levels: Public, internal, confidential, personal information, sensitive information, etc. Prefer existing information security classifications.
3. Approved tool list: Products and plans with verified data terms, and permitted features.
4. Use categories: Allowed, conditional or prohibited for combinations of data level, tool and purpose.
5. Responsibilities and procedures: Output review responsibility, tool requests and exception approvals, incident reporting and responses to violations.

Add a reason and alternative to each rule to improve compliance. After "Do not enter customer information", add "Conversations may be used for training. Use approved tool A if needed."

Designing allowed, conditional and prohibited categories

This chapter's practical skill is designing allowed, conditional and prohibited use categories. Tool names alone leave gaps: risk differs by input data and purpose even in the same tool. Consider four axes together.

First set a basic category from data and tool combinations.

Then raise the category based on purpose and action permissions. Add human review if outputs go directly to customers. For decisions concerning individuals, such as recruitment, assessment or lending, require prior risk assessment, Chapter 2 high-impact review and Chapter 3 automated decision review. If AI transmits data externally or takes hard-to-reverse actions, require approval before action. Examples of category-changing conditions help employees judge for themselves.

Risk-proportionate approval procedures

If every application requires the same written approval process, even simple requests take weeks, encouraging shadow AI. A common approach uses three routes.

Design approval steps so each output becomes the next input. Define purpose and data level; assess vendor terms and risk accordingly; set conditions from the assessment; implement them in admin settings; train users and deploy; reassess using operational records. Record the approval scope (work, data, users, features and period) so one approval does not quietly expand to other data or uses.

Exception management

Necessary uses outside the rules inevitably arise. Blocking exceptions drives hidden use; granting them too easily undermines rules. Good exception procedures have four components.

1. Application details: Requester, purpose, data level, period and alternatives considered.
2. Compensating controls: Measures reducing added risk, such as masking, limited users and log reviews.
3. Expiry and reclamation: End date, methods for deprovisioning accounts and reclaiming data at expiry, and extension review conditions.
4. Feedback from repeated exceptions: Repetition signals a need to revise rules or approved tool lists.

An exception without an expiry date is permanent permission outside the rules.

Accountability: who decides and who reviews?

Responsibilities blur as more departments participate. NIST AI RMF calls for documented roles, responsibilities and communication lines for mapping, measuring and managing risk (GOVERN 2.1), and executive responsibility for risk decisions in AI development and deployment (GOVERN 2.3).

Follow two principles. First, separate the system developer from the person authorized to accept risk. Developers prepare assessment materials because they know the system best, but a higher authority decides risk acceptance after independent review. Second, raise acceptance authority with risk level. Assigning low risk to department heads, medium to committees and high to management speeds decisions and preserves records. Review departments (legal, privacy and information security) provide opinions; decisions and reasons are recorded under the approving authority's name.

Admin controls

Implement rule conditions in admin settings wherever possible. Settings are more reliable than memory. Official documentation gives the following examples.

After changes, verify actual behavior with an ordinary user account, and record when and by whom settings changed. Overlapping group and OU settings, as in Gemini app, can produce unintended results through precedence rules. Limit admin rights to the minimum necessary personnel and review admin activity regularly.

Permission cleanup and oversharing

AI searching internal data can find everything a user is permitted to see. Microsoft explains that Copilot shows only organizational data for which a user has at least viewing permission, and recommends correct use of permission models in services such as SharePoint. The issue is files people could access but never looked for. Folders mistakenly shared company-wide were merely hidden from attention; AI search can immediately surface them in answers.

Make permission cleanup a prerequisite for AI search rollout. Locate company-wide sharing links and "Everyone" permissions, apply sensitivity labels (classifications enforcing access and encryption) or restrictions to sensitive locations, confirm search indexes respect source permissions (Chapter 3, Going deeper), and repeat checks after rollout. This controls AI while addressing longstanding information security debt.

Controls for agents and connected features

Agents do more than generate answers: they invoke tools to retrieve, modify, send and pay. OWASP Top 10 for LLM Applications (2025) includes prompt injection (LLM01) and Excessive Agency (LLM06) among major risks. Indirect prompt injection, where instructions hidden in external documents or email change AI behavior, is a typical example; excessive permissions magnify harm.

Controls fall into five groups.

1. Least privilege: Connect only the tools and scope needed. Separate read and write permissions.
2. Confirmation before action: Require human approval before hard-to-reverse actions such as external sending, payment, deletion or permission changes.
3. Limits: Enforce amount, volume and target-scope ceilings in systems independently of the model.
4. Records: Log which inputs led to which tool calls and arguments.
5. Stop and recover: Provide an immediate stop switch and reversal procedures for abnormal behavior.

Vendor usage policies point the same way. Anthropic's current policy requires agentic uses to comply too. The revision effective 12 November 2026 states that users building or deploying agents are responsible for actions through tools, browsers and connected systems as well.

Training and shadow AI

Shadow AI is processing work data through AI accounts or tools the organization has not approved. Convenience, rather than malice, usually motivates it: no approved tool, a slow application process or unclear permissions.

An effective response combines four things: make approved tools easy to access, communicate short and concrete rules, train through role-specific cases, and remeasure usage. Training should cover data classification, approved tools and reasons, output verification responsibility, what feedback buttons mean and how to report mistakes. Reducing adverse consequences for self-reported mistakes helps discover incidents early. Since tools and terms change often, short training on introduction and changes is more effective than annual group sessions. Define survey boundaries to avoid excessive collection of employee prompt content. That is personal information processing too.

Supply chain and third-party management

Most AI comes from outside. Models, APIs, datasets and AI features embedded in SaaS are all supply chain components. NIST AI RMF calls for third-party AI risk policies and procedures (GOVERN 6.1), and regular monitoring of third-party resource risks and benefits (MANAGE 3.1).

Supplier due diligence should cover Chapter 1's five questions, security certifications, incident notification terms, subprocessor change notices, model change and end-of-support communications, and return or deletion at contract end. A supplier's management system certification such as ISO/IEC 42001 describes its management framework; it does not replace the organization's responsibility to assess risks of its own use cases (Chapter 5).

Going deeper

Formal approval and automation bias

Many rules require final human review, but whether that review works needs separate checking. Automation bias is the tendency to trust AI suggestions excessively and accept them without review. NIST's Generative AI Profile identifies overconfidence, automation bias and over-reliance in human–AI interaction as "Human-AI Configuration" risks.

Effective review gives reviewers sufficient information and time, authority and responsibility to change outcomes, and measures whether review occurs. If unchanged-approval rates, review times or errors found afterward are abnormal, redesign review to focus deeply on higher-risk cases or double-review random samples. A perfunctory click is also unlikely to count as meaningful intervention for Chapter 3's automated decisions.

Changing and retiring AI systems

AI incidents often arise during change or retirement rather than adoption: a vendor ends model support, automatically upgrades models or closes a service. NIST AI RMF calls for safe decommissioning procedures (GOVERN 1.7), and mechanisms and responsibilities for replacing, disengaging or deactivating systems with unintended performance or outcomes (MANAGE 2.4).

On changes, ask whether prior assessments remain valid. Public benchmark scores do not guarantee performance on your work, so revalidate high-risk systems with your own work data. At retirement, use a checklist for revoking connections, notifying users, separating retention from destruction (retain legally required records separately), and fallback operating procedures.

At work

Cheongram University: creating staff AI use rules

Cheongram University, with 12,000 students, decides to create AI use rules for faculty and staff. Academic affairs wants AI for grade processing and syllabuses, research administration for project reports, and admissions for applicant guidance. A one-month IT survey finds widespread personal-account use alongside the university's business plan. Some instructors paste entire student assignments for grading comments.

The task force reuses existing security classifications in four levels, classifying grades and counseling records as personal information. Assignment grading assistance is conditional: use an approved plan, remove student IDs, names and identifiers in the text, and leave final grades to instructors. Removing names alone is insufficient because assignment content can identify students. The admissions chatbot answers external users directly, so it takes the enhanced route, with AI disclosure, answer-scope restrictions and human handoff conditions.

Research administration objects to blanket rules because overseas collaboration contracts differ. The task force handles research data through project-specific exceptions, requiring contractual conditions and expiry dates in approval documents. It drops a blanket personal-account ban, opens approved university-account tools to all staff first, and plans to measure personal-account use again after one semester.

Admin settings implement SSO, retention periods and disabled feedback submission. The admissions chatbot has web search disabled and uses only official guidance. During the first month, each department receives ten-minute case-based training.

Acorn Games: a support agent handling refunds

Online game company Acorn Games wants to add payment lookup, refund execution and game-item grants to its customer support agent. Customer service wants shorter queues; security worries that malicious instructions could cause mass refunds or item grants.

The information security officer breaks capabilities into actions and creates a permissions table. Payment history lookup is read-only and limited to the requesting customer's account. The agent can execute only small refunds eligible for automatic approval under policy; larger refunds and refusals require a human. This avoids adverse decisions without human involvement (Chapter 3, Automated decisions). Item grants are limited to a predefined list such as outage compensation, with daily volume and per-account quantity limits enforced by the payment system. Prompt-only limits are rejected because prompt injection can disable them.

Conversations containing customer image attachments or external links cannot use action tools and are handed to staff, preventing hidden instructions from changing agent actions. Every tool call is logged with input, tool, arguments and result. A switch automatically disables action tools when refund volume exceeds three times normal. Customers are told on the opening screen that they are speaking to an AI agent.

Before launch, the task force tests conversations deliberately containing malicious instructions to verify that the permissions table actually blocks them. The outcome is "Lookup allowed; small refunds conditionally allowed; other monetary actions require human approval", with a review of scope after three months of logs.

Common mistakes

1. "A blanket ban is safest." A ban without alternatives increases hidden use.
2. "Making every use equally conditional is fair." Disproportionate controls miss what needs blocking and block what could be allowed.
3. "Write rules first, check use later." Without knowing current use, rules miss reality. Start with the inventory.
4. "Developers know best, so they decide risk acceptance too." Developers prepare evidence; a higher authority accepts risk after independent review.
5. "Turning it off for an OU always blocks it." Gemini app configuration group settings override OU settings.
6. "AI shows only permitted data, so permission cleanup is unnecessary." Incorrectly shared files immediately surface in answers. Clean up before rollout.
7. "Put agent prohibitions in the prompt." Systems outside the model must enforce limits and approval.
8. "Clicking Approve means human review." Approval without information, time, authority or measurement is a formality.

Summary at a glance

• Governance starts with an AI use inventory. Find AI features added to SaaS too.
• Separate policy, standards, procedures and guidance; revise principles slowly and guidance quickly.
• Set categories by data level × tool conditions × impact of use × action permissions.
• Separate simplified, standard and enhanced approvals, and record scope. Exceptions need expiry dates and compensating controls.
• Separate builders from risk acceptors, and assign acceptance authority by risk level.
• Implement rule conditions in admin settings and verify behavior with ordinary user accounts.
• Clean up permissions before deploying AI search.
• Agents need least privilege, pre-action confirmation, system-enforced limits, logs and a stop switch.
• Reduce shadow AI with approved tools, short rules, role-specific training and remeasurement.
• Supplier certification does not replace our risk assessment.

Check the official docs

• Google Workspace Admin Help — Turn the Gemini app on or off: OU and group settings, precedence and conversation retention.
• Anthropic — Configure custom data retention controls for Enterprise plans: Custom retention periods and permissions.
• Microsoft Learn — Data, privacy, and security for web search in Microsoft Copilot: Admin web search policy and user toggle.
• Microsoft Learn — Data, Privacy, and Security for Microsoft Copilot: Permissions model, agent management and Purview retention.
• NIST AIRC — AI RMF Core: GOVERN 1.6, 1.7, 2.1, 2.3, 6.1 and MANAGE 2.4, 3.1.
• OWASP — Top 10 for LLM Applications: Prompt injection and excessive agency.

Self-check

4-1. List at least five AI use inventory columns, and identify the AI use most easily overlooked when building it.

4-2. Use one example (such as entering customer personal information) to show the difference among policy, standards, procedures and guidance in one sentence each.

4-3. Give two cases where the use category should rise despite using the same approved tool, considering purpose and action permissions.

4-4. List at least five mandatory exception approval items, and state what the responsible staff member should do about repeated identical exceptions.

4-5. Explain why admins should recheck behavior with ordinary user accounts after changing AI service settings, using overlapping settings as an example.

4-6. Describe the characteristics of agent actions needing prior human approval, and explain why limits belong in systems rather than prompts.

4-7. (Design) Invent a 300-person company and build a category table with four data levels and two tool types. Mark whether each conditional cell can be implemented through admin settings.

Hands-on tasks