Chapter 4. Responsible use and data

Learning objectives

• Explain common prohibited categories in major AI companies' usage policies and requirements for human review and disclosure in high-risk domains.
• Distinguish consumer and business plans in training use, retention, human review and administrator access, and describe administrator controls.
• Decide whether sensitive information may be entered into AI, minimizing it and reducing identifiability when it is necessary.
• Recognize hidden instructions in material and verify AI output by checking sources claim by claim.
• Include copyright, bias and disclosure of AI use in checks before external publication.

Introduction

Moon, on a logistics company's planning team, was rushing to prepare a client briefing on Friday afternoon. Logging into the company-approved AI tool seemed troublesome, so Moon pasted a table containing client contract terms and contacts into a personal AI phone app. The output was excellent and the presentation went well. But the following week's security training presented the same situation as an example of what not to do. Moon felt this was unfair: nothing was wrong with the result.

The mistake was not using AI. It was failing to assess what material was entered, into which environment and in what quantity. Moon had not asked whether the personal app's data settings offered protection equivalent to the company contract, whether unnecessary personal information was included, or who could see conversations in that environment. The incident had already occurred at the moment of pasting, regardless of the output.

This chapter addresses that judgment. It first examines common prohibitions in AI companies' policies and conditions for high-risk uses, then separates data storage and use into five dimensions. It covers decisions about sensitive input, identifiability, hidden instructions, source checking, fact verification, copyright and bias. This chapter corresponds to Diligence in AI fluency.

Key concepts

Common prohibited categories in usage policies

A usage policy defines what an AI company prohibits on its service. OpenAI, Anthropic and Google use different wording but have similar underlying categories.

These are not identical policies. Category names, exceptions and additional conditions differ, so read the original policy for the service you use. OpenAI also states that its rules do not replace legal requirements, professional obligations or ethical responsibilities. Permission under a usage policy does not imply permission under law or company rules. Usage policies, law and internal rules are three separate checks.

High-risk domains: meaningful human review and disclosure

High-risk decisions are the category most frequently encountered at work. Comparing the three companies' wording reveals a common principle.

All three require human review or oversight; Anthropic explicitly specifies qualifications and AI disclosure. This book combines them into two practical principles. First, AI output in these domains is decision-support material only; decisions follow direct examination by a qualified person. Clicking "Approve" without opening the result does not constitute review. A high measured accuracy does not waive this principle. Second, tell affected people that AI was used.

Do not conceal that it is AI

Anthropic requires customer-facing chatbots to tell users that they are communicating with AI rather than a person, and lists fake reviews and bots claiming to be human as prohibited examples. Google prohibits deceptively misrepresenting generated content as solely human-created or impersonating real people without disclosure.

Korean law contains a corresponding provision. Article 31 of the Framework Act on the Development of Artificial Intelligence and the Establishment of a Foundation for Trust, referred to here as the AI Basic Act, requires AI business operators offering products or services using high-impact or generative AI to inform users in advance, label generative AI outputs as generated, and clearly disclose or label synthetic audio, images and video that are difficult to distinguish from reality. Article 2(4) defines high-impact AI as AI systems that may significantly affect or pose risks to human life, physical safety or fundamental rights and are used in listed domains, including judgments such as hiring and loan screening.

These duties fall on providers of AI products and services, but companies introducing customer chatbots or AI-generated content should check the provision with legal staff. Start from the principle of not hiding AI where knowing about it could change the other person's behavior.

Five dimensions of data

The most common confusion about data is treating several settings as one. In reality, at least five dimensions operate independently.

Safety on one dimension does not ensure safety on the others. "Not used for training" answers only the first; it says nothing about administrator access or storage duration. Also examine paths outside chat. Attachments may remain in projects, feedback may include a whole conversation, and saved memory may outlast the conversation. Shared-link visibility also differs by product and plan; for example, ChatGPT Business links can be opened only by members of the same workspace.

Consumer and business plans

Consumer plans are free or paid subscriptions individuals obtain themselves; business plans are contracted by organizations for their members. The following key details were checked in official documentation on October 20, 2026.

Google explicitly advises Gemini Apps users not to enter confidential material they would not want reviewers to see.

Administrator controls

Administrator controls let an organization's administrators configure and supervise AI tools centrally. This is a major reason to choose business plans. Official documentation describes controls such as these.

Administrator controls protect organizational material, but also mean administrators may see members' conversations. "Business means nobody can see it" is mistaken. Exclusion from training and exclusion from administrator access are different dimensions.

Deciding whether to enter sensitive information

Before entering material, consider purpose and environment as well as information type. Ask four questions in order.

1. Is this information essential to the task? Remove phone numbers if the summary does not need them.
2. Has the company approved this environment? Check whether the account is personal or organizational.
3. Can the scope be reduced? Include only necessary columns and time periods.
4. Do contracts or internal rules prohibit it? Client confidentiality agreements also apply to AI tools.

A traffic-light classification speeds decisions.

Data minimization means excluding information unnecessary for the task. Traffic lights are only a starting point. Company rules should identify whom to consult for yellow or red material, such as privacy or security officers.

Identifiability: removing names is not enough

Article 2 of Korea's Personal Information Protection Act defines personal information about a living individual to include not only directly identifying data such as names and resident registration numbers, but also information that can identify someone when easily combined with other information. Ease of combination is judged reasonably by considering the availability of other information and the time, cost and technology needed for identification.

The practical message is clear. Remove names but retain department, position, joining year and region, and colleagues may still identify people easily. Free text such as "Since I returned from parental leave last month" can also provide clues.

The Act separately addresses sensitive information, including beliefs, union or political-party membership, political views, health and sex life (Article 23); unique identifying information assigned by law to distinguish individuals (Article 24); and resident registration numbers (Article 24-2). Their processing is restricted in principle. Always consult the responsible officer before entering such information into AI.

Pseudonymization removes or replaces part of personal information so a particular individual cannot be identified without additional information (Article 2(1-2)). Pseudonymized information remains personal information under the Act; pseudonymization does not permit unrestricted AI input. Practical ways to reduce identifiability include the following.

In August 2025, Korea's Personal Information Protection Commission published guidance on personal-information processing for generative AI development and use. It divides the lifecycle into purpose setting, strategy development, training and development, and application and management, recommending safeguards for each stage. It is useful reading when an organization builds or adopts AI services.

Hidden instructions in documents: prompt injection basics

Prompt injection is a vulnerability in which input text changes a model's behavior in unintended ways. OWASP's 2025 generative-AI risk list places it first and distinguishes direct injection, through user-provided input, from indirect injection, through outside material such as websites and files. OWASP warns that text can influence a model even when invisible to people. Hidden webpage text and small text within images are still readable input to a model.

Indirect injection is especially relevant to non-developers as AI increasingly reads outside email, webpages and supplier documents. Risk grows with what the AI can do. Even summarization is not entirely safe: OWASP describes a case where summarizing a malicious webpage caused a model to embed an externally linked image in its output, leaking conversation content. AI that can send mail or move files increases the possible damage. Users and administrators should understand these recommended defenses.

1. Clearly separate outside material. Label it as untrusted to reduce its influence, as discussed in Chapter 2.
2. Grant least privilege. Provide only access necessary for the task.
3. Require human approval for high-risk actions. Confirm before sending, paying or deleting.
4. Report unusual results. Notify the designated contact if summaries include unrelated recommendations or requests.

Checking sources and verifying facts

Compare AI output with original sources before sending it outside the organization. An effective procedure is as follows.

1. Separate claims. List factual assertions, numbers and quotations in the output.
2. Require evidence for each claim. Request source names and relevant passages or page numbers.
3. Open the sources yourself. Check that they exist, the passages support the claims, and dates, subjects and conditions match.
4. Recalculate numbers. Recompute ratios and totals from original data.
5. Remove unsupported claims or mark them "Needs checking."

A source's existence and its support for a claim are different things. A real article may contain another year's statistics or be a secondary source quoting something else. Trace back to primary material where possible: original statistical tables, legal text or official company announcements.

Some actions resemble verification but are not: asking the same model "Is that right?", trusting fluent prose, or trusting an answer because another person's AI-generated result reaches the same conclusion. None provides external evidence.

Copyright and attribution are different questions

Check copyright separately before using AI output externally. Naming a source does not grant permission, and rewriting with AI does not remove rights in the original expression. Ask three questions.

1. Do you have the right to use the input? Reworking someone else's manuscript or paid report remains subject to the original material's terms.
2. Does the result contain someone else's expression? Check before use if it closely resembles a specific work or phrase.
3. Can the result be protected as your work? Article 2 of Korea's Copyright Act defines a work as a creative production expressing human thoughts or emotions. Policy guidance from the Korea Communications Commission likewise explains that automatically generated AI output is not protected under current copyright law; only portions containing added human creative expression may qualify. Eligibility and scope are assessed case by case.

AI companies' terms often assign output rights to users. OpenAI likewise states that business customers own outputs to the extent permitted by law. This is an agreement between provider and user, not a guarantee of legal protection or non-infringement. Seek legal review before using costly, long-lived outputs such as logos, trademarks or publications. The Korea Copyright Commission's collection of four AI-copyright guides is another reference.

Checking bias and stereotypes

Models can reproduce social biases in training data, associating occupations with particular genders or portraying regions and age groups through fixed images. What seems minor in an internal draft can cause discrimination or reputational problems in recruitment notices, advertising or customer communications.

Check bias alongside factual accuracy. Before external publication, ask whether depictions of gender, age, disability and background reflect the actual audience, whether expressions generalize about groups, and whether evaluations of people or their scores systematically favor particular groups. If bias appears, state the desired composition in the instructions and have people select and revise the output.

Going deeper

Reading beyond "not used for training"

The most prominent line in a data policy is often "We do not use customer data for training." Stopping there misses four things.

1. Scope: Which plans and products does the statement cover? Personal and business plans from the same provider often differ.
2. Defaults and exceptions: "By default" has exceptions, often including explicit sharing consent or feedback submissions.
3. Other dimensions: Separately check retention periods, safety retention, human review and administrator access.
4. Connected paths: Check whether attachments, connected apps, memory and shared links follow the same rules.

Policies change frequently. When copying data-policy information into internal guidance, include the original link and verification date, then recheck periodically.

The first 30 minutes after an incident

The worst response to discovering sensitive material in the wrong environment is concealment or indiscriminate deletion. Concealment costs the organization response time; deletion removes evidence of what was exposed. Follow an ordered response.

1. Stop. Halt further input, sharing and agent execution.
2. Record the scope. Note what material was entered, when, and into which account and service.
3. Report. Truthfully notify security or privacy staff through internal procedures. If it constitutes a personal-information breach, Article 34 of PIPA requires the organization to notify affected individuals and report without delay; delayed internal reporting also delays compliance.
4. Clean up as directed. Follow the responsible officer's instructions for unsharing and deleting conversations and saved memory. Deletion from your screen may not immediately end provider retention.
5. Prevent recurrence. Make approved tools easy to use and improve pre-input checks.

At work

Solid Trading's sales administration team: a client evaluation report

Jang, a sales administration manager at industrial-parts distributor Solid Trading, prepares a quarterly client evaluation. The material consists of a contract register containing client contacts and the previous quarter's delivery performance.

Jang first applies traffic lights: delivery records are yellow internal material; contact details and contract unit prices are red. Since a company-approved business AI environment exists, personal accounts are excluded. Next comes minimization. The evaluation needs only client names, on-time delivery rates and defect rates, so Jang creates and uploads a copy without contact columns. Contract prices are absent from the evaluation criteria and are omitted entirely.

After receiving a draft, Jang asks, "Next to each client evaluation statement, state the supporting row number in the performance table." One client's defect rate is understated; tracing the row reveals another client's value was mixed in. Jang corrects the sentence and deletes an unsupported comparison with the industry average because no supporting row or source can be found.

Finally, recognizing that the report could inform consequential decisions such as reducing business, Jang states at the beginning that scores are reference material only and final decisions will be made by people at the team leaders' meeting. The AI-assisted portions are disclosed too.

Straightline Construction's estimating team: pasting into a personal account

Hong, a new estimator at interior-construction company Straightline Construction, wants to improve a quotation-explanation email on the subway home. Hong pastes the quotation into a free AI app on a personal phone. It includes the customer's name, home address, contact information and construction price. Reading security rules the next morning reveals the mistake.

Hong first considers deleting the conversation and telling nobody, then rejects this: deletion may not immediately end provider retention, and concealment deprives the company of an opportunity to take necessary action for the customer. Hong next considers turning off model improvement, but this is insufficient: that changes only training use and cannot undo retention or exposure of already entered material.

Before deletion, Hong records the scope—the entire quotation for one customer—the time, app name and account type, then informs the team leader and privacy officer. The officer checks the app's settings and retention policy in official help, directs conversation deletion and confirmation that no feedback was submitted, and creates an internal record. The team then makes approved tools accessible on phones and switches to quotation templates with customer-information fields removed for AI use. Quick reporting and accessible approved tools reduce incidents more effectively than punishment.

Common mistakes

1. "Turning off training is equivalent to a business plan." Training is only one of five dimensions. Retention, human review, administrator controls and contractual protections are separate.
2. "Temporary chat leaves no record anywhere." It may still be retained for safety. ChatGPT states up to 30 days; Gemini Apps states 72 hours.
3. "Removing names makes data anonymous." Data remains personal information if easily combined with other information to identify someone. Consider contacts, addresses, positions and free-text clues together.
4. "High accuracy permits automatic decisions." High-risk areas require meaningful review by qualified people regardless of accuracy, except for narrow policy-defined exceptions.
5. "Even administrators cannot see business chats." Business plans default to no training use, but organizational administrators may access conversations under their policies.
6. "Hidden instructions do not matter for summaries." Distorted summaries can harm, and action permissions increase risk. Separate external material and grant least privilege.
7. "If the source link opens, it is verified." A real source may not support the claim. Compare claim by claim.
8. "Terms give me output rights, so using it as a trademark is safe." Rights clauses do not guarantee protection or non-infringement. Obtain legal review for long-term uses.

Summary at a glance

• Common prohibited categories are illegality and violence, child safety violations, privacy violations, fraud and deception, security violations, and high-risk decisions without human review. Check policies, law and internal rules separately.
• In high-risk domains, qualified people examine content and decide, and affected people are told about AI use.
• Customer chatbots must identify themselves as AI; fake reviews and misrepresented provenance are prohibited. Article 31 of the AI Basic Act sets disclosure and labeling duties for businesses.
• Separate training use, retention, human review, administrator access and deletion. Safety in one dimension does not ensure safety in another.
• Disabling training in a personal account does not replace business protection. Feedback may be used even after training is disabled.
• Ask four questions before input: Is it essential? Is the environment approved? Can it be reduced? Do contracts or rules prohibit it?
• Personal information includes data identifiable through easy combination with other information. Reduce identifiability through removal, replacement, grouping and invented material.
• Prepare for hidden external instructions with clear boundaries, least privilege and approval for high-risk actions.
• Verify claim by claim: open sources, compare passages, recalculate numbers, and remove unsupported claims or mark them "Needs checking." Chapter 1's grounding does not replace this.
• Copyright is separate from attribution; check bias too before publication.

Check the official docs

• OpenAI — Usage policies: Prohibited categories and automation of sensitive-domain decisions without human review.
• Anthropic — Usage Policy: Human review and disclosure for high-risk uses, and AI disclosure for chatbots.
• Google — Generative AI Prohibited Use Policy: Restrictions on automated high-risk decisions and misrepresented provenance.
• OpenAI — Data controls in ChatGPT / Enterprise privacy: Personal training settings, temporary chats, feedback, business no-training defaults and administrator access.
• Anthropic — Is my data used for model training?: Personal Claude training use and Incognito chats.
• Google — Gemini Apps Privacy Hub: Human review and retention periods.
• OWASP — LLM01:2025 Prompt Injection: Direct and indirect injection and defenses.

Self-check

4-1. Name three prohibited categories common to all three companies' usage policies.

4-2. Name the five independent dimensions to consider when assessing an AI service's data handling.

4-3. Classify these as green, yellow or red: a recruitment notice on the company website; an unpublished price list for products launching next quarter; a list of employee health-check results.

4-4. A table of annual-leave usage by team has names removed but retains department, position and joining year. Explain why it may still be personal information using PIPA's definition.

4-5. Give one user action that may allow a conversation to be used for training even after model improvement is disabled on a personal account. Explain why it creates an exception.

4-6. An AI-generated market-analysis memo cites a real news article. Give two situations where opening the link does not complete verification.

4-7. Design five lines of AI-use rules for a company with 30 employees. Include approved tools, prohibited input, verification, high-risk decisions and incident reporting.

Hands-on tasks