Chapter 1. Audit Planning, Scoping, and AI Inventory

Learning objectives

  • Select items to include in an AI usage inventory according to the audit objectives.
  • Identify gaps in self-reported inventories using independent records matched to usage pathways.
  • Distinguish between what audit must do and what management must do when shadow AI is found.
  • Prioritize AI system risks based on impact and likelihood, and determine the audit scope and depth.
  • Map the AI Basic Act, the Personal Information Protection Act (PIPA), NIST AI RMF, and ISO/IEC 42001 as audit criteria.
  • Explain decisions that must be made during the planning phase, such as assurance versus advisory, objectivity, and the use of specialists.

Introduction

Consider a scenario where the chief audit executive of a mid-sized manufacturer received this question at a board meeting: "Everyone seems to be using AI these days. Are our company's AI risks fine?" The chief audit executive knew that there were only four approved tools on the inventory managed by the IT team. However, the previous week in the cafeteria, employees from the sales team were overheard comparing the writing AI tools each of them used, and the procurement team stated that a quotation analysis feature provided by a vendor "turned on by itself."

To answer the question "Are we fine?", one must first know "what is there." Whatever is not on the inventory is excluded from risk assessments, control checks, and reports. And because audit resources are always constrained, after knowing what is there, one must decide where to go and how deep to go. Finally, one must also decide on what basis to state that things are "fine." Depending on whether the benchmark is laws and regulations, international standards, or internal company guidelines, the exact same fact may or may not constitute a finding.

This chapter addresses these three elements in order: inventory, scope, and criteria. First, it defines the questions answered by an AI audit and sets the boundaries of "AI" as the subject of the audit, and covers the elements to include in an inventory and how to identify inventory gaps. Next, it addresses how to determine the audit scope and depth based on risk, and how to map laws, regulations, and frameworks to use as audit criteria. Finally, it outlines key decisions often overlooked in the planning phase, such as the distinction between assurance and advisory, objectivity, and the use of specialists.

Key concepts

Questions answered by AI audits

Internal auditing is an independent evaluation of whether an organization's governance, risk management, and control processes are effective, assisting in their improvement. An AI audit is no different. The only difference is that the evaluation targets include the AI systems and the people, procedures, and contracts surrounding them. The questions an AI audit answers generally comprise the following five:

  1. Does the organization know the AI it uses? (Inventory and scope, this chapter)
  2. Are controls designed properly to reduce AI risks, and do they actually operate? (Chapter 2)
  3. Is the evidence supporting that judgment sufficient and reliable? (Chapter 3)
  4. Are models and data as accurate and fair as claimed, and are they managed when they change? (Chapter 4)
  5. When problems were found, were they communicated so they could be corrected, and were they actually corrected? (Chapter 5)

Let us also be clear about what an auditor does not do. Auditors do not approve or prohibit AI tools, do not directly design or operate controls, and do not discipline employees. The US GAO's Generally Accepted Government Auditing Standards (Yellow Book) cites responsibilities such as designing, implementing, and maintaining internal controls, and deciding which auditor recommendations to implement, as management responsibilities, and considers an auditor's independence impaired if they assume such responsibilities (GAO 3.78, 3.81). By analogy, an auditor is a building safety inspector, not the construction contractor. If an inspector starts repairing the wall directly, they will end up evaluating their own work during the next inspection.

What is considered AI

Before creating an inventory, boundaries must be established. The AI Basic Act defines "artificial intelligence" as the implementation of human intellectual capabilities—such as learning, reasoning, perception, judgment, and language comprehension—through electronic methods, and defines an "artificial intelligence system" as an AI-based system that, with varying levels of autonomy and adaptiveness, infers outputs such as predictions, recommendations, or decisions that influence physical and virtual environments for a given set of objectives (Article 2, subparagraphs 1 and 2). Systems that generate text, audio, images, video, and the like are separately defined as "generative artificial intelligence" (Article 2, subparagraph 5).

In audit practice, while using legal definitions as a starting point, it is advisable to ensure that organizational policy explicitly defines the scope of the inventory. Frequently debated boundary cases include the following:

Case Clue for judgment
Excel models scoring strictly based on rules If there is no learning or reasoning, it is generally outside the AI inventory, but if it serves as a point of comparison for AI handling the same decisions, record it together
Summarization and drafting features newly enabled in existing SaaS This is generative AI. The organization must decide whether to include it in the inventory or create a policy exception
External chatbots used with personal accounts If company data enters them, they are subject to the inventory and risk assessment
Classification services operated by external vendors Even if operations are outsourced, it remains the company's risk. Record alongside supplier information

NIST's Generative AI Profile (NIST AI 600-1) recommends inventorying the organization's generative AI systems and specifying any inventory exceptions for generative AI embedded in application software within organizational policy (GV-1.6-001, GV-1.6-002). Exceptions are permissible, but they must be documented in policy.

Roles and elements of an AI inventory

NIST AI RMF calls for establishing systems for inventorying AI systems and allocating resources to those systems in line with organizational risk priorities (GOVERN 1.6). The inventory is the starting point of an audit as well as a risk management tool for management. However, an inventory listing only names, departments, and costs cannot be used to judge risk. The elements of an inventory must be determined in reverse, starting from "What questions will be answered with this inventory?"

Element Question answered
Intended use and the decisions where outputs are used Who is affected and how
Affected individuals (customers, employees, citizens) Who suffers harm if harm occurs
Data handled and whether it includes personal information or sensitive information Whether the Personal Information Protection Act (PIPA) and security controls apply
Foundation model, version, deployment method (web, API, embedded feature, self-hosted) Whether vendor model changes affect it
Supplier and contractual or entrustment relationship Where data goes; whether third-party risk and trustee oversight are required
Human oversight and monitoring roles and responsible personnel Whether someone is in place to overturn results
Approval status, approval date, next review date Whether it is within the management framework
Known issues and incident history Whether risks have already materialized

NIST AI 600-1 cites examples of generative AI inventory elements such as data provenance information (provenance, signatures, versions, watermarks), known issues from internal error tracking and external incident reporting, human oversight roles and responsibilities, considerations regarding intellectual property, personal information, and sensitive data, foundation models and their versions, and deployment methods (GV-1.6-003). Auditors can evaluate whether the inventory contains these elements as the "design" of the inventory itself.

How to test the completeness of an inventory

The most common weakness of an inventory is completeness. Self-reported inventories, such as department surveys or department head confirmation letters, miss what is unknown, forgotten, or concealed. The Yellow Book states that because surveys generally provide self-reported information, auditors should evaluate the design and administration of the survey to determine its reliability (GAO 8.106), and explains that evidence obtained through an auditor's direct examination or computation is generally more reliable than indirectly obtained evidence (GAO 8.104).

Therefore, auditors cross-check self-reported inventories against independent records where usage footprints remain. Which records are useful depends on the usage pathway.

Usage pathway Where footprints remain Missed cases
Services accessed via web on the office network Web proxy, DNS, and firewall logs Access that does not pass through company networks, such as remote work or off-site work
Apps connected to company Single Sign-On (SSO) SSO app lists and sign-in logs Services signed up for using personal accounts
Corporate card payments or expense reimbursement after personal payment Corporate card and expense settlement statements Free services, features included in existing contracts
Adoption through procurement procedures Procurement and contract registers Low-value subscriptions, free trials
AI features embedded in existing SaaS Admin settings of each service, vendor change notices Cases where administrators do not review settings screens
APIs called directly by developers API key issuance logs, cloud billing statements Cases where personal keys are used

No single source can cover every pathway. Auditors first identify the organization's actual working practices (remote work ratio, payment policies, access blocking policies) and select sources matched to pathways where omissions are most likely. For instance, in a company that blocks external AI websites and refuses expense reimbursement for AI purchases, the avenue for new tools to enter is narrow, but the avenue where AI is enabled within already utilized services remains open.

Shadow AI

Shadow AI refers to AI used outside organizational approval or management. In most cases, it arises not out of malice, but out of necessity—because approval processes take weeks, approved tools do not fit the work, or free services are too easily accessible.

When shadow AI is discovered, there are three things an auditor must do:

  1. Verify the scale with independent evidence. Confirm the frequency of use, user departments, and duration through access and payment records.
  2. Assess the magnitude of the risk. Even with the same frequency of use, the risk varies entirely depending on what data was transmitted. Check on a sample basis whether uploaded file types or input contents contain customer information.
  3. Identify the root cause. Determine whether approval processes fail to keep pace with operational demand, whether approved alternatives are lacking, or whether employees are unaware of policies. Knowing the cause allows recommendations to reduce workarounds.

Conversely, handing over user lists for disciplinary purposes, having the audit team approve tools on behalf of management, or excluding them on the grounds that "they are outside the audit scope because they are not approved" are all improper. The first two are management's responsibilities, and the last erases the greatest risk from the audit scope.

Risk-based scoping

Audit resources are allocated in proportion to risk. NIST AI RMF calls for establishing processes to determine the level of risk management activities based on organizational risk tolerance (GOVERN 1.3), and identifying and documenting the likelihood and magnitude for each identified impact (MAP 5.1). IIA Standards also require auditors to understand the objectives and risks of the activity under review, the organization's risk tolerance, and applicable criteria, and then identify and prioritize significant risks (IIA 13.2).

Signals that can be used to prioritize risks of AI systems include the following:

Signal Example elevating risk
Decisions where outputs are used Direct impact on individual rights or safety, such as loans, hiring, insurance claims, or medical triage
Human intervention Humans follow almost every recommendation as-is, or confirmations occur automatically without intervention
Frequency of change Frequent model replacements or judgment criterion changes without re-validation records
Data External data of unclear provenance, sensitive information, or data collected for other purposes
Third-party reliance Outsourced development or operations without audit rights or change notice provisions in contracts
Prior results Recommendations from prior audits or assessments left unimplemented or unverified

Even for the same system, the focus of the audit shifts depending on which signal stands out. If a model has remained unchanged for two years but contracts are deficient, third-party management takes center stage over change management; if newly purchased data has entered the system, data provenance becomes the primary focus.

Determining audit depth

Not every system needs to be examined at the same depth. In fact, reviewing everything at the same depth means the riskiest systems will not be addressed adequately. In practice, audit depth is planned across several tiers.

Depth Activities Suitable subjects
Tier 1: Management framework verification Verify inventory registration, risk assessment, and approval records Internal administrative assistance, no personal information, outputs not used directly in decisions
Tier 2: Key control testing Test the design and operating effectiveness of the one or two most critical controls (human review, change approval, etc.) Moderate impact, or implementation of recommendations for systems confirmed in prior audits
Tier 3: In-depth audit In addition to control testing, verify performance and bias claims, data provenance, and third-party management Systems impacting individual rights or safety that have not yet been audited

Lowering the audit depth does not mean removing the system from the inventory. Maintaining an inventory and determining audit depth are separate decisions.

Meaning and sources of audit criteria

Audit criteria represent the "condition that should exist." The Yellow Book explains that criteria may include laws, regulations, contracts, standards, measures, expected performance, established business practices, and benchmarks (GAO 8.124). IIA Standards require identifying the most relevant criteria to evaluate the activity under review, using criteria established by the board and management if appropriate, or collaborating to identify appropriate criteria if they are inadequate (IIA 13.4).

Therefore, the claim that "it cannot serve as a criterion because it is a voluntary framework" is incorrect. Even frameworks that are not legal obligations, such as the NIST AI RMF, can serve as criteria if adopted by the organization or widely used in the industry. However, the rationale for why that criterion is suitable must be stated in the audit plan and report.

Regulatory mapping 1: AI Basic Act

The AI Basic Act contains provisions frequently used as audit criteria. Here, we summarize only what is necessary for audit planning. Determining whether a specific system is subject to legal obligations requires legal judgment.

  • High-impact artificial intelligence (Article 2, subparagraph 4). Refers to artificial intelligence systems that have a significant impact on or pose concerns of risk to human life, physical safety, and fundamental rights, utilized in energy supply, drinking water production processes, healthcare delivery, medical devices, nuclear facilities, biometric information analysis in criminal investigations, judgments or evaluations that significantly affect individual rights and obligations such as recruitment and loan reviews, transportation, decision-making by state agencies and public institutions such as verification and determination of eligibility for public services, student assessments in early childhood, primary, and secondary education, and other areas prescribed by Presidential Decree.
  • Obligation to ensure transparency (Article 31). To provide products or services utilizing high-impact artificial intelligence or generative artificial intelligence, providers must notify users of that fact in advance, label generated outputs with the fact of generation, and notify or label synthetic audio, images, and video that are difficult to distinguish from reality so that users can clearly recognize them.
  • Confirmation of high-impact artificial intelligence (Article 33). Business operators must examine in advance whether high-impact classification is applicable, and if necessary, may request confirmation from the Minister of Science and ICT. For an auditor, "Is there a record of prior examination?" is the first check item.
  • Responsibilities of business operators (Article 34). Business operators providing high-impact artificial intelligence must implement the measures under each subparagraph of paragraph 1, namely: subparagraph 1, establishment and operation of risk management plans; subparagraph 2, establishment and implementation of explanation plans; subparagraph 3, establishment and operation of user protection plans; subparagraph 4, human oversight and supervision; and subparagraph 5, preparation and retention of documents enabling verification of the details of such measures. Here, business operators include both "artificial intelligence development business operators" who develop and provide AI, and "artificial intelligence deploying business operators" who provide products or services using AI provided by development business operators (Article 2, subparagraph 7).
  • Article 27 of the Enforcement Decree. The key contents of the risk management plan and explanation plan, the user protection plan, and the name and contact details of the supervisor/manager must be posted (trade secrets may be excluded), and the basis of the measures must be retained in document form for five years. If a deploying business operator that received a system from an AI business operator that fulfilled all or part of the measures under Article 34, paragraph 1, subparagraphs 1 through 3 has not made significant functional modifications, such as substantially altering the original purpose or intended use, it is deemed to have fulfilled all or part of those measures. Because deeming is limited to subparagraphs 1 through 3, human oversight and supervision (subparagraph 4) and document preparation and retention (subparagraph 5) are not subject to deeming. Deploying business operators may request necessary materials from development business operators.
  • Impact assessment (Article 35). When providing products or services utilizing high-impact artificial intelligence, operators shall endeavor to evaluate the impact on fundamental rights in advance, and when conducting assessments, the characteristics of vulnerable groups regarding artificial intelligence must be reflected in consideration of the nature of the products or services. Because the legal strengths differ—conducting assessments is "shall endeavor to" while reflecting the characteristics of vulnerable groups is "shall"—they are recorded separately in the criteria matrix, and if an assessment was performed, records of that reflection are included in the request for evidence.

In audit planning, these provisions translate into "What evidence will be requested?" Records of prior examination, risk management plans, posted screens, supervisor designation documents, the five-year retention mechanism, and, for deploying business operators, materials received from the development business operator and whether functional modifications occurred are placed on the request list.

Regulatory mapping 2: Personal Information Protection Act (PIPA)

Because most AI systems handle personal information, the Personal Information Protection Act (PIPA) also frequently serves as a criterion. Two provisions are frequently used in audit planning:

  • Entrustment of processing (Article 26). When entrusting personal information processing work, it must be done in a document containing prohibitions on processing outside the entrustment purposes, technical and managerial protective measures, and other matters; the entrusted work and the trustee must be disclosed so that data subjects can easily check them; and the trustee must be supervised through training and inspections of processing status. Trustees may not use personal information beyond the entrusted scope. If an AI service is outsourced, the entrustment document, public disclosure, and supervisory records enter the request list.
  • Automated decisions (Article 37-2). If a decision made by processing personal information through a fully automated system (excluding automated administrative dispositions by administrative agencies pursuant to Article 20 of the General Act on Public Administration) has a significant impact on the rights or obligations of a data subject, the data subject may reject that decision (subject to statutory exceptions). Separately, where a personal information controller has made an automated decision, the data subject may request an explanation of that decision. The personal information controller must disclose the criteria and procedures of automated decisions and the manner in which personal information is processed so that they can be easily checked. Because this provision applies to "fully automated" decisions, whether human intervention is substantive is the key fact determining whether the provision applies. Auditors verify whether human intervention is not merely perfunctory, and, if the decision is subject to the provision, whether procedures are in place to handle rejection and explanation requests.

Framework mapping 3: NIST AI RMF, GAO Framework, ISO/IEC 42001

In addition to statutory enactments, three frameworks are widely used as criteria for AI audits.

Framework Structure How used in audit
NIST AI RMF 1.0 Four functions: GOVERN, MAP, MEASURE, MANAGE, and subcategories. GOVERN is a foundational function that permeates across the other three functions Convert subcategories into inspection questions to identify gaps in controls
GAO AI Accountability Framework (GAO-21-519SP) Four principles: Governance, Data, Performance, Monitoring, and key practices. Presents audit questions, procedures, and evidence types for each practice Used as the backbone of the audit program
ISO/IEC 42001:2023 Requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System (AIMS) for organizations developing, providing, or using AI If the organization has set certification as an objective, it becomes one of the audit criteria

When dealing with ISO/IEC 42001, keep one thing firmly in mind: ISO only develops standards and does not certify; certification is performed by external certification bodies. The conclusion of an internal audit has meaning only within the audited scope and criteria, and cannot substitute for certification.

Assurance, advisory, and objectivity

Internal audit activities broadly fall into two categories: assurance provides an independent conclusion on whether controls are effective, while advisory counsels management on designing or improving controls. The distinction lies not in timing, but in purpose. A request for feedback to assist management in designing controls is performed as advisory, whereas a request seeking an independent conclusion on whether controls are adequate or operated against criteria is performed as assurance. Requests for feedback during pre-launch design stages are generally advisory, whereas requests for feedback on whether controls that operated over a given period functioned properly are assurance.

IIA Standards impose several requirements to safeguard objectivity (IIA 2.2). Internal auditors must not evaluate activities for which they previously had responsibility, and objectivity is presumed to be impaired if an auditor performs assurance engagements for activities for which they had responsibility within the previous 12 months. To perform an assurance engagement in an area where advisory services were previously provided, the chief audit executive must confirm that the nature of the advisory engagement does not impair objectivity and assign personnel accordingly. Assurance engagements for functions for which the chief audit executive is personally responsible must be overseen by an independent party outside the internal audit function. If the chief audit executive temporarily assumed responsibilities outside of internal auditing, assurance for that area must be provided by an independent third party during the period assumed and for 12 months thereafter, and the chief audit executive must establish a plan to transition those responsibilities to management (IIA 7.1).

Using specialists and prior results

AI audits frequently demand expertise not present within the audit team. It is difficult for an auditor alone to judge the accuracy of medical imaging interpretation or the statistical validity of a credit rating model. The Yellow Book states that if necessary, specialists should be engaged, but their competence and qualifications, as well as the effects that bias, conflicts of interest, and outside influence may have on their judgment, should be evaluated (GAO 4.12–4.14), and the objectives and scope of the specialist's work, the intended use, procedures and findings, assumptions, and methods should be documented (GAO 8.32). NIST AI RMF also recommends that internal experts who were not frontline developers of the system or independent assessors participate in regular assessments (MEASURE 1.3). An individual who directly championed the implementation cannot serve as an independent assessor, no matter how outstanding an expert they may be.

Prior results also serve as raw material for planning. The Yellow Book requires auditors to inquire of management about prior audits, attestations, or studies related to the audit objectives and whether related recommendations were implemented, using this information in risk assessments and determining the scope of procedures (GAO 8.30). The answer that "it was implemented" is merely a starting point; whether it was implemented is confirmed through testing.

Going deeper

Whether to include embedded AI in the inventory

As AI features are added across business software, the principle of "including all AI in the inventory" quickly becomes unmanageable. If everything—from spelling suggestions in document editors and reply recommendations in email to automatic live captions in video conferences—is included, the inventory expands to hundreds of lines, and truly critical systems get buried. Conversely, excluding embedded features entirely leaves out meaningful risks, such as CRM capabilities that summarize customer consultation records and send them to external servers.

The fact that NIST AI 600-1 specifies that inventory exceptions for embedded generative AI should be "determined by organizational policy" means that this tension should be made explicit through policy decisions. Rather than taking issue with exceptions themselves, auditors evaluate the criteria for those exceptions. Sound exception criteria draw the line based on "what data goes out" and "whether outputs are used for decisions." For example, an organization might define that "embedded features that do not send data outside the company and whose outputs are not directly used in human judgment are recorded at the product level rather than the feature level." Auditors verify whether those criteria have been applied consistently and whether features newly enabled after establishing the criteria have been reviewed.

The boundary between audit criteria and legal judgment

The AI Basic Act has not been in effect for long, and it leaves many details to Presidential Decrees, public notifications, and guidelines. Therefore, whether "this system is high-impact" can vary depending on factual circumstances and interpretation. If an auditor makes a legal determination on this question, two risks arise: the risk that the conclusion is incorrect, and the risk that the auditor substitutes for the judgment that management ought to make.

Therefore, in practice, the question is reframed. Instead of asking "Is it high-impact?", auditors ask: "Did the company review in advance whether high-impact classification was applicable, is that review documented, what was the basis for the review, and who approved it?" Because the prior review itself is a procedure required by law (Article 33, Paragraph 1), an auditor can verify the answer to this question as a matter of fact. If the review results appear questionable, the auditor reports those doubts along with their basis and recommends a legal review.

Assurance after advisory: when the same team reviews twice

Because AI governance is being established for the first time in many organizations, management often asks internal audit to "review the design together." There is no need to decline this request. Advisory engagements are legitimate internal audit responsibilities and provide significant value in reducing risks prior to launch. The problem arises one year later when providing assurance over those same controls.

The boundary to maintain during advisory work is that "audit provides opinions, but management makes the decisions and handles operations." If an auditor writes operating procedures on management's behalf, directly designs approval screens, or even assumes the role of Operations Manager, it is no longer advisory work but an assumption of managerial responsibility, and subsequent assurance becomes an evaluation of one's own work. Conversely, if management decided on and operated the controls, the Chief Audit Executive can confirm objectivity and manage the engagement by adjusting staffing, even if the team that provided advisory services performs the assurance engagement. The written opinions issued during the advisory phase and management's decision records serve as evidence demonstrating objectivity later on.

At work

Pine Creek Construction: different AI inventories at every site

The internal audit team at the general construction firm Pine Creek Construction (3,400 employees, 46 construction sites) included an "AI usage management" audit in its annual plan for the first time. The list provided by the IT team contained five approved tools, all of which had been contracted by headquarters.

The audit team first examined the company's way of working. More than half of the employees worked at site offices, and these site offices used separate telecommunication lines that bypassed the headquarters proxy. Site operating expenses could be charged to corporate cards under the sole authority of site managers. One audit team member suggested pulling headquarters proxy logs first, but the team lead pointed out that those logs would hardly reflect site usage. They decided to use headquarters logs only for headquarters.

For the job sites, they searched corporate card transaction statements for AI service payments. Because some charges were billed under payment agency names, merchant names alone were insufficient, so they examined amount patterns alongside receipts. As a result, they discovered that seven paid AI services were being used across eleven job sites. Among these, two were services where construction progress photos were uploaded to identify safety violations, and the photos contained the faces of subcontractor workers. The audit team placed these two services at the very top of the risk assessment.

There was still another route. The construction management SaaS used by the company had enabled "automated daily work report summarization" by default in an update the previous quarter. Because there was neither a payment nor a new connection, no trace existed on corporate cards or the proxy. The audit team identified this feature by reviewing the SaaS administrator console and vendor notices, noting separately that company policy contained no criteria regarding embedded features.

The audit team did not compile a list of users of unapproved services. Instead, they interviewed five site managers to ask why they had not gone through the approval process. The answers were consistent. The approval application form was structured around headquarters IT standards, so personnel at the sites did not know what to write, and processing took two months. The root cause of the finding was framed as an "approval process unsuited to field realities." Along with the supplemented inventory results (expanding from 5 to 13), the audit team recommended policy criteria for embedded features and a streamlined approval process tailored for field sites.

Sunrise Credit Union: where to spend 1,000 hours

The internal audit office at Sunrise Credit Union had 1,000 hours available for AI audits this year. The supplemented inventory contained six AI systems: a personal credit loan review support model, a delinquent member consultation prioritization model, a customer inquiry chatbot, an internal regulation search assistant, a meeting minutes summarization tool, and a fraud detection service introduced this year.

The audit office tabulated the impact, likelihood, and whether prior verification had taken place for each system. The loan review support model affected whether members received loans and had undergone an in-depth audit the previous year, resulting in three recommendations. The fraud detection service was operated by an external vendor and could block transactions, but it had not yet been audited and contract review was still incomplete. The delinquent consultation prioritization model determined only the order of consultations, while the actual decisions were made by counselors.

The Chief Audit Executive initially received a proposal to divide the hours equally among the six systems. The rationale was that it was "fair," but that would allocate barely around 170 hours to the fraud detection service, making it impossible to review contracts, operations, and the impact of false-positive blocks. There was also an opinion suggesting that substantial hours should again be dedicated to the loan model audited the previous year. The justification was that it was efficient because prior working papers could be reused, but efficiency was not a valid basis for risk-based allocation.

The final plan was established as follows: 450 hours for the fraud detection service (Level 3 in-depth audit: contract and trustee management, relief procedures for members whose transactions were blocked due to false positives, and review of vendor reports); 250 hours for the loan review model (Level 2: implementation testing for the three recommendations from the previous year); 100 hours each for the chatbot and delinquent consultation model (Level 2: one key control each); and 50 hours each for regulation search and meeting minutes summarization (Level 1: review of management framework). For fraud detection, they decided to involve an external data science expert, documenting the scope of work and methodology after confirming that the expert had no conflicts of interest with the vendor. The plan also recorded the rejected proposals (equal allocation, prioritizing familiar systems) along with the reasons for their rejection.

Common mistakes

  1. "Only approved tools are within the audit scope." Unapproved use may pose the greatest risk. Finding out-of-inventory usage and bringing it into scope is the first task of an audit.
  2. "Since we received written confirmations from department heads, the inventory is complete." Self-reporting omits what is unknown and what is concealed. Completeness is verified through independent records that leave traces of use.
  3. "A single proxy log captures everything." Usage that does not go through the corporate network and features embedded in existing services are invisible to proxies. Sources are selected to match the usage path.
  4. "If shadow AI is discovered, hand over users for disciplinary action." Disciplinary action and tool approval are the responsibility of management. The audit determines the scale, risk, and root causes.
  5. "Fairly examine all systems at the same depth." When risks differ but depth is the same, the highest-risk systems will not be addressed sufficiently.
  6. "Voluntary frameworks cannot serve as audit criteria." Criteria also include standards, industry practices, and organizational guidelines. However, you must state why they are appropriate.
  7. "Since there were no issues in the internal audit, we can state compliance with ISO/IEC 42001." Certification is performed by certification bodies. The conclusion of an internal audit is stated only within the audited scope and criteria.
  8. "The professor who championed the adoption is the foremost expert, so assign the verification to them." Experts must be evaluated for objectivity alongside competence. If a conflict of interest exists, engage an independent expert.

Summary at a glance

Topic What to remember
Role of the audit Evaluates and recommends. Approval, operation, and disciplinary action are management's responsibility (GAO 3.78, 3.81)
Boundary of AI Starts from the legal definition (Article 2, Subparagraphs 1, 2, and 5), and embedded feature exceptions are documented in policy (AI 600-1 GV-1.6-002)
Inventory fields Defined backward from questions. Data, base model/version, usage mode, oversight lead, vendor (GV-1.6-003)
Inventory completeness Self-reporting is the starting point. Verified through independent records by usage path (proxies, SSO, cards/expenses, administrator settings)
Shadow AI Identifies scale, risk (what data went out), and cause (approval procedures)
Risk-based scoping Considers impact, likelihood, and prior verification together. Prominent signals determine the audit focus
Audit depth Management system verification → key controls testing → detailed audit. Separate from inventory maintenance
Audit criteria Laws and regulations, standards, guidelines, and practices. If management criteria are appropriate, use them (IIA 13.4)
AI Basic Act Preliminary review (Article 33), responsibilities (Article 34), publication, 5-year retention, and deemed deployer provisions (Enforcement Decree Article 27)
Objectivity Do not assess activities for which responsibility was held within the past 12 months. Functions managed by the chief audit executive require external independent oversight; temporary responsibilities require third-party assurance during that period and for 12 months thereafter (IIA 2.2, 7.1)

Check the official docs

Self-check

1-1. Explain the meaning of "defining backward from questions" when establishing fields for an AI inventory, and give two fields needed to assess personal information risks.

1-2. A company with a 70% remote work rate intends to verify the completeness of its AI inventory using only web proxy logs. State the limitations of this method and two sources to supplement it.

1-3. State three things an auditor must identify after discovering shadow AI usage, and state one thing an auditor must not do, along with the reason.

1-4. Explain the principle in risk-based scoping that "prominent signals determine the focus of the audit," using a model operated by an external development vendor as an example.

1-5. Answer whether non-statutory frameworks such as the NIST AI RMF can be used as audit criteria in light of the Yellow Book and IIA standards, and state what must be documented in the report when using them.

1-6. State three items from Article 27 of the Enforcement Decree of the AI Basic Act that an auditor can convert into an evidence request list.

1-7. (Design) In an organization where the chief audit executive served as the secretary of the AI Policy Committee for one year, the board requested assurance on the operations of that committee. In three lines, design how to structure the engagement in accordance with IIA standards.

Hands-on tasks

Exercise

Supplementing the AI Inventory with Independent Sources (45 minutes)

  1. Describe your organization's (or a hypothetical organization's) way of working in five lines: remote work ratio, expense policy, access blocking policy, major SaaS, and API usage.
  2. Referring to the "Usage paths" table in this chapter, select three paths that are highly likely to be omitted, and determine the appropriate independent source for each.
  3. For each source, write down "what to extract (fields)," "whom to request it from," and "what cannot be captured with this source."
  4. Select five fields to add to the inventory from the inventory fields table in this chapter and add them as columns.

Done when: The fields to extract and the limitations are documented for all three sources, and a one-line explanation is provided for each of the five added fields describing what question it answers.

Exercise

Determining Risk Ranking and Audit Depth (45 minutes)

  1. Select five AI systems from the supplemented inventory (hypothetical ones are also acceptable).
  2. For each system, write one line each for the decision where the output is used, human intervention, update frequency, data, third-party dependency, and prior results.
  3. Rate impact and likelihood as High, Medium, or Low, and determine the audit depth (Stages 1–3) and allocated time.
  4. Write down one discarded allocation plan and the reason for discarding it.

Done when: Audit depth and time are determined for all five systems, the rationale for selecting the system examined in the greatest depth is explained through risk signals, and the reason for the discarded plan is documented.