| AI usage list (inventory) |
A list that organizes the AI systems developed, introduced, and used by an organization, along with their purpose, data, person in charge, supplier, and other details. It is the starting point of the audit scope. |
| Shadow AI |
AI used outside of organizational approval or management. It arises mainly from operational necessity and slow approval procedures. |
| Built-in AI features |
AI features such as summarization and draft creation provided within existing business software. Whether additional payment, licensing, or activation is required depends on the product and contract. |
| Usage pathways |
The routes through which AI enters work. These include office web access, personal payments, company contracts, built-in features, APIs, and more, with traces left in different locations. |
| Risk-based audit |
A method of allocating audit targets, depth, and time according to the magnitude of impact and the likelihood of occurrence. |
| Audit depth |
The level of audit detail determined for each system, such as management system verification, key control testing, or in-depth auditing. |
| Audit criteria (criteria) |
The condition that should exist. This can be laws and regulations, standards, contracts, organizational guidelines, expected performance, industry practices, and more. |
| Assurance engagements |
Internal audit engagements that provide an independent conclusion on whether controls or activities are effective. |
| Advisory engagements |
Internal audit engagements that advise management on designing or improving controls. Decision-making and operation remain with management. |
| Objectivity |
A state in which judgment is not swayed by influence or conflicting interests. It is impaired if an auditor evaluates activities for which they were previously responsible. |
| Self-review threat |
A threat to independence that arises when an auditor evaluates work or judgments that they personally performed previously. |
| Management responsibilities |
Tasks that impair independence if assumed by an auditor, such as policy decision-making, designing, implementing, and maintaining controls, and deciding whether to implement recommendations. |
| Specialist |
A person possessing specialized techniques or knowledge required for an audit. They are used after evaluating their competence and objectivity and documenting their scope of work. |
| High-impact artificial intelligence |
AI systems used in the domains prescribed by Article 2, Subparagraph 4 of the AI Basic Act that can have a material impact on life, physical safety, and fundamental rights. |
| Preliminary review |
The act of a business operator reviewing in advance whether an AI is applicable to the high-impact category (Article 33, Paragraph 1 of the AI Basic Act). |
| Deemed business user |
A provision under which a business user that receives a system from an AI business operator that has implemented all or part of the measures in Article 34, Paragraph 1, Subparagraphs 1 through 3, and uses it without significant functional modifications, is deemed to have implemented all or part of those measures (Article 27, Paragraph 3 of the Enforcement Decree). |
| Entrustment of processing |
Entrusting the processing of personal information to a third party. An entrustment document, public disclosure, and training and supervision of the trustee are required (Article 26 of the Personal Information Protection Act (PIPA)). |
| Automated decisions |
Decisions made by fully automated systems processing personal information (excluding automated dispositions by administrative agencies). If they have a material impact on rights or obligations, the subject can refuse them and request explanations, etc. (Article 37-2 of the Personal Information Protection Act (PIPA)). |
| NIST AI RMF |
The AI Risk Management Framework from the U.S. NIST. It consists of four functions: GOVERN, MAP, MEASURE, and MANAGE. |
| GAO AI Accountability Framework |
Key practices and audit questions for AI auditing organized by the U.S. GAO under four principles: governance, data, performance, and monitoring. |
| ISO/IEC 42001 |
An international standard specifying requirements for establishing, implementing, maintaining, and continually improving an AI management system. Certification is conducted by a certification body. |
| Controls |
Policies, procedures, and system devices that prevent risks from leading to consequences. |
| Preventive controls |
Controls that prevent errors or fraud from occurring in advance. |
| Detective controls |
Controls that detect errors or fraud that have occurred in a timely manner. |
| Automated controls |
Controls executed by a system without human intervention. While the risk of human error during repetitive execution is low, their effectiveness can be compromised by incorrect configurations, data, or program changes. |
| Design effectiveness |
Whether a control is designed so that, if executed as prescribed, it can achieve the control objectives. |
| Operating effectiveness |
Whether a control was actually executed as designed and whether the person executing it had the necessary authority and competence. |
| Design deficiency |
A state in which a necessary control is missing or is designed such that it cannot achieve its objective even if operated as prescribed. |
| Operating deficiency |
A state in which a properly designed control was not executed as prescribed, or the person executing it lacked the authority or competence. |
| Walkthrough |
A procedure that traces a single transaction from beginning to end while combining inquiry, observation, and document inspection. It is primarily used for design evaluation. |
| Reperformance |
A procedure in which the auditor independently executes procedures or controls originally performed by the company. |
| Recalculation |
A procedure in which the auditor verifies the mathematical accuracy of information through recalculation. |
| Deviation |
An instance in which a control is not executed as prescribed in a sample item. |
| Superficial approval |
A state where a review procedure exists, but AI results are actually passed through as-is. |
| Automation bias |
The tendency of humans to over-rely on or value outputs from an automated system higher than other sources. |
| Least privilege |
The principle of granting only the minimum permissions strictly necessary for work. It applies to both human accounts and AI agents. |
| Baseline strategy (benchmarking) |
An approach that continues to rely on prior test results after confirming that general controls are effective and automated controls have not changed. |
| Service organization report |
A report prepared by an independent auditor regarding the controls of an external service organization. Its type, scope, and period must be read. |
| Complementary user entity controls |
Controls assumed to be operated by the user company for the service organization's controls to be effective. |
| Sufficiency |
The quantity of audit evidence. |
| Appropriateness |
The quality of audit evidence. It consists of relevance and reliability (with the Yellow Book also including validity). |
| Information produced by the entity |
Reports, logs, and summaries produced by the systems or employees of the audited organization. Their accuracy and completeness must be verified before being used as evidence. |
| 100% examination |
A method of examining all items in a population. It is particularly suitable when procedures can be automated. |
| Specific item selection |
A method of selecting and examining items that are material or risky. Results cannot be projected to the population. |
| Audit sampling |
A method of applying procedures to a portion of a population to evaluate the characteristics of the population. |
| Stratification |
A method of dividing a population into similar groups based on characteristics related to the audit objectives and drawing samples from each group. |
| Tolerable deviation rate |
The maximum rate of deviation that can be accepted without altering planned reliance on controls. |
| Sampling risk |
The risk that a conclusion drawn from a sample may differ from the conclusion reached if the entire population were examined. It increases as the sample becomes smaller. |
| Reproducibility |
The property of being able to obtain the same result again with the same input. Generative models are not completely deterministic. |
| Confabulation |
A phenomenon where generative AI creates incorrect or false content stated with confidence. It is commonly called hallucination. |
| Confusion matrix |
A table organizing classification results into four cells: true positives, false positives, false negatives, and true negatives. |
| Recall |
The proportion of actual positives that the model identified as positive. |
| False negative rate |
The proportion of actual positives that the model missed. It is equal to 1 minus recall. |
| Specificity |
The proportion of actual negatives that the model correctly identified as negative. |
| Precision |
The proportion of actual positives among those determined as positive by the model. |
| False positive rate |
The proportion of actual negatives that the model incorrectly determined as positive. |
| Baseline |
A comparison target against which performance is compared, such as majority-class determination, existing methods, or human performance. |
| Validation |
Confirmation through objective evidence that the requirements for a specific intended use have been met. |
| Reliability |
The ability to operate as required without failure under given conditions for a given period of time. |
| Accuracy |
The degree of closeness of a result to a true value or an accepted true value. |
| Robustness |
The ability to maintain performance levels across diverse situations. |
| Data leakage |
An issue where information from evaluation data contaminates training, causing performance to be measured higher than it actually is. |
| Proxy variable |
Another variable (such as place of residence, usage patterns, etc.) that serves as a substitute even when sensitive variables are removed. |
| Label bias |
A skew contained within the ground-truth labels themselves. It is prone to occur when past human decisions are used as the ground truth. |
| Drift |
A phenomenon during operation where the input data (data drift) or the relationship between inputs and outputs (concept drift) differs from that during training. |
| Regression evaluation |
An evaluation comparing before and after changes to verify whether existing functions continue to work as expected after a change. |
| Finding |
The gap between criteria and condition identified during an audit. It is developed into criteria, condition, cause, and effect. |
| Root cause |
The deeper factor that created the gap between criteria and condition. It is the target that recommendations must aim to address. |
| Risk acceptance |
A response that accepts risk without reducing it. If it exceeds tolerance limits, it is reported to executive management and the board of directors. |
| Follow-up review |
A risk-based procedure to verify whether recommendations or action plans have actually been implemented and have resolved the root cause. |