Specialty · Specialty exam
AI Product Planning
Validates the ability to plan and launch AI features—from determining where to apply AI, setting requirements and success criteria, and designing evaluations, to managing failure modes and human oversight.
Daily practice · Free
10 questions · 25 min · No sign-up needed
Specialty exam · Proctored
35 questions · 60 min · Raw score of 75 or more · $3.99 / one attempt
Specialty exam detailsExam overview
- Questions
- 35
- Time
- 60 min
- Pass mark
- 75 pts
- Recommended experience
- Yellow Belt (AI Prompting) level and product planning experience
Every question has five choices and is marked on an OMR answer sheet. Multiple-choice and true-or-false questions ask for the right combination from a box; ordering and matching questions ask for the right arrangement. Judgment questions (L3) are worth 3 points and all others 2; the raw score is converted to a 100-point scale and graded 1 to 9.
Exam domains
1. Evaluating AI Applicability20%
2. Requirements and Success Criteria20%
3. Evaluation Design25%
4. Failure Handling and Human Oversight20%
5. Launch and Operations15%
Weights follow the Specialty exam. Practice and mock exams may mix domains differently.
What this exam verifies
- Determine whether to use AI or alternative approaches by evaluating the problem, data, and risks
- Define requirements and measurable success criteria for AI features
- Select pre- and post-launch evaluation methods and scoring approaches
- Design mitigation strategies by failure type and establish human oversight checkpoints
- Determine launch scope, operational metrics, and iteration cycles
Who it is for
- Product managers and product planners designing AI features
- Business managers selecting and prioritizing AI adoption initiatives
- Product leaders defining launch criteria for AI features
Model training, serving implementation, and writing code are not covered (within the scope of Red Belt: AI Feature Development).
Domain details
Domain 1. Evaluating AI Applicability
- Knowledge: Appropriate solutions among rules, search, and models, Problems suited and unsuited for AI, Data and cost constraints, Applicability of high-impact domain regulations
- Skills: Determining whether and how to apply AI for each initiative, Filtering out requirements that do not require AI
Domain 2. Requirements and Success Criteria
- Knowledge: Verifiable success criteria, Input and output contracts, Trade-offs among quality, latency, and cost, User-facing failure experiences
- Skills: Translating ambiguous requirements into measurable criteria, Distinguishing launch-blocking criteria from improvement goals
Domain 3. Evaluation Design
- Knowledge: Evaluation dataset construction, Code-based, model-based, and human scoring, Offline evaluations and online metrics, Regression evaluation
- Skills: Selecting scoring methods appropriate for the feature, Determining launch readiness based on evaluation results
Domain 4. Failure Handling and Human Oversight
- Knowledge: Hallucinations, refusals, and formatting errors, Fallback paths and graceful degradation, Human review checkpoints and authority, User notice (Article 31 of the Framework Act on Artificial Intelligence) and user feedback
- Skills: Designing mitigations by failure type, Defining conditions and procedures for human intervention
Domain 5. Launch and Operations
- Knowledge: Phased rollouts and experimentation, Operational metrics and alerting, Model and prompt change management, Incident response and post-mortems
- Skills: Setting launch scope and abort criteria, Prioritizing improvements using operational data
Official documentation
Anthropic
국가법령정보센터
Specialty Course · AI Product Planning
Practice, mock exams and every course book are free. Pay only when you take an official test.