LLM API Developer Course

Contents

1. How to use this book
2. Chapter 1. API basics and message design

   An API call is an order ticket · Message roles and priorities · Trust boundaries: separate instructions from material · Who holds the conversation state? · What is a token? · Context windows and token budgets · Maximum output is a ceiling · Turn stop reasons into states · Sampling parameters: principles and current behavior · Reasoning effort and thinking settings · Model selection: tiers, features and limits · From requirements to models and parameters · Why identical requests produce different answers · Three ways to handle long conversations

3. Chapter 2. Structured output and tool calling

   Structured output is a mold · How to enable it for each vendor and choose a method · Strict schema rules and vendor differences · Derive schemas from business concepts · Validate in three stages · A tool definition is an execution contract · Build the tool-calling loop as a state machine · Return results to the conversation · Parallel tool calls · tool_choice and forced calls · Reasoning models and tool loops · The server holds execution permissions · When the number of tools grows · Structured results during streaming

4. Chapter 3. Performance and cost optimization

   Break latency into stages · Seven principles for reducing latency · How streaming works · Design the entire streaming path · How prompt caching works · Comparing caching across vendors · Arrange prompts for more cache hits · Cache break-even points · Batch APIs · Choose a service tier · Model-tier routing and escalation · Judge latency/cost trade-offs · When is response caching dangerous? · Compare optimization experiments fairly

5. Chapter 4. Reliability and error handling

   Success, failure and unknown outcomes · Error classification table · A decision table for retryable errors · Rate-limit structure · How to read a 429 · Exponential backoff and jitter · Retry in one place, within a budget · Design timeouts in layers · Idempotency: two presses, one action · Recover interrupted responses · Recover partial failures · Prevent failure from spreading · Anatomy of a retry storm · Validate recovery policies by injecting failures

6. Chapter 5. Evaluation and operations

   Operations is instrument flying · Define success criteria first · Build an evaluation set · Choose a grading method · Calibrate an LLM grader · Interpret results: variation and segments · Pin model versions · Regression evaluation before switching models · Logging and tracing · Handle personal information · Define operational metrics · Divide costs and limits by feature · Avoid dependence on evaluation tools · Feed feedback back into evaluation sets

7. Appendix A. Glossary
8. Appendix B. Self-check answers and explanations
9. Appendix C. Exam day guide
10. Appendix D. Reference documents