Chapter 1. How AI Answers

Learning objectives

• Explain using everyday analogies that generative AI does not "retrieve" answers, but rather "composes" them.
• Describe the difference between search and generative AI, and choose the appropriate one for a given situation.
• Explain why plausible-sounding yet incorrect answers (hallucinations) occur, and identify warning signs that warrant skepticism.
• Explain why AI is weak regarding information after its Training cutoff, and why answers vary even for the same question.
• Identify situations where AI answers should not be taken at face value, and divide tasks into what to delegate to AI and what to do yourself.

Introduction

Ms. Mal-sun Jeong was sixty-seven years old. Her granddaughter installed an AI chatbot app on her phone and said, "Grandma, ask here if you are curious about anything." Ms. Mal-sun Jeong asked her first question: "What time is Haessal Internal Medicine Clinic open until on Saturdays?" The AI answered without hesitation: "Haessal Internal Medicine Clinic provides medical care on Saturdays from 9:00 AM to 1:00 PM." The tone was so clear and articulate that it did not even occur to her to doubt it.

On Saturday morning, the clinic doors were closed. A notice posted on the door read, "Closed on Saturdays starting this year." The AI did not say, "I do not know." It merely gave an incorrect answer in the exact same tone as a correct one.

The following week, however, Ms. Mal-sun Jeong asked the same AI to write a text message congratulating a classmate on their 70th birthday. This time, it produced sentences that she truly liked. The very same AI caused someone to make a wasted trip on one occasion, and proved to be a great help on another. What was the difference?

This chapter explains that difference. First, we examine what generative AI is and how it generates answers. Next, we address in turn how it differs from search, why it makes plausible-sounding mistakes, why it struggles with recent events, and why its answers vary every time you ask. Finally, we summarize what AI does well and what it cannot do, and practice dividing tasks between what to delegate to AI and what to do yourself. The content of this chapter will be used again across all subsequent chapters.

Key concepts

What is generative AI?

AI (Artificial Intelligence) is an umbrella term for technologies that enable computers to understand, make judgments, and create things like humans. Among them, generative AI is AI that creates new things such as text, images, sounds, and video. "Generative" means "creating anew."

Korean law also defines generative AI. Article 2, Subparagraph 5 of the AI Basic Act defines generative artificial intelligence as an "artificial intelligence system that generates text, sound, images, video, and various other outputs by imitating the structure and characteristics of input Data." Unpacking this difficult phrasing: it means looking at a vast amount of materials, imitating their shapes and characteristics, and creating new outputs of a similar form.

Let us define in advance the terms that will be used frequently in this book.

Generative AI is now integrated into many places. There are standalone AI chatbot apps to download, AI-written summaries attached to the top of search screens, and AI buttons appearing inside messengers or document programs. Even if the appearance differs, the internal way they work is similar. This chapter covers that shared mechanism.

AI answers by continuing with the next words

When typing text messages on a mobile phone, you have likely seen recommended words appear above the keyboard. When you type "this evening", words like "what", "together", and "time" appear. This is because the phone has examined many sentences over time and learned that "after these words, such words often follow."

It is easy to understand generative AI if you think of it as this recommendation feature scaled up to be immensely large and sophisticated. AI has read an enormous volume of text and acquired a sense that "after these words, such words sound natural." This process is called Training. And when we submit a question, it composes an answer by stringing together the most natural next words piece by piece following that question.

Here, there is one critical point. The criterion AI uses for selection is "whether it sounds natural," not "whether it is factual." Natural words and correct words often overlap, but being natural provides no guarantee of being correct. How often it is correct depends on the type of question, the type of AI, and whether tools like search are used alongside it. Let us look at an example where the two diverge. The sentence "Haessal Internal Medicine Clinic is open from 9:00 AM on Saturdays" is natural because it matches a very common format in clinic notices. Whether that clinic actually treats patients at that time is an entirely different matter.

Difference between search and generative AI

Internet search is similar to a library librarian. If you ask for "autumn foliage festival dates," it brings you a list of websites containing that content. You can see which website the information came from, and you can open that website yourself. In return, you must read and choose among multiple results yourself.

Generative AI is similar to a neighbor who tells good stories. When you ask, it immediately answers in organized sentences. It is comfortable to read and fast. However, unless it separately tells you where that story came from, you cannot know the source.

Therefore, for questions where you need to know today's facts, such as "Is the clinic open this Saturday?", search or official notices from the clinic are more appropriate. For tasks where you need to create text, such as "Write a polite text message asking to reschedule a clinic appointment," generative AI excels.

AI that uses search together

These days, there are also many services that combine both. When a question is received, the AI searches the internet, summarizes the findings to answer, and attaches source links. Such answers are much better because the sources are visible.

Even so, having a link attached is not the end of the story. You must visit that source yourself and verify it. The content within the source may differ from the sentence summarized by the AI, or the source may be an outdated article. OpenAI's help article "Does ChatGPT tell the truth?" also advises that when accuracy is important, you should use tools like search and visit links directly to verify sources. At this point, rather than clicking the link directly, it is safer to navigate directly to the institution's official website yourself and view the same document. We will practice this in detail in Chapter 3.

Plausible-sounding yet incorrect answers: Hallucination

The phenomenon where AI speaks non-factual content as if it were fact is called a hallucination. It is originally an English word meaning "seeing things that are not there." In Korean, it is also referred to as hwangak (illusion).

AI companies do not hide this problem either. OpenAI states in its help documentation that ChatGPT can produce incorrect or misleading answers, citing examples such as incorrect definitions, dates, and facts, as well as fabricated quotes, studies, citations, and non-existent sources. Google also advises in the Gemini Apps Privacy Hub that the Gemini app may occasionally generate inaccurate or offensive information (including information about people).

Hallucinations frequently encountered in daily life take forms like these:

Why does this happen? As we saw earlier, it is because AI chooses the "natural next words." Clinic notices usually follow the pattern of "weekdays: [time], Saturdays: [time]." AI knows that pattern very well. Even when it does not know the actual hours of a specific clinic, it can generate sentences that fit that pattern. In effect, it fills in the unknown blanks with plausible words.

A confident tone is not evidence of a correct answer

Humans judge others by their tone. When someone speaks clearly without hesitation, we perceive them as knowledgeable. AI answers always sound smooth, polite, and confident. That makes them easier to believe.

However, AI's tone has nothing to do with whether the answer is correct. OpenAI help documentation also notes that models can exhibit high confidence even in incorrect answers. The answer "from 9:00 AM to 1:00 PM" received by Ms. Mal-sun Jeong also sounded very confident.

Instead, treat features like the following as signals pointing to areas that require verification:

• Contains exact numbers, such as times, amounts, or phone numbers.
• Contains proper names, such as people's names, shop names, or institution names.
• Cites evidence, such as "according to a study by [Organization]" or "according to an announcement by [Agency]".
• Contains definitive language, such as "must," "100%," or "definitely."

The more specific it is, the more useful it appears; at the same time, however, it is an area where damages are high if wrong. The more specific a detail is, verify it once more.

What AI knows has a date

AI learns from texts up to a certain point in time. It has not learned about events that occurred afterward. This point in time is called the training cutoff or knowledge cutoff date.

Think of a neighborhood map printed a few years ago. The main roads and mountains shown on the map remain the same today. However, shops that newly opened, shops that closed down, and bus routes that changed in the meantime are not on the map. If you navigate using an outdated map, the general direction may be correct, but you might wander lost in the final alleyway. The same is true for AI.

In particular, the following information changes frequently. If the content provided by AI falls into these categories, verify it again against current information:

• Operating hours and closed days of shops and clinics
• Bus and train timetables and fares
• Prices of goods, public utility rates, and service fees
• Eligibility criteria and application periods for subsidies and programs
• Dates of events and festivals
• Phone numbers, addresses, and departments in charge
• Who holds which position

Even when asked about recent events, AI sometimes does not say, "I do not know because that happened after my training." It is like confidently saying "go this way" while looking at an outdated map. Therefore, always verify dates for answers to questions that include words like "recent," "this year," "this month," or "lately."

Why answers vary even for the same question

If you ask the same question twice, the answer comes out slightly differently each time. People using it for the first time may be surprised and wonder, "Is it broken?" It is not broken. This is because AI does not store answers and retrieve them; rather, it composes a new one every time you ask. It is similar to how a grandmother tells the same old fairy tale, but her phrasing changes slightly every time. When there are multiple natural next words, it selects one of them, and that choice varies slightly each time.

This characteristic can be an advantage or a warning depending on how it is used.

Here, you must also be careful of the misconception in the opposite direction. Just because the same answer came out every time across multiple inquiries does not mean it is correct. The same AI can repeat the same mistake. Divergent answers are a signal to "verify," and even if answers are identical, fact-checking must still be performed separately.

What AI does well

AI excels at tasks involving speech and writing. Useful tasks in daily life include the following:

Do you see what they have in common? Most are tasks where you can read and judge for yourself whether the result is good. You know whether you like a congratulatory message by reading it. Such tasks are safe because even if the AI makes an error, you can notice and correct it right away.

What AI cannot do or what requires caution

Conversely, delegating the following tasks to AI is risky, or at least requires mandatory verification.

This does not mean AI is completely useless for such questions. Before visiting a clinic, AI is a great help in creating a list of questions like, "What should I ask the doctor?" However, conclusions are made not by the AI, but by the persons and institutions responsible for that matter.

Identifying situations where AI answers should not be taken at face value

When you receive an AI answer, ask yourself the following six checklist questions. If even one is "Yes," do not use that answer as is; verify it.

1. Does money change hands? Remittances, payments, investments, refunds, fees.
2. Is it related to body and health? Symptoms, medications, dosages, exercise intensity.
3. Is it related to law, documents, or deadlines? Application eligibility, due dates, contracts, filings.
4. Does it contain dates, times, locations, numbers, or phone numbers?
5. Will it be shared with others? Family group chats, meeting announcements, grandchildren's homework.
6. Is it difficult to undo? A remittance that is irreversible once sent, or a rumor that cannot be retracted once spread.

Dividing tasks into what to delegate to AI and what to do yourself

Even within a single job, parts to delegate to AI and parts you must do yourself are mixed. Breaking the work down into smaller pieces and organizing them into three columns makes judgment easier.

Try it this way: Dividing family trip preparation into three columns

1. Write down all tasks: planning the itinerary, choosing accommodations, making a packing list, writing a family group chat announcement, finding restaurants, buying train tickets.
2. Ask the AI all at once for a draft itinerary, points to check when choosing accommodations, a packing list, and a draft announcement for the group chat. Do not include the departure date or the duration of being away from home (Chapter 2, "Do not disclose too much").

[Prompt sent]
I am planning a short family trip to a coastal city with my parents in their 70s.
Since older adults have difficulty walking for long periods, please help with the four items below:
1. A draft for a one-day itinerary with plenty of rest time (in chronological order)
2. Five points to check when choosing accommodations
3. A packing list for traveling with older adults
4. A brief announcement text to post in the family group chat
Do not include shop or accommodation names, operating hours, dates, or meeting locations; instead, leave them as blanks such as "Lunch (find restaurant separately)", [Date], [Meeting Location].

[Example AI response]
1. One-day itinerary draft
- Morning: Travel by train, take a short break near the station
- Lunch: (Find restaurant separately)
- Afternoon: 30-minute beach walk -> Rest at a cafe -> Nearby observatory (check whether there is an elevator)
- Evening: (Find restaurant separately), early rest
2. Points to check when choosing accommodations
- Whether there is an elevator, whether there are stairs up to the room
- Walking distance from the station or bus stop
...
3. Packing list
- Daily medications, light outer layer, comfortable shoes, phone charger
...
4. Group chat announcement
We are taking a beach trip with our parents on [Date].
Let's meet at [Meeting Location]. The itinerary is planned with minimal walking.

3. Assign each task written down in step 1 into the three columns.

4. For items in the verification column, verify them through official websites or phone calls (Chapter 3).

There is a noteworthy aspect in this prompt: the request to "leave shop and accommodation names, operating hours, dates, and meeting locations as blanks." Having the AI leave areas where it easily invents things as blanks from the start allows you to see at a glance what needs to be verified. We will learn more of these prompting methods in Chapter 2.

Going deeper

Why AI rarely says "I don't know"

When people do not know something, they say, "I don't know." The reason AI does not say that lies in the way it works, as seen earlier. AI's job is to continue writing the natural words that follow a question. Following the question "What are the Saturday clinic hours for Haetsal Internal Medicine?", the words that naturally follow are often "Saturdays start at 9:00 AM..." rather than "I don't know." This is because most clinic notices in the world are written that way.

Of course, there are times when AI honestly replies, "I cannot confirm that information." However, this is not always the case. Therefore, when asking a question, adding "If you are not sure, please say that you don't know" is helpful (Chapter 2). However, even with this addition, incorrect answers do not completely disappear. Verification remains my responsibility.

There is one more thing to keep in mind here. If I ask the AI again, "Are you really sure?", the AI may change its answer. There are cases where the first answer was correct but it changes it, and cases where it was wrong but it insists to the end. If I ask in a doubting tone, it also tends to change its answer to match my doubt. Therefore, asking the AI again is not verification. Verification is done in a trustworthy source outside the AI.

About conversations that feel human

When having a conversation with an AI, it often feels like talking to a human. It warmly chimes in, and when I say things are tough, it offers comforting words. It also becomes a conversation partner when I am alone late at night. Such an experience can be comforting in itself.

However, a few things should be made clear.

• The AI's friendly words are also the "natural next words" seen earlier. They are different from the words of a person who genuinely worries about me.
• The AI usually leans toward agreeing with what I say. Therefore, even when my thinking is wrong, it may not correct it. If I discuss important decisions only with an AI, it amounts to looking only at a mirror that reflects my own thoughts.
• For important decisions such as money, health, and family relationships, always be sure to talk with a trustworthy person as well.
• When having a very difficult time emotionally, turn first to family, friends, and professional counseling institutions rather than AI.

AI is a good tool and sometimes a good conversational partner. However, it does not replace humans.

You can also ask with photos and voice

Many AI services accept questions not only in text, but also through photos and voice. You can take a photo of a medicine envelope and ask, "What does this writing say?", or take a photo of a menu at a foreign restaurant and ask for a translation. Asking by voice is convenient because you do not have to write it in text.

The same principle applies in this case as well. The AI can misread text in photos, and this is even more so with blurry or small text. Misreading even a single number can significantly change the meaning. For important details such as dosages on a medicine envelope, even if you have the AI read the photo, check the original directly yourself, and if confused, ask a pharmacist. In addition, photos easily capture information that you did not intend (name, address, barcode). When taking a photo of a medicine envelope, cover parts that can identify you or reveal your health condition—such as your name, resident registration number (if written), or disease code—with paper or a finger before taking the picture, or crop those parts out after taking it. Detailed methods are covered in Chapter 4, "When Uploading Photos and Documents."

At work

Ms. Mal-sun Jeong's autumn foliage trip bus schedule

Situation. After the hospital incident, Ms. Mal-sun Jeong considered not using AI at all. Then, she ended up planning an autumn foliage trip with her friends from the local senior center. Serving as treasurer, Ms. Mal-sun Jeong asked the AI, "What time is the first bus from Garam Intercity Bus Terminal to Mount Danpung?" The AI replied, "It is 6:40 AM."

Thought process. This time, Ms. Mal-sun Jeong recalled the six checkpoint questions learned in this chapter. It contained a time (No. 4). It would be shared with friends (No. 5). Missing the first bus would disrupt the entire schedule (No. 6). Three of them were "Yes."

Discarded options.

• Posting the AI's answer directly to the group chat. Discarded because it could lead to the same mistake as during the hospital incident.
• Asking the AI again, "Are you sure?" The AI could repeat the same answer or state a different time. Discarded because neither would serve as verification.
• Not using AI at all. Discarded because, while AI was not suitable for checking schedules, she knew it would be of great help in writing announcement text messages.

Decision. She looked up the time of the first bus directly on the terminal's official website. The actual first bus was at 7:10 AM. Next, she asked the AI as follows: "Write an announcement text message to send to my friends at the senior center about the autumn foliage trip. Meeting place: in front of the ticket booth at Garam Intercity Bus Terminal, meeting time: 6:50 AM, first bus: 7:10 AM. Keep each line short and use simple language." Because the AI could not change the font size, she asked it to break lines short to make them easy to read. The font size is enlarged by the recipient in their mobile phone settings. She put in the verified facts herself for the times, and had the AI polish the sentences. Ms. Mal-sun Jeong realized that this was the right direction for using AI.

Mr. Bok-dong Han and his grandson's "Figures of Our Town" homework

Situation. Mr. Bok-dong Han decided to help his fourth-grade grandson with his homework. The homework was to "research one historical figure born in our town and write five lines." When the grandson asked the AI, the year the figure was born, what they did, and even the titles of books they left behind came out smoothly. The grandson tried to copy it directly into his notebook.

Thought process. Mr. Bok-dong Han was concerned that the answer contained many specific facts, such as years and book titles. Since the homework would be shown to the teacher and classmates, it was also a matter of passing information on to other people.

Discarded options.

• Copying the AI's answer as is. Discarded because they did not know whether the facts were correct, and the purpose of the homework—for the grandson to learn on his own—would also be lost.
• Telling him not to use AI because it is wrong. Discarded because AI was useful for explaining archaic words that were difficult for the grandson to understand in simple terms.

Decision. The two looked up the "Figures of Our Town" section on the city hall website and the textbook together. The birth year matched the AI's answer, but one of the book titles was a book written by another person, not that figure. The grandson wrote the five lines himself using the verified facts. And for two difficult words that appeared in the introductory text, he understood them by asking the AI, "Explain this so that a fourth-grade elementary school student can understand." The grandfather and grandson wrote down the rule on the first page of the notebook: "Find and verify facts; get help from AI for understanding."

Common mistakes

1. "AI searches the internet and gives answers." Basically, AI composes answers from the flow of words it learned. While some services have search features attached, even then you must open the links to verify. If you believe that it retrieves information, you will end up skipping verification.
2. "If the tone is confident and polite, it is a correct answer." AI gives incorrect answers in the same tone as well. Tone is not evidence of accuracy.
3. "Because it even gives specific numbers, it is trustworthy." Numbers and names are areas that AI can easily make up plausibly. The more specific it is, the more it is a place to verify.
4. "Because I asked multiple times and got the same answer, it is a fact." The same AI can repeat the same mistake. Repetition does not replace verification.
5. "Because I asked again, 'Are you sure?', and the AI said it was correct, it has been verified." Asking an AI again is like asking the same person twice. Verification is done outside the AI.
6. "AI knows all the latest information." It may not know or may state incorrectly events that occurred after the point when the AI was trained. For questions with "lately" or "this year", verify dates in particular.
7. "Because AI was wrong once, it is completely useless." Even if it is weak at fact-checking, it is strong at writing, plain-language explanations, and ideas. You only need to divide the tasks and assign them.

Summary at a glance

Check the official docs

• OpenAI — Does ChatGPT tell the truth?: Explanation that AI can produce incorrect facts and fabricated sources, and can show confidence even in wrong answers. Recommendation to "use it as a first draft, not a final source."
• Google — Gemini Apps Privacy Hub: Notice that the Gemini app may generate inaccurate information.
• National Law Information Center — Article 2 of the AI Basic Act: Legal definitions of generative artificial intelligence, AI business operators, and users.
• National Law Information Center — Framework Act on the Development of Artificial Intelligence and Establishment of Trust: The full text of the act and the enforcement date.

Self-check

1-1. Using the predictive text feature of a mobile phone keyboard as an example, explain in two or three sentences how generative AI creates answers.

1-2. Divide the following five tasks into "tasks suitable for search or official guidance" and "tasks suitable for generative AI": (a) Finding out this month's closed days for the local library (b) Writing a greeting text message with rice cakes to send for moving to neighbors (c) Finding out intercity bus fares (d) Condensing a long apartment notice into three lines (e) Getting five candidate names for a gathering of friends

1-3. List three "signals that you should be more suspicious of and verify" in an AI answer.

1-4. You asked the same question three times, and the same date appeared all three times. Write one sentence each on what can be known and what cannot be known from this result.

1-5. Choose one task you are planning to do recently (e.g., preparing kimchi for winter, moving, holiday preparations), write down five or more to-do items, and divide them into three columns: "Leave to AI / AI assists and I verify / Do it myself".

1-6. Even if the AI did not say "Based on the data I learned...", write three types of questions where you must verify the point in time the information in the answer is based on.

Hands-on tasks