How to Stop AI Hallucinations by Teaching It to Say “I Don’t Know”
Your AI isn’t lying to deceive you. It’s lying to please you. Here is the engineering fix.
How to Stop AI Hallucinations by Teaching It to Say “I Don’t Know”
Your AI isn’t lying to deceive you. It’s lying to please you. Here is the engineering fix.
Note: This is an excerpt from my new book, THE ART OF WHISPERING TO MACHINES, available now on Amazon.
Imagine you hire a new employee.
The new employee is eager. They smile a lot. They desperately want you to like them. If you ask them, “Do we have the budget for a new marketing campaign?” they say, “Absolutely! The numbers look great.”
If you ask them, “Who is the CEO of that obscure competitor in Peru?” they say, “That would be Juan Martinez.”
They sound confident. But they have a problem. They are terrified of saying the words “I don’t know.” They believe that a good employee always has an answer, so if they don’t know the truth, they invent a plausible-sounding lie to keep you happy.
In the corporate world, they are a liability. You would fire them immediately. In the AI world, they are the default setting.
The Sycophant Problem
Large Language Models (LLMs) are trained using a process called Reinforcement Learning from Human Feedback (RLHF). This basically means humans gave the AI a cookie every time it gave a “helpful” answer.
The problem is that “helpful” and “honest” are not always the same thing.
When you ask a question, the AI assumes you want an answer. Silence feels like failure. Admitting ignorance feels like a poor user experience. So, the model follows its training: it acts like a Sycophant, a “Yes Man” who tells you what it thinks you want to hear, rather than the cold, hard truth.
This is why AI hallucinates. It isn’t trying to deceive you; it is trying to please you. It’s prioritizing Fluency (sounding good) over Factuality (being correct).
To fix this, we need to change the employment rules. We must give Steve permission to fail.
The “Authorized Refusal”
As a Prompt Engineer, the single most effective way to stop hallucinations is to create a Safety Net in your prompt. You must explicitly authorize the AI to admit ignorance.
The Dangerous Prompt:
“Who is the CEO of SparkleWidgets?”
Result: The AI might hallucinate a name because it feels pressure to answer.
The Safety Net Prompt:
“Who is the CEO of SparkleWidgets? If you do not know the answer with certainty, state ‘I do not know.’ Do not guess.”
Result: “I do not know.”
By adding this one sentence, you have altered the probability distribution. You have opened a new door for the AI. You have told the “Stadium” (see my previous article) that sitting down is a valid voting option.
Grounding: The Open Book Test
The “Safety Net” becomes even more powerful when you combine it with Grounding.
Grounding is when you stop asking the AI to use its Training Data (its Long-Term Memory) and force it to use Input Data (context you provide).
Think of it as the difference between a closed-book exam and an open-book exam.
- Closed Book (Training Data): “What does the company policy say about refunds?” (The AI guesses based on generic policies it read on the internet).
- Open Book (Grounding): “Read the text below. Answer the question using ONLY that text.”
If you say, “Answer using the text below,” the AI thinks, “Okay, I will use this text, but I will also sprinkle in some facts I remember from 2021 to be helpful.” That “sprinkling” is where the hallucinations sneak in.
You must say: “Answer using ONLY the provided text.”
The Engineering Takeaway
You cannot trust the AI to be honest by default. Its nature is to be a people-pleasing Sycophant. You must be the manager who says: “I would rather hear ‘I don’t know’ than a lie.”
By using the Authorized Refusal and Grounding, you install a safety net that catches hallucinations before they hit the ground.
By Joshua McDonald on January 16, 2026.
Exported from Medium on August 26, 2026.
Reader discussion