In our last post we compared an AI model to a chef who has tasted millions of dishes but never read a single recipe. Ask that chef for something real, and you get a genuinely good answer. Ask for something that does not exist, and the chef will not say "I don't know." The chef will invent a dish that sounds completely plausible. That invention, stated with total confidence, is what the industry calls hallucination. It is the single most important thing to understand before you trust AI with anything that matters.
A real case that shows exactly how expensive this can get
In February 2026, an Omaha, Nebraska lawyer named Greg Lake filed a divorce appeal brief that leaned on AI for legal research. Of the 63 citations in that brief, 57 turned out to be defective: cases and quotes that did not exist, invented by the AI tool he used. He initially denied using AI when questioned about it. Two days before the Nebraska Supreme Court acted, he admitted the truth. On 16 April 2026, the court suspended him from practicing law, pending a full disciplinary hearing.
This was not an isolated accident. A public database maintained by legal researcher Damien Charlotin has tracked roughly 1,490 court cases worldwide, most of them in the United States, where a lawyer relied on AI-invented material and a court caught it. Lawyers who already knew AI could hallucinate, and had been warned by judges, kept citing fake cases anyway. That gap between knowing the risk exists and actually checking for it every single time is the exact gap this post wants to close for you.
Hallucination is not AI lying to you on purpose. It is AI doing exactly what it was built to do: produce the most fluent, plausible-sounding next words, whether or not those words are true.
Why this happens: fluent is not the same as correct
Remember the core idea from the last post. An AI model does not store facts in a filing cabinet and look them up. It predicts the next most likely word based on patterns in what it has read. Most of the time, the most likely-sounding answer is also the correct one, because true things tend to appear in text more often and more consistently than false things.
But that link breaks down in a few predictable situations, and recognising them is most of the skill.
1. You are asking about something rare or specific
A well-known Indian tax provision or a famous court judgment appears in the model's training data thousands of times, so it gets the pattern right. A niche circular, a specific page number, or a smaller company's internal policy appears rarely or never. The AI does not have a way to say "I have not seen this enough to be sure." It fills the gap with something that sounds like the kind of thing that should exist there.
2. You are asking for exact numbers, dates or citations
This is exactly what tripped up the Nebraska brief. Case names and citation formats are highly patterned, so AI is excellent at producing something that looks like a real citation. But looking right and being right are different tasks, and AI is only reliably good at the first one unless it is actually searching a live, verified database as it answers.
3. You are asking about very recent events
A model's knowledge stops at its training cutoff. Ask it about something that happened last week without giving it search access, and it may either say it does not know, or worse, blend older patterns into a confident-sounding but wrong answer about a recent event.
4. Your question nudges it toward a particular answer
Ask "why is Company X's stock falling", assuming it is falling, and many AI tools will find you reasons, even if the stock is flat or rising. The model is completing your pattern, not fact-checking your premise.
How to actually catch it: a habit, not a tool
There is no button that turns hallucination off. What works is a simple habit: treat every specific, checkable detail in an AI answer as unverified until you have verified it yourself. Names, numbers, dates, case citations, section numbers, quotes attributed to a person or report: check these particular things every time, even when the rest of the answer is clearly excellent.
This one prompt is worth building into your routine. It forces the AI to separate its checkable claims from its general reasoning, which makes verification faster than re-reading a full paragraph looking for the one wrong number.
A second habit that helps: ask the AI to rate its own confidence, and treat a high confidence score with mild suspicion rather than full trust. Confident language is what the model is built to produce regardless of whether it is right, so a confident tone is not evidence of accuracy on its own.
Where hallucination matters most at work
Not every task carries the same risk. A first draft of a marketing email can tolerate a small error, because you will read it before sending. A number that goes into a client's financial statement, a legal citation in a filing, or a compliance date you repeat to your manager cannot. The Nebraska case is a useful gut check: would you be comfortable if a court, a client or your CFO checked this specific detail against the original source? If the honest answer is no, you have not finished the task yet. You have finished the first draft.
A quick filter for how much to trust an answer
Ask yourself three questions before you use an AI-generated fact anywhere that matters. Is this the kind of detail that is common and well documented, or rare and specific? Did I give the AI the source material directly, or is it recalling this from memory? What actually happens if this one detail turns out to be wrong? The more specific, unsourced and consequential the detail, the more it deserves a direct check against the original document, website or law.
The takeaway
AI hallucination is not a temporary glitch that better models will fully solve. It is a direct consequence of how these tools work: they generate the most fluent likely answer, not the most verified one. Understanding this does not mean trusting AI less. It means trusting it correctly, using it heavily for the pattern-shaped work it is genuinely excellent at, and building the habit of checking the specific, checkable details before they leave your desk. This is precisely the kind of judgment the AiM AI Fluency Test measures: not whether you can get an answer out of AI, but whether you know which parts of that answer to trust.
In the next post in this series, we will move from understanding AI to directing it well: how to write your first genuinely good prompt. If you want a structured, guided path through all of this rather than piecing it together post by post, our AI courses are built exactly for working professionals who want to use AI properly at work.