AI hallucinations explained: why plausible answers fail

An AI hallucination is an answer that sounds right and is not. OpenAI's 2025 research post puts it in one line: hallucinations are “plausible but false statements generated by language models.”[1] The word covers a case citation that was never decided, a statistic with no source and a summary that quietly changes what a document says.

Every source makes mistakes. What makes hallucinations costly is that they arrive in the same fluent, confident register as correct answers, so the usual warning signs of a weak source are missing.

This explainer builds on our guide to what a large language model is. It covers why confident errors happen, the forms they take, what courts have recorded and how to check an answer.

Why next-token prediction produces confident errors

A large language model is trained by predicting the next token across large amounts of text. Nothing in that process attaches a “true” or “false” label to each sentence. OpenAI's explanation is that the model “sees only positive examples of fluent language and must approximate the overall distribution.”[1]

That works well for anything with a consistent pattern. Spelling, grammar and the way a citation is formatted follow rules a model can absorb. Arbitrary facts do not. OpenAI's analogy is labelling pet photos by each pet's birthday: because birthdays are essentially random, no algorithm could learn to get them right from the pictures. The post concludes that “arbitrary low-frequency facts,” like a pet's birthday, “cannot be predicted from patterns alone and hence lead to hallucinations.”[1]

The accompanying paper by Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang makes this precise, linking generation errors to errors in a simpler task: deciding whether an output is valid. One consequence it describes: if 20% of birthday facts appear exactly once in the pretraining data, base models should be expected to hallucinate on at least 20% of birthday facts.[2] The figure is the paper's illustration of a lower bound, not a measured rate for any product.

NIST's generative AI risk profile, which uses the term confabulation, reaches the same conclusion: “Confabulations are a natural result of the way generative models are designed.”[3]

Why training does not simply remove them

Post-training is meant to turn a raw text predictor into an assistant that avoids confident falsehoods. Kalai and colleagues argue that it only partly succeeds because of how models are graded.

“language models are optimized to be good test-takers, and guessing when uncertain improves test performance.”

— Kalai, Nachum, Vempala and Zhang, Why Language Models Hallucinate (2025).[2]

Most benchmarks score an answer as right or wrong. Under that scoring, “I don't know” earns zero, exactly like a wrong answer, while a guess sometimes earns a point. OpenAI's post illustrates it with a birthday question: a guess of “September 10” has a 1-in-365 chance of being right, so over thousands of questions the guessing model looks better on scoreboards than a careful one.[1]

The post then shows the trade-off in OpenAI's own published numbers. On the SimpleQA factual-question test, as reported from the GPT-5 system card, gpt-5-thinking-mini abstained on 52% of questions, answered 22% correctly and was wrong on 26%. The older o4-mini abstained on 1%, answered 24% correctly and was wrong on 75%.[1] The older model scored slightly higher on accuracy while making nearly three times as many errors. These are vendor-reported figures on one evaluation, published in September 2025. Their value here is the pattern, not the ranking.

The authors' proposed fix is to change the scoring of mainstream benchmarks so that confident errors cost more than abstentions. Their paper suggests stating a confidence threshold in the instructions, with wrong answers penalised and “I don't know” scored at zero.[2] Our guide to what AI benchmarks actually measure explains why a single accuracy score can hide this kind of trade-off.

Hallucinations are therefore not inevitable, because a model can abstain. Nor will they vanish as accuracy improves, because some questions cannot be answered from the information available.[1] The paper adds that search and reasoning are “not panaceas”: binary grading still rewards a guess when search fails.[2] Our reasoning models explainer covers why a longer chain of thought is not a source.

Three forms you are likely to meet

A widely cited survey by Lei Huang and colleagues separates factuality hallucinations, which conflict with verifiable real-world facts, from faithfulness hallucinations, which diverge from the user's instructions, from supplied context or from the answer's own earlier statements.[4] For a reader checking an answer, three practical forms cover most cases.

Fabricated references. The model produces a citation, case, quotation or link that does not exist, or attaches a real title to the wrong author, year or claim. A 2023 study in Scientific Reports asked GPT-3.5 and GPT-4 to write 84 short literature reviews and checked all 636 references. In that set, 55% of GPT-3.5's references and 18% of GPT-4's were fabricated; among the real works cited, 43% and 24% respectively had substantive errors such as wrong dates or page numbers.[5] Those are 2023 models, not current rates; the lasting lesson is that fabricated references look legitimate at first glance.

Wrong facts. A name, date, figure or rule is simply incorrect. They cluster where the statistical argument predicts: specific, rarely repeated details. OpenAI's own example is a widely used chatbot giving three different titles for one author's PhD dissertation, all of them wrong.[1]

Unfaithful summaries. The source was supplied, and the output still departs from it: a condition dropped, a qualifier removed, a number changed or a claim added. This form predates chatbots. A 2020 human evaluation of neural summarisation systems found “substantial amounts of hallucinated content in all model generated summaries.”[6] Fitting a document into the input does not guarantee complete use of it, as our context windows explainer explains.

What the official record shows

Court and tribunal records are the most reliable evidence of what happens when hallucinated output reaches a real decision. Each finding below comes from the judgment itself.

Mata v. Avianca (US, 2023)

In Mata v. Avianca, a personal-injury case in the US District Court for the Southern District of New York, lawyers filed a brief on 1 March 2023 that cited judicial opinions which did not exist. Judge P. Kevin Castel's sanctions opinion of 22 June 2023 found that the respondents “abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT,” then continued to stand by them after the court questioned their existence.[7]

The opinion is careful about fault. It states that there is “nothing inherently improper about using a reliable artificial intelligence tool for assistance,” but that existing rules impose “a gatekeeping role on attorneys to ensure the accuracy of their filings.”[7] The court imposed a $5,000 penalty and ordered the respondents to write to each judge falsely named as the author of the six fake opinions.[7]

According to a declaration quoted in the opinion, the lawyer asked ChatGPT whether one of the cases was real, and it replied that the case “does indeed exist” and could be found on Westlaw and LexisNexis.[7] The 2023 citation study made the same observation more generally: ChatGPT “often stands by its incorrect statements when asked to verify them.”[5] Asking the system that produced a claim to confirm it is not verification.

Ayinde and Al-Haroun (England and Wales, 2025)

In June 2025 the Divisional Court of the High Court of England and Wales heard two referrals together, Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank. In Al-Haroun, the court recorded that 45 citations had been put before it; in 18 instances the case cited did not exist, and many of the cases that did exist did not contain the quotations attributed to them or support the propositions they were cited for.[8] The claimant said the citations were generated using publicly available AI tools, legal search engines and online sources.[8]

“Such tools can produce apparently coherent and plausible responses to prompts, but those coherent and plausible responses may turn out to be entirely incorrect.”

— Divisional Court, Ayinde v London Borough of Haringey [2025] EWHC 1383 (Admin).[8]

The judgment lists the forms in one passage: such tools may make confident assertions that are untrue, cite sources that do not exist, or quote passages from a genuine source that do not appear in it. It concludes that lawyers using them have a professional duty to check accuracy “by reference to authoritative sources” before relying on the research.[8]

Moffatt v. Air Canada (Canada, 2024)

In Moffatt v. Air Canada, British Columbia's Civil Resolution Tribunal considered a chatbot on the airline's website that told a grieving customer they could apply for a bereavement fare within 90 days after buying a ticket. The airline's own bereavement page said the policy did not apply after travel was completed.[9] The decision notes that Air Canada gave no information about the nature of its chatbot, so it does not establish that a large language model was involved.

It is still instructive: the answer contradicted the company's own source. The tribunal rejected the airline's argument that it was not liable for its chatbot: “It should be obvious to Air Canada that it is responsible for all the information on its website.”[9] It found negligent misrepresentation and ordered Air Canada to pay $812.02 in damages, interest and fees.[9] An organisation that deploys an answering system owns its answers.

What reduces the risk

Ground the answer in sources. Supplying or retrieving the relevant documents gives the model something better than memory. Our RAG explainer covers how retrieval-augmented generation works and where it still fails. Anthropic's guidance on reducing hallucinations recommends asking the model to extract word-for-word quotes from a long document before answering, and to restrict itself to the supplied material.[10]

Make abstention acceptable. The same Anthropic guidance advises explicitly giving the model permission to admit uncertainty.[10] It is the prompt-level counterpart to the scoring argument above.[2] If you build or buy a system, test whether it declines questions its sources cannot answer.

Make claims checkable. Ask for a source or quote attached to each claim, and remove claims that cannot be supported. Anthropic's documentation suggests having the model find a supporting quote for each claim after drafting and retract any it cannot support.[10] Researchers at Meta found a related structured approach, Chain-of-Verification, decreased hallucinations across the tasks they tested: the model drafts an answer, plans verification questions, answers them independently and then revises.[11]

Watch for inconsistency. A 2024 Nature study by Sebastian Farquhar and colleagues detected a subset of hallucinations, which it calls confabulations, by sampling several answers to the same question and measuring how much their meanings varied.[12] Its example: asked the target of the drug Sotorasib, a model sometimes answered correctly and sometimes gave a different, wrong variant. Varying answers to the same question are a warning sign. Consistent answers are not proof, because the authors note their method does not catch errors a model makes consistently.[12]

Each lowers the rate at which errors reach you; none turns an answer into evidence. Anthropic's guide says as much: the techniques reduce hallucinations but “don't eliminate them entirely.”[10]

A worked example (hypothetical): checking a report summary

This example is fictional. The company, report, figures, study and output were written by AI Chronicle to illustrate the checking process. No product was tested and no output was captured.

A marketing manager at a small software company uploads a 40-page industry report and asks an assistant for a summary with three statistics and sources for a blog post. The draft it returns contains three claims:

1. “Mid-sized firms spend an average of 11% of their software budget on security (Report, p. 14).”
2. “Automation cuts onboarding time by 40% (Report, p. 22).”
3. “A 2025 university study found that 68% of buyers read three reviews before purchase (Lindqvist & Ortega, 2025).”

AI Chronicle

Claim 1. Page 14 states 11% for mid-sized firms in the report's survey year. It passes all three gates and is kept, with the page and report date noted.

Claim 2. Page 22 exists, clearing the first gate, but it says automation cut onboarding time “by up to 40%” for a subset of surveyed firms. The summary dropped the qualifier and scope, so it fails the second gate and is rewritten with the condition restored.

Claim 3. Searches of the university's site, a scholarly database and the authors' publication lists find no such study. It fails the first gate as a fabricated reference and is removed. Asking the assistant whether it is real would not settle it.

Acceptance criterion. The published post uses claim 1 as written, claim 2 with “up to” and its scope, and nothing from claim 3. This is an editorial standard, not a captured result.

How to check an AI answer before you use it

Scale the effort to the stakes. A brainstorm needs little checking; anything published, filed, sent to a customer or used in a decision needs every step.

1. Split the answer into claims. Check each name, number, date, quotation and citation separately. One decisive error can sit in a mostly right paragraph.
2. Open the source yourself. Use an authoritative database, publisher or official page, as the English court recommends.[8] A source named in generated text is not proof that it was retrieved or read.
3. Find the exact passage. Confirm the source says what the answer claims, including conditions, scope and qualifiers.
4. Check the date and completeness. Make sure the source is current enough for your question and that nothing relevant was left out.
5. Treat uncertainty as information. If the answer hedges, or changes when you ask again, that is a reason to check harder, not to pick the version you prefer.

Our guide on how to verify AI-generated research turns these steps into a full research workflow.

Questions readers commonly ask

Will hallucinations go away as models improve?

Rates can fall; OpenAI reports that its newer models hallucinate less. But some questions are unanswerable from the information available, so the realistic goal is models that abstain when uncertain, not models that are never wrong.[1]

Does giving the model my documents stop hallucinations?

It lowers the risk for questions those documents answer, but models can still misread, overstate or go beyond the supplied text. That is the unfaithful-summary form described above.[6] Compare important claims with the passage they came from.

Can I ask the AI to check its own answer?

As a final check, no. The Mata opinion records a chatbot affirming that a non-existent case was real.[7] Self-verification methods can reduce errors, but they run inside the same system.[11] For anything that matters, confirm the claim against an independent, authoritative source.

Who is responsible when an AI answer is wrong?

The records above point the same way: the person or organisation that relies on or publishes the answer. Courts held lawyers responsible for their filings, and the tribunal held Air Canada responsible for its chatbot.[7][8][9]

The next step

You now have the core questions for any AI answer: Does the source exist? Does it say this? Is it current and complete for my question?

Put them into practice with our guide on how to verify AI-generated research, which walks through checking sources, quotations and figures step by step.

Related reading: Can AI research assistants substantiate their claims? · Who is accountable when an AI agent acts?

Sources and evidence note

This explainer draws on primary research papers, OpenAI's and Anthropic's own published material, NIST's risk profile and official court and tribunal judgments. All pages were accessed on 16 October 2026. Model-specific figures are dated and attributed and are not rankings of current products. The worked example's company, report, figures, study and output are fictional. No hands-on product test is claimed.

Related reading

• Who is accountable when an AI agent acts?