Trigpoint
Collections › Software & Systems Architecture › Hallucination in NLG
“A bit more progress and it will stop hallucinating”: it will not carry that.

Hallucination in NLG

Hallucination in NLG · Ji et al. (2023)
Abstracts checked ✓ Machine learning · failure modes Read first: Retrieval-augmented generation (RAG) Tell the editor privately ▸
In three sentences Hallucination is generation that reads fluently while being unfaithful to its source, or making no sense at all. The danger is not that it is wrong. It is that it is wrong fluently: fluency lends an appearance of credibility that the truth value does not support. And its causes span the data, training and decoding layers, so for now it can be narrowed, and it should not be assumed to have reached zero.

↳ Seen a counter-example?

Holds when
When checking model output and you have a clear source to compare it against.
Evidence
Conceptual framework; Abstracts checked
Breaks down when
In open-ended generation with no clear source, the two types apply only by extrapolation; faithful to the source is not necessarily factual.
Hallucination in NLG Ji et al. (2023) Causes sit in three layers, and so do the remedies Data layer mainly source vs reference mismatch Training layer representation, exposure bias Decoding layer diverse sampling (conjectured) Beyond its reliable range seldom “I do not know” it keeps sampling fluently Intrinsic: contradicts source Extrinsic: unverifiable contradiction by comparison, omission by existence check Each layer has mitigations; there is no single fix change the evidence supply, the objective, the decoding; it narrows; never assume it is zero Figure: trigpoint.pub
Figure 1 · The figure splits the survey's two broad groups of causes into three layers: in the data layer the main cause is a mismatch between source and reference (with unfiltered duplicates as another case); the training layer has poor representation learning, exposure bias and parametric knowledge bias; and at decoding, diversity-oriented sampling correlates with hallucination (the mechanism is the survey's conjecture). Each layer has its own mitigations, there is no single point of repair, and no layer claims to reach zero. The three layers feed into the middle, which marks the least intuitive thing on this page: beyond its reliable range the model tends not to say I do not know and carries on sampling fluently, a tendency tied to training and grading that reward guessing. The hallucinations produced this way split into two types, marked on the right: intrinsic, which contradicts the source, and extrinsic, which cannot be verified from the source. The mechanism is a conceptual sketch and deliberately carries no hallucination rates or proportions. View full size ▸

1. What happened

The definition adopted by the 2023 survey by Ji and colleagues in ACM Computing Surveys is generated content that is nonsensical or unfaithful to the provided source content[1]Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730. Bibliographic details checked against the official record. Claims matched against the full text. The journal version has ten authors; the preprint arXiv:2202.03629 was first posted 2022-02-08, and later versions added authors, thirteen by the July 2024 version. The definition the survey adopts in §2 is taken from earlier studies: generated content that is nonsensical or unfaithful to the provided source content.. The classification has two types: intrinsic, which contradicts the source, and extrinsic, which cannot be verified from the source[2]The definitions of the classification (the paper's §2.1): intrinsic is output that contradicts the source content; extrinsic is output that cannot be verified from the source content..

The mechanism is worth going through slowly, because it decides what you can do about it. A generative model is fundamentally a compression of its training distribution, sampling one token at a time at decoding. When a task needs information beyond what the model has reliably compressed (out of distribution, in the long tail, or the world changed after training), the model tends not to stop and say I do not know. It carries on sampling fluently. That is hallucination. Part of that tendency is rewarded into it: a 2025 study argues that current training and grading give credit for a guess and none for admitting uncertainty, so models lean towards guessing when unsure[6]Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. Bibliographic details checked against the official record. Consistent with the abstract. The gist of the abstract: models sometimes guess when uncertain instead of admitting uncertainty, and the authors argue the cause is that training and evaluation procedures reward guessing over acknowledging uncertainty.. It is not that a model cannot say it does not know; under the current rewards it seldom does.

The survey by Ji and colleagues sorts the causes into two broad groups: data, and training and inference. This page splits the second into training and decoding, giving three layers, which is why there is no single point of repair: the data layer (mainly source and reference that already disagree; duplicates left unfiltered during data collection are another case), the training layer (poor representation learning, exposure bias between training and generation, and a model leaning on knowledge from pre-training rather than the source in hand), and the decoding layer (diversity-oriented sampling correlates with hallucination, though the survey marks the mechanism as conjecture). The mitigations can be divided the same three ways: change the supply of evidence (retrieval augmentation, see Retrieval-Augmented Generation), change the objective (train against a measure of faithfulness), and change decoding (constrain generation).

Attribution where due: Assembling the survey belongs to Ji and nine co-authors, in Survey of Hallucination in Natural Language Generation, ACM Computing Surveys, 2023[1]Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730. Bibliographic details checked against the official record. Claims matched against the full text. The journal version has ten authors; the preprint arXiv:2202.03629 was first posted 2022-02-08, and later versions added authors, thirteen by the July 2024 version. The definition the survey adopts in §2 is taken from earlier studies: generated content that is nonsensical or unfaithful to the provided source content..

2. Why this matters to you

What this page gives you is not a warning. It is a calibration: how much to discount fluency by.

When people judge credibility, fluency of language and tidiness of structure are strong cues. Those cues work reasonably well on things written by people, because people talking nonsense usually also talk messily. A generative model severs that correlation: its fluency has almost nothing to do with whether the content is true. So your intuition here is not unreliable. It is calibrated on the wrong distribution.

The second thing is the practical consequence of it seldom saying I do not know. You cannot rely on the model raising a flag to catch hallucination: when unsure it tends to guess, and a wrong guess reads just as fluently. That puts the burden of checking on your side, and it has to be a process rather than vigilance: vigilance fatigues, a process does not.

The third thing is where this shelf takes its position: since no layer can guarantee it has reached zero, and the model will not reliably raise a flag, trustworthy output cannot rest on a guarantee about model quality. It can only rest on checkability.

In practice two moves are enough. First, turn where is the basis for this sentence into a fixed question, rather than picking sentences to check by feel. Second, handle the two types separately: what contradicts the source you gave (intrinsic) is caught by comparison; what the source never mentions (extrinsic) is caught by checking existence. The two need different methods, and blending them leaks on both sides. So before you start checking, ask one question first: do I have a source in hand to compare against.

Verification layer · Members
The guided layer of this node is open to all readers.
Verification details and failure boundaries are for members.
US$8.25 a month on average · or subscribe monthly ▸

Lab cases Written only where there is evidence

No field record for this node yet.

Common misreadings

Page verification: Abstracts checked ✓ · Page version v2 · Content last revised 2026-11-14 · Corrections ↓

  • A common account says the intrinsic and extrinsic classification was proposed by the survey by Ji and colleagues. The paper's own §2.1 states that it follows the classification of earlier studies. This site treats it as the anchor that organised the whole field; it is not the originator of the classification. The definition of hallucination the survey adopts in §2 is likewise taken from earlier studies. #
  • A common account says newer models no longer hallucinate. What the survey by Ji and colleagues lists is causes and mitigations by type, and no layer claims it can reach zero. This site therefore treats it as a standing risk that can only be narrowed and must not be assumed to be at zero; that is this site's design premise, not a conclusion of the survey. The point is not pessimism. It is the premise for deciding whether to build a checking process at all. #
  • A common account says the term only arrived in the era of large language models. An earlier attested usage appears, for example, in a 2018 paper on a vision-language task; the survey itself records that the term first appeared in computer vision research in 2000, with a more positive meaning at the time. #
Verification record

Records how far each claim was checked, not a guarantee that it is true.

Corrections
  • 2026-10-12 · An earlier version of this page went beyond the 2023 survey by Ji and colleagues in three places, and all three have been corrected against the full text. First, it presented hallucination arising from compression plus sampling, narrowable but not reducible to zero, as the source's conclusion; the survey lists causes and mitigations by type and does not claim hallucination cannot be removed. The shelf's position is kept, but its premise is now that no layer can guarantee zero and the model does not reliably raise a flag, and it is marked as this shelf's own position. Second, it gave the training-layer cause as a maximum-likelihood objective that rewards fluency over faithfulness; the survey lists poor representation learning, exposure bias and parametric knowledge bias, and names a mismatch between source and reference as the main data-layer cause. Third, it said faithfulness and factuality are not the same thing; the survey says they may or may not be the same.
References

How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Consistent with the abstract” means the claim agrees with the paper's abstract. It does not include sentence-by-sentence checking against the full text. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.

  1. [1] Ji, Z. et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730.
  2. [2] Ji, Z. et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730.
  3. [6] Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. https://arxiv.org/abs/2509.04664 ↩

↳ Is a citation wrong?

Tell me when a new shelf opens

Tell me when a new shelf opens

Shelf notices are sent only when a new shelf opens or when the reading rules change.

Trial

Tell the editor privately: a correction, a counter-example or where it applies

Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.

What are you submitting?
Set a public name to be credited by name ▸
Submit anonymously, without signing in

Signing in lets you follow the status on your page. What you have written is kept.

You are signed in: this note is linked to your account, visible only to you and the editor, so you can follow its status on your page. The text is never published. Full privacy policy ▸

Submit anonymously instead, not linked to your account ▸

No email is collected. Your original text and nickname are read only by the editor and are never published; the site publishes only a summary the editor writes. The human check is provided by Cloudflare Turnstile. To limit abuse, the system keeps a hash of your IP address, not the address itself. Your original text is stored with our cloud provider and can be deleted on request. Full privacy policy ▸