Members' shelf
All 9 nodes on Naming & Classification are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Members' shelf
All 16 nodes on System Protection & Threat Modeling are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Hallucination in NLG
- Holds when
- When checking model output and you have a clear source to compare it against.
- Evidence
- Conceptual framework; Abstracts checked
- Breaks down when
- In open-ended generation with no clear source, the two types apply only by extrapolation; faithful to the source is not necessarily factual.
1. What happened
The definition adopted by the 2023 survey by Ji and colleagues in ACM Computing Surveys is generated content that is nonsensical or unfaithful to the provided source content[1]Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730. Bibliographic details checked against the official record. Claims matched against the full text. The journal version has ten authors; the preprint arXiv:2202.03629 was first posted 2022-02-08, and later versions added authors, thirteen by the July 2024 version. The definition the survey adopts in §2 is taken from earlier studies: generated content that is nonsensical or unfaithful to the provided source content.. The classification has two types: intrinsic, which contradicts the source, and extrinsic, which cannot be verified from the source[2]The definitions of the classification (the paper's §2.1): intrinsic is output that contradicts the source content; extrinsic is output that cannot be verified from the source content..
The mechanism is worth going through slowly, because it decides what you can do about it. A generative model is fundamentally a compression of its training distribution, sampling one token at a time at decoding. When a task needs information beyond what the model has reliably compressed (out of distribution, in the long tail, or the world changed after training), the model tends not to stop and say I do not know. It carries on sampling fluently. That is hallucination. Part of that tendency is rewarded into it: a 2025 study argues that current training and grading give credit for a guess and none for admitting uncertainty, so models lean towards guessing when unsure[6]Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. Bibliographic details checked against the official record. Consistent with the abstract. The gist of the abstract: models sometimes guess when uncertain instead of admitting uncertainty, and the authors argue the cause is that training and evaluation procedures reward guessing over acknowledging uncertainty.. It is not that a model cannot say it does not know; under the current rewards it seldom does.
The survey by Ji and colleagues sorts the causes into two broad groups: data, and training and inference. This page splits the second into training and decoding, giving three layers, which is why there is no single point of repair: the data layer (mainly source and reference that already disagree; duplicates left unfiltered during data collection are another case), the training layer (poor representation learning, exposure bias between training and generation, and a model leaning on knowledge from pre-training rather than the source in hand), and the decoding layer (diversity-oriented sampling correlates with hallucination, though the survey marks the mechanism as conjecture). The mitigations can be divided the same three ways: change the supply of evidence (retrieval augmentation, see Retrieval-Augmented Generation), change the objective (train against a measure of faithfulness), and change decoding (constrain generation).
2. Why this matters to you
What this page gives you is not a warning. It is a calibration: how much to discount fluency by.
When people judge credibility, fluency of language and tidiness of structure are strong cues. Those cues work reasonably well on things written by people, because people talking nonsense usually also talk messily. A generative model severs that correlation: its fluency has almost nothing to do with whether the content is true. So your intuition here is not unreliable. It is calibrated on the wrong distribution.
The second thing is the practical consequence of it seldom saying I do not know. You cannot rely on the model raising a flag to catch hallucination: when unsure it tends to guess, and a wrong guess reads just as fluently. That puts the burden of checking on your side, and it has to be a process rather than vigilance: vigilance fatigues, a process does not.
The third thing is where this shelf takes its position: since no layer can guarantee it has reached zero, and the model will not reliably raise a flag, trustworthy output cannot rest on a guarantee about model quality. It can only rest on checkability.
In practice two moves are enough. First, turn where is the basis for this sentence into a fixed question, rather than picking sentences to check by feel. Second, handle the two types separately: what contradicts the source you gave (intrinsic) is caught by comparison; what the source never mentions (extrinsic) is caught by checking existence. The two need different methods, and blending them leaks on both sides. So before you start checking, ask one question first: do I have a source in hand to compare against.
Lab cases Written only where there is evidence
No field record for this node yet.
- A common account says the intrinsic and extrinsic classification was proposed by the survey by Ji and colleagues. The paper's own §2.1 states that it follows the classification of earlier studies. This site treats it as the anchor that organised the whole field; it is not the originator of the classification. The definition of hallucination the survey adopts in §2 is likewise taken from earlier studies. #
- A common account says newer models no longer hallucinate. What the survey by Ji and colleagues lists is causes and mitigations by type, and no layer claims it can reach zero. This site therefore treats it as a standing risk that can only be narrowed and must not be assumed to be at zero; that is this site's design premise, not a conclusion of the survey. The point is not pessimism. It is the premise for deciding whether to build a checking process at all. #
- A common account says the term only arrived in the era of large language models. An earlier attested usage appears, for example, in a 2018 paper on a vision-language task; the survey itself records that the term first appeared in computer vision research in 2000, with a more positive meaning at the time. #
Records how far each claim was checked, not a guarantee that it is true.
Corrections- 2026-10-12 · An earlier version of this page went beyond the 2023 survey by Ji and colleagues in three places, and all three have been corrected against the full text. First, it presented hallucination arising from compression plus sampling, narrowable but not reducible to zero, as the source's conclusion; the survey lists causes and mitigations by type and does not claim hallucination cannot be removed. The shelf's position is kept, but its premise is now that no layer can guarantee zero and the model does not reliably raise a flag, and it is marked as this shelf's own position. Second, it gave the training-layer cause as a maximum-likelihood objective that rewards fluency over faithfulness; the survey lists poor representation learning, exposure bias and parametric knowledge bias, and names a mismatch between source and reference as the main data-layer cause. Third, it said faithfulness and factuality are not the same thing; the survey says they may or may not be the same.
How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Consistent with the abstract” means the claim agrees with the paper's abstract. It does not include sentence-by-sentence checking against the full text. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.
- [1] Ji, Z. et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730.
- [2] Ji, Z. et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), Article 248. DOI 10.1145/3571730.
- [6] Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. https://arxiv.org/abs/2509.04664 ↩
Tell me when a new shelf opens
Tell me when a new shelf opens
Shelf notices are sent only when a new shelf opens or when the reading rules change.
Tell the editor privately: a correction, a counter-example or where it applies
Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.
We have your note