Members' shelf
All 9 nodes on Naming & Classification are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Members' shelf
All 16 nodes on System Protection & Threat Modeling are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Retrieval-augmented generation (RAG)
- Holds when
- When answers depend on a specific, changing body of facts that can be kept as a searchable text corpus.
- Evidence
- Conceptual framework; Abstracts checked
- Breaks down when
- Wrong retrieval still yields a fluent answer that makes the error hard to notice; for tasks needing no outside knowledge, retrieval adds little.
1. What happened
On one open-domain question-answering set the paper reports, a little over one answer in ten was still right even when none of the retrieved passages contained it. A purely parametric generative model seals knowledge inside its weights, and that brings three structural weaknesses: knowledge goes out of date (frozen at the training cut-off), it is untraceable (there is no way to point at what an answer rested on), and the model still generates with high confidence outside its distribution.
What this 2020 paper by Lewis and colleagues does is split memory into two layers: alongside parametric memory (the weights) it hangs a layer of non-parametric memory, an external corpus of text retrieved at the moment of generating[1]Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401. Bibliographic details checked against the official record. Claims matched against the full text. First version of the preprint 2020-05-22, twelve authors. The abstract describes the approach as combining parametric and non-parametric memory for language generation; it lists providing provenance for decisions as an open problem of purely parametric models, as the reason for adding an external memory. In the original design the retriever and the generator are trained together, but the document encoder and the index stay fixed, and only the query side and the generator are fine-tuned (the paper's §2.4).. The flow is retrieve, then answer: turn the question into a retrieval, fetch some passages, and have generation condition on them. In the original design the retriever and the generator are trained together: the document side, its encoder and its index, stays fixed, while the question encoder and the generator are fine-tuned together. So part of the retrieval step is learnable rather than merely bolted on.
Each of the three weaknesses gets a treatment: updating knowledge becomes swapping an index rather than retraining, the retrieved passages can be pulled out and inspected (the paper lists marking provenance as an open problem of purely parametric models, as the reason for adding an external memory; the original model does not output citations), and the distribution of what is generated is narrowed by outside evidence.
The ceiling needs stating clearly, though[4]Boundaries: that generation is mostly dragged down when retrieval goes wrong is an inference, not a conclusion of the paper; the paper's §4.1 also reports that on one open-domain question-answering set 11.8% of answers were still right when the answer was in none of the retrieved documents, with parametric memory filling in. That a citation being present is not a citation being used correctly is likewise an inference (the original model does not output citations). That gains are limited on tasks needing no outside knowledge comes from a preliminary observation in the paper's Appendix H, and the paper's account of why is conjecture.: if the retrieval layer is weak, generation will mostly be dragged down with it. Not inevitably wrong: the paper reports that on one open-domain question-answering set a little over one answer in ten was still right when the answer was in none of the retrieved passages, because the layer in the weights fills in. So the cap on quality moves from how much the model remembers to how good the corpus is and how accurate the retriever is. This does not abolish the knowledge problem. It externalises it, and the benefit of externalising is that the problem becomes governable (you can inspect it, update it, scope it), not that it goes away.
2. Why this matters to you
You are probably using this pattern already, in its weakest form.
Handing files to a model and having it read them before answering is the manual version of retrieval augmentation. It works, and how well it works depends entirely on what you handed over. This page asks you to move your attention from how to ask to what to give it, because the second is where the ceiling is.
Three things transfer directly.
First, build the layer you trust before worrying about phrasing. Most people put their effort into adjusting how they ask, and what phrasing improves is expression rather than grounds. If what you want answered depends on a particular body of facts, those facts have to exist first, be organised, and be known by you to be right.
Second, a citation being present is not the same as a citation being used correctly. That is the most usable sentence on this page. A model can attach a source that genuinely exists and in the same breath say something that source does not say. The checking action is not is there a source attached. It is does that source say this sentence.
Third, this approach adds little on tasks that need no external knowledge[4]Boundaries: that generation is mostly dragged down when retrieval goes wrong is an inference, not a conclusion of the paper; the paper's §4.1 also reports that on one open-domain question-answering set 11.8% of answers were still right when the answer was in none of the retrieved documents, with parametric memory filling in. That a citation being present is not a citation being used correctly is likewise an inference (the original model does not output citations). That gains are limited on tasks needing no outside knowledge comes from a preliminary observation in the paper's Appendix H, and the paper's account of why is conjecture.. Pure reasoning, rewriting for style, restructuring: attaching retrieval does not necessarily help, and it can drag in irrelevant material. The test is simple. When this task comes out wrong, is it because the model did not know something, or because it thought about it wrongly. Retrieval helps with the first and not the second.
Lab cases Written only where there is evidence
No field record for this node yet.
- A common account says that once retrieval is attached there will be no hallucination. If retrieval goes wrong, generation will mostly be dragged down with it, and it will answer fluently. The approach externalises the knowledge problem rather than eliminating it, and a good part of the bottleneck on quality moves to the retrieval corpus and the retriever. #
- A common account says that having a citation means having grounds. A citation being present is not the same as a citation being used correctly. An answer can attach a genuine source and in the same breath say something that source does not say. #
- A common account says retrieval-augmented generation began with this paper. It is generally taken as the source of the term, but one of its precursors is a February 2020 paper that put retrieval augmentation at the pre-training stage, three months earlier; this paper's own introduction lists that work alongside a 2019 retrieval-based question-answering model. #
Records how far each claim was checked, not a guarantee that it is true.
Corrections- This site marks the conference acceptance of this paper as secondary: the information comes from a note the preprint page makes about itself, not from the conference's official record. The conference acceptance of one of the precursors (the February 2020 paper by Guu and colleagues) is not displayed on its preprint page, and this site does not claim it.
- 2026-10-12 · An earlier version of this page went beyond the paper in three places, and all three have been corrected against the full text. First, it said that when retrieval goes wrong generation answers along with the error; the paper reports in §4.1 that 11.8% of answers were still right when the answer was in none of the retrieved documents, so this now reads mostly dragged down. Second, it said an answer can carry its sources; the original model does not output citations, and the paper says the retrieved knowledge can be inspected, so this now reads that retrieved passages can be inspected. Third, it said the retriever and the generator are fine-tuned together; in the original design the document encoder and the index stay fixed, and only the question side and the generator are fine-tuned. Separately, adds little and can subtract on tasks that need no outside knowledge now cites a preliminary observation in the paper's appendix, and can subtract has been removed.
How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.
- [1] Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401.
- [4] Editorial note, no external source; see note [4] in the text.
Tell me when a new shelf opens
Tell me when a new shelf opens
Shelf notices are sent only when a new shelf opens or when the reading rules change.
Tell the editor privately: a correction, a counter-example or where it applies
Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.
We have your note