Trigpoint
Collections › Software & Systems Architecture › Retrieval-augmented generation (RAG)
On one question set, with the answer in no retrieved passage, just over one in ten was still answered right.

Retrieval-augmented generation (RAG)

Retrieval-Augmented Generation (RAG) · Lewis et al. (2020)
Abstracts checked ✓ Machine learning · external evidence Read first: Single source of truth (concept history) Tell the editor privately ▸
In three sentences Split the model's memory in two: what is pressed into the weights, and what is fetched at the moment of generating. Knowledge can then be updated and the retrieved evidence can be inspected, and the surface available to hallucination narrows. But the ceiling on quality moves along with it, from how much the model remembers to how good your retrieval corpus is.

↳ Seen a counter-example?

Holds when
When answers depend on a specific, changing body of facts that can be kept as a searchable text corpus.
Evidence
Conceptual framework; Abstracts checked
Breaks down when
Wrong retrieval still yields a fluent answer that makes the error hard to notice; for tasks needing no outside knowledge, retrieval adds little.
Retrieval-augmented generation (RAG) Lewis et al. (2020) The ceiling moves from what it remembers to what you give it Parametric memory (weights) frozen at the training cut-off cannot point at what it used Non-parametric memory swap the index to update what it used can be inspected Retrieve first fetch some passages the query side is learnable Then answer generate conditioned on those passages still there, no longer the only basis Figure: trigpoint.pub
Figure 1 · Memory split into two layers: the layer pressed into the weights, which cannot be updated and cannot point at what it relied on, and the layer hanging outside, which can be swapped and can be inspected. The layer in the weights is still there, but it is no longer the only basis for an answer. Generation retrieves first and answers second. The flow is a conceptual sketch and carries no effect figures. View full size ▸

1. What happened

On one open-domain question-answering set the paper reports, a little over one answer in ten was still right even when none of the retrieved passages contained it. A purely parametric generative model seals knowledge inside its weights, and that brings three structural weaknesses: knowledge goes out of date (frozen at the training cut-off), it is untraceable (there is no way to point at what an answer rested on), and the model still generates with high confidence outside its distribution.

What this 2020 paper by Lewis and colleagues does is split memory into two layers: alongside parametric memory (the weights) it hangs a layer of non-parametric memory, an external corpus of text retrieved at the moment of generating[1]Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401. Bibliographic details checked against the official record. Claims matched against the full text. First version of the preprint 2020-05-22, twelve authors. The abstract describes the approach as combining parametric and non-parametric memory for language generation; it lists providing provenance for decisions as an open problem of purely parametric models, as the reason for adding an external memory. In the original design the retriever and the generator are trained together, but the document encoder and the index stay fixed, and only the query side and the generator are fine-tuned (the paper's §2.4).. The flow is retrieve, then answer: turn the question into a retrieval, fetch some passages, and have generation condition on them. In the original design the retriever and the generator are trained together: the document side, its encoder and its index, stays fixed, while the question encoder and the generator are fine-tuned together. So part of the retrieval step is learnable rather than merely bolted on.

Each of the three weaknesses gets a treatment: updating knowledge becomes swapping an index rather than retraining, the retrieved passages can be pulled out and inspected (the paper lists marking provenance as an open problem of purely parametric models, as the reason for adding an external memory; the original model does not output citations), and the distribution of what is generated is narrowed by outside evidence.

The ceiling needs stating clearly, though[4]Boundaries: that generation is mostly dragged down when retrieval goes wrong is an inference, not a conclusion of the paper; the paper's §4.1 also reports that on one open-domain question-answering set 11.8% of answers were still right when the answer was in none of the retrieved documents, with parametric memory filling in. That a citation being present is not a citation being used correctly is likewise an inference (the original model does not output citations). That gains are limited on tasks needing no outside knowledge comes from a preliminary observation in the paper's Appendix H, and the paper's account of why is conjecture.: if the retrieval layer is weak, generation will mostly be dragged down with it. Not inevitably wrong: the paper reports that on one open-domain question-answering set a little over one answer in ten was still right when the answer was in none of the retrieved passages, because the layer in the weights fills in. So the cap on quality moves from how much the model remembers to how good the corpus is and how accurate the retriever is. This does not abolish the knowledge problem. It externalises it, and the benefit of externalising is that the problem becomes governable (you can inspect it, update it, scope it), not that it goes away.

Attribution where due: The architecture belongs to Lewis and eleven co-authors, in the preprint first posted on 22 May 2020, which is also generally taken as the source of the term[1]Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401. Bibliographic details checked against the official record. Claims matched against the full text. First version of the preprint 2020-05-22, twelve authors. The abstract describes the approach as combining parametric and non-parametric memory for language generation; it lists providing provenance for decisions as an open problem of purely parametric models, as the reason for adding an external memory. In the original design the retriever and the generator are trained together, but the document encoder and the index stay fixed, and only the query side and the generator are fine-tuned (the paper's §2.4)..

2. Why this matters to you

You are probably using this pattern already, in its weakest form.

Handing files to a model and having it read them before answering is the manual version of retrieval augmentation. It works, and how well it works depends entirely on what you handed over. This page asks you to move your attention from how to ask to what to give it, because the second is where the ceiling is.

Three things transfer directly.

First, build the layer you trust before worrying about phrasing. Most people put their effort into adjusting how they ask, and what phrasing improves is expression rather than grounds. If what you want answered depends on a particular body of facts, those facts have to exist first, be organised, and be known by you to be right.

Second, a citation being present is not the same as a citation being used correctly. That is the most usable sentence on this page. A model can attach a source that genuinely exists and in the same breath say something that source does not say. The checking action is not is there a source attached. It is does that source say this sentence.

Third, this approach adds little on tasks that need no external knowledge[4]Boundaries: that generation is mostly dragged down when retrieval goes wrong is an inference, not a conclusion of the paper; the paper's §4.1 also reports that on one open-domain question-answering set 11.8% of answers were still right when the answer was in none of the retrieved documents, with parametric memory filling in. That a citation being present is not a citation being used correctly is likewise an inference (the original model does not output citations). That gains are limited on tasks needing no outside knowledge comes from a preliminary observation in the paper's Appendix H, and the paper's account of why is conjecture.. Pure reasoning, rewriting for style, restructuring: attaching retrieval does not necessarily help, and it can drag in irrelevant material. The test is simple. When this task comes out wrong, is it because the model did not know something, or because it thought about it wrongly. Retrieval helps with the first and not the second.

Verification layer · Members
The guided layer of this node is open to all readers.
Verification details and failure boundaries are for members.
US$8.25 a month on average · or subscribe monthly ▸

Lab cases Written only where there is evidence

No field record for this node yet.

Common misreadings

Page verification: Abstracts checked ✓ · Page version v2 · Content last revised 2026-11-14 · Corrections ↓

  • A common account says that once retrieval is attached there will be no hallucination. If retrieval goes wrong, generation will mostly be dragged down with it, and it will answer fluently. The approach externalises the knowledge problem rather than eliminating it, and a good part of the bottleneck on quality moves to the retrieval corpus and the retriever. #
  • A common account says that having a citation means having grounds. A citation being present is not the same as a citation being used correctly. An answer can attach a genuine source and in the same breath say something that source does not say. #
  • A common account says retrieval-augmented generation began with this paper. It is generally taken as the source of the term, but one of its precursors is a February 2020 paper that put retrieval augmentation at the pre-training stage, three months earlier; this paper's own introduction lists that work alongside a 2019 retrieval-based question-answering model. #
Verification record

Records how far each claim was checked, not a guarantee that it is true.

Corrections
  • This site marks the conference acceptance of this paper as secondary: the information comes from a note the preprint page makes about itself, not from the conference's official record. The conference acceptance of one of the precursors (the February 2020 paper by Guu and colleagues) is not displayed on its preprint page, and this site does not claim it.
  • 2026-10-12 · An earlier version of this page went beyond the paper in three places, and all three have been corrected against the full text. First, it said that when retrieval goes wrong generation answers along with the error; the paper reports in §4.1 that 11.8% of answers were still right when the answer was in none of the retrieved documents, so this now reads mostly dragged down. Second, it said an answer can carry its sources; the original model does not output citations, and the paper says the retrieved knowledge can be inspected, so this now reads that retrieved passages can be inspected. Third, it said the retriever and the generator are fine-tuned together; in the original design the document encoder and the index stay fixed, and only the question side and the generator are fine-tuned. Separately, adds little and can subtract on tasks that need no outside knowledge now cites a preliminary observation in the paper's appendix, and can subtract has been removed.
References

How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.

  1. [1] Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401.
  2. [4] Editorial note, no external source; see note [4] in the text.

↳ Is a citation wrong?

Tell me when a new shelf opens

Tell me when a new shelf opens

Shelf notices are sent only when a new shelf opens or when the reading rules change.

Trial

Tell the editor privately: a correction, a counter-example or where it applies

Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.

What are you submitting?
Set a public name to be credited by name ▸
Submit anonymously, without signing in

Signing in lets you follow the status on your page. What you have written is kept.

You are signed in: this note is linked to your account, visible only to you and the editor, so you can follow its status on your page. The text is never published. Full privacy policy ▸

Submit anonymously instead, not linked to your account ▸

No email is collected. Your original text and nickname are read only by the editor and are never published; the site publishes only a summary the editor writes. The human check is provided by Cloudflare Turnstile. To limit abuse, the system keeps a hash of your IP address, not the address itself. Your original text is stored with our cloud provider and can be deleted on request. Full privacy policy ▸