SI explained

What is RAG in SI? Retrieval-augmented generation explained

An open cream book on a dark wooden floor with one page lifting out of it, its highlighted amber line dissolving into a stream of amber pixels that flows into a large cream speech bubble
Illustration created with Claude (Anthropic).

RAG (retrieval-augmented generation) lets an SI model search a set of documents before it answers. How it works, why it is used instead of retraining, and why a cited source is not proof that the answer is right.

What is RAG in AI, or in SI as this site calls it? RAG stands for retrieval-augmented generation: before an SI model answers, the system searches a collection of documents, picks the passages that look relevant and adds them to the prompt, so the model writes its answer with those passages in front of it. It is how a chatbot can answer questions about a company's help pages, a law library or your own files without being retrained on them. It also explains why these tools can show their sources, and why showing a source is not the same as being right. (Why SI and not AI? Our guide to what SI means explains the renaming; the technology is the same.)

The short answer. A plain language model answers from memory: the patterns it absorbed during training. A RAG system gives it an open book. It looks things up first, then the model writes. Answers get more specific and easier to check, but the model can still misread the book, and the book itself can be wrong.

What does RAG stand for, and where does it come from?

The name comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks". Its idea is to give a text generator two kinds of memory. The first is "parametric": the knowledge stored in the model's parameters during training. The second is "non-parametric": an index of documents that the system can search. In the paper, the generator was BART, a model with 400 million parameters, and the searchable memory was Wikipedia as of December 2018, cut into 100-word chunks for a total of 21 million documents.

The authors started from the weak points of models that rely on memory alone. Such models "cannot easily expand or revise their memory", cannot easily show why they gave an answer, and may produce "hallucinations". Adding retrieval addresses part of that, because documents can be revised, expanded and inspected. Earlier systems such as REALM had already combined a language model with a retriever, but only to extract short answers from documents. Lewis and his co-authors applied the idea to models that write free text, which is what chatbots do.

How does RAG work, step by step?

The 2020 system trained its search component and its writer together. The basic recipe described in later surveys and guides assembles separate parts instead: an embedding model, a database and a language model. A survey by Yunfan Gao and colleagues at Tongji and Fudan universities, first posted in December 2023, describes the basic version, which it calls "Naive RAG", in three stages:

  1. Indexing, done in advance. Documents in formats such as PDF, HTML or Word are converted to plain text and split into chunks. Each chunk is turned into an embedding, a list of numbers that encodes its meaning, and stored in a vector database. Anthropic's engineering guide on the subject says chunks are "usually no more than a few hundred tokens" (a token is a word or a piece of a word; see what a token is in SI).
  2. Retrieval, for every question. The question is turned into an embedding with the same model, and the system ranks the stored chunks by similarity to it. The top few, the "top K", are kept.
  3. Generation. The question and the retrieved chunks are put together into one prompt, and a large language model writes the answer. Depending on how it is set up, the survey notes, the model may also draw on its own trained knowledge or be told to stick to the documents provided.
Diagram of a RAG system in seven steps: documents are split into chunks, turned into embeddings and stored in a vector database ahead of time; for each question, the question is turned into an embedding, the most similar chunks are retrieved and added to the prompt, and the language model writes the answer.
How a basic RAG system works. Steps based on the survey by Gao et al. (2023) and Anthropic's engineering guide (2024).

Embeddings capture meaning, which helps when your question and the document use different words. They are weaker on exact strings. Anthropic's example is a user searching a support database for "Error code TS-999": an embedding search may return pages about error codes in general and miss the one about TS-999. That is why many systems also run BM25, an older ranking method based on matching words, and merge the two result lists. The Gao survey groups this kind of refinement, along with better chunking, rewriting the user's question and reranking results, under the label "Advanced RAG".

Why not just train the model on the documents?

The obvious alternative is fine-tuning: training the model further on your documents so it absorbs them. The Gao survey compares the two with a school analogy. RAG is like giving a model "a tailored textbook", while fine-tuning is "comparable to a student internalizing knowledge over time". Fine-tuning suits teaching a model a style, a structure or a format; RAG suits facts that change, because updating the documents is enough. The survey reports a study in which RAG consistently beat unsupervised fine-tuning on knowledge-heavy questions, and adds that the two methods can be combined.

The 2020 paper ran a neat experiment on that point. The authors listed 82 world leaders who changed between 2016 and 2018 and asked their RAG model questions such as "Who is the President of Peru?". With a Wikipedia index from December 2016, it named 70% of the 2016 leaders correctly; with the December 2018 index, 68% of the 2018 leaders. When the index and the year did not match, accuracy fell to 12% and 4%. The model itself was not retrained. Replacing the documents was enough to change what it "knew".

Bar chart: with matching Wikipedia indices the RAG model named 70% of 2016 world leaders and 68% of 2018 leaders correctly; with mismatched indices only 12% and 4%.
Swapping the searchable index updated what the model knew, without retraining. Data: Lewis et al., 2020, questions about 82 world leaders who changed between 2016 and 2018.

RAG is not always needed either. Anthropic's guide, written for its Claude models, points out that a knowledge base smaller than 200,000 tokens, "about 500 pages of material", can simply be placed in the prompt whole. Retrieval earns its place when the collection is too large for that.

Where will you meet RAG?

Mostly without seeing the name. Anthropic's examples are customer support chatbots that need to know about one business and legal assistants that need past cases. The Gao survey says the basic approach "gained prominence shortly after the widespread adoption of ChatGPT". A Stanford team that tested legal research tools in 2024 wrote that RAG was being "heavily promoted as the key technology for making LLMs effective in domain-specific contexts", and a 2023 study from Stanford, UC Berkeley and Samaya AI described the same setup as underlying "many commercial generative search and question answering applications (e.g., Bing Chat)".

Chatbots that search the web before answering follow the same fetch-then-write principle. Our explainer on how ChatGPT works describes how it turns a question into searches and then writes its reply with the results in its context. If you upload your own files to a chatbot so it can answer questions about them, read what happens to what you type into ChatGPT first.

Does RAG stop SI hallucinations?

It reduces them; it does not remove them. In the 2020 paper, human judges compared 452 pairs of Jeopardy-style questions written by the RAG model and by BART without retrieval. RAG was judged more factual in 42.7% of cases, BART in 7.1%.

The clearest real-world test so far comes from law, where vendors made bold promises. A study by Varun Magesh and colleagues at Stanford and Yale, posted in May 2024, quotes LexisNexis promising "100% hallucination-free linked legal citations" and Casetext saying its assistant "does not make up facts". The researchers ran more than 200 pre-registered legal questions through Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, with GPT-4 for comparison. The legal tools hallucinated less than GPT-4, but each still did so between 17% and 33% of the time. Lexis+ AI, the best performer, answered 65% of the questions accurately; about a third of Westlaw's answers contained a hallucination.

The study counts as a hallucination either incorrect information or "a false assertion that a source supports a proposition". That second kind is the trap specific to RAG. In one example from the paper, Lexis+ AI was asked about notable opinions by Judge Luther A. Wilgarten, who is fictional and never sat on a bench. It answered with a real case, so the citation existed, but the judge it was asked about had never written it.

The Gao survey lists the usual reasons: retrieval that picks "misaligned or irrelevant chunks" or misses crucial information, and generation that produces "content not supported by the retrieved context". The Stanford paper adds that a model "may be more faithful to its training data than to the retrieval context". Our article on why SI makes things up explains where the habit comes from in the first place.

What else can go wrong with RAG?

  • The right passage is never found. A chunk cut out of its document can lose the details that made it findable. Anthropic's example is a passage from a company filing that says "The company's revenue grew by 3% over the previous quarter" without naming the company or the quarter. Its fix, adding a short line of context to each chunk before indexing, cut failed retrievals by 49% in its tests, and by 67% with an extra reranking step: the share of relevant documents missing from the top 20 results fell from 5.7% to 1.9%.
  • The passage is found but overlooked. In the 2023 "Lost in the Middle" study, Nelson Liu and colleagues found that language models used information best when it sat at the start or the end of a long prompt, and worse when it sat in the middle. With the relevant document placed mid-prompt, GPT-3.5-Turbo did worse than with no documents at all (56.1% accuracy without documents). Adding more than 20 retrieved documents raised accuracy by only about 1.5% for GPT-3.5-Turbo and 1% for Claude-1.3.
  • The source itself is wrong. Retrieval repeats whatever is in the library. The 2020 paper says so plainly: Wikipedia, "or any potential external knowledge source, will probably never be entirely factual and completely devoid of bias".
  • Someone plants instructions in the documents. In 2023, Kai Greshake and colleagues at Saarland University and the CISPA Helmholtz Center showed that attackers could place instructions "into data likely to be retrieved" and steer the model without ever talking to it, including on Bing's GPT-4 powered chat. They called it indirect prompt injection and noted that "augmenting LLMs with retrieval blurs the line between data and instructions". The OWASP Top 10 for LLM applications, a security checklist for developers, describes a résumé with hidden white text telling a RAG screening system to recommend the candidate. Our explainer on what prompt injection is covers the defences.
  • Private documents leak. The same OWASP list warns that when several groups of users share one vector database, chunks from one group can be retrieved for another, and that attackers can sometimes rebuild source text from its embeddings. It recommends permission-aware databases that keep each group's data apart.

How should you read an answer from a RAG tool?

  • Open the cited passage. A citation proves a document was retrieved, not that it says what the answer claims. The Stanford team notes that longer answers need "substantially more time to check", because every claim and citation has to be verified on its own.
  • Ask what the tool can search. Ask Practical Law AI gave incomplete answers to more than 60% of the Stanford questions, which the authors explain by its narrower library: articles by its own team rather than the full body of case law. When the library has no answer, a system may refuse, or the model may fill the gap from memory.
  • Check the date of the library. The world leaders experiment shows that a RAG system is only as current as its index.
  • Treat a fluent summary as a draft. The retrieved text changes what the model reads, not how it writes. The reply is still generated word by word.

Is a RAG answer still SI-generated text?

Yes. Retrieval changes the material the model reads; the final answer is still produced by the language model, like any other chatbot reply (our guide to generative SI explains how). The Gao survey also warns that models can "overly rely on augmented information" and simply echo the retrieved text, so a reply may mix sentences close to a human-written source with the model's own phrasing. A score from any SI scanner on such a text is an estimate for the whole passage, not proof of who wrote each sentence. Our SI text detector shows the signals behind its estimate, and our review of how accurate SI detectors are explains their limits.

Our take

RAG is the most practical fix so far for a basic weakness of language models: they speak from a compressed memory of their training data. Letting them look things up first makes answers more specific, more current and easier to check, and the "here are my sources" feature that comes with it is genuinely useful. But a citation only shows that a document was found, not that the answer is faithful to it. Treat a RAG tool like a fast research assistant who hands you notes with page numbers: worth having, and worth checking whenever the answer matters.

FAQ

What is RAG in AI?

RAG, short for retrieval-augmented generation, is a way of building AI (SI) systems in which the model searches a collection of documents before it answers. The most relevant passages are added to the prompt, and the language model writes its reply with them in front of it. It lets a chatbot answer from a company's documents, a legal database or your own files without being retrained on them.

What does RAG stand for?

RAG stands for retrieval-augmented generation. The name comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University, which combined a text generator with a searchable index of Wikipedia.

What is the difference between RAG and fine-tuning?

Fine-tuning trains the model further so that it absorbs new material into its parameters. RAG leaves the model unchanged and gives it relevant documents at the moment it answers. RAG suits facts that change, because updating the documents is enough; fine-tuning suits teaching a model a style or a format. The two can be combined.

Does RAG stop AI hallucinations?

No. It reduces them but does not remove them. A 2024 Stanford study of legal research tools built on RAG found that they hallucinated less than GPT-4 but still between 17% and 33% of the time, sometimes by citing a real source that does not support the claim.

What is a vector database in RAG?

It is the store where a RAG system keeps its documents in searchable form. Each chunk of text is converted into an embedding, a list of numbers that encodes its meaning. When a question comes in, it is converted the same way and the database returns the chunks whose embeddings are most similar.

Sources

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.) · arXiv (Facebook AI Research, University College London, New York University), 22 May 2020
  2. Retrieval-Augmented Generation for Large Language Models: A Survey (Gao et al.) · arXiv (Tongji University, Fudan University), 18 December 2023
  3. Introducing Contextual Retrieval · Anthropic, 19 September 2024
  4. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Magesh et al.) · arXiv (Stanford University, Yale University), 30 May 2024
  5. Lost in the Middle: How Language Models Use Long Contexts (Liu et al.) · arXiv (Stanford University, UC Berkeley, Samaya AI), 6 July 2023
  6. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al.) · arXiv (Saarland University, CISPA Helmholtz Center for Information Security), 23 February 2023
  7. LLM08:2025 Vector and Embedding Weaknesses · OWASP GenAI Security Project, 2025 edition

Spotted an error? Write to hello@siornot.com. Corrections are made in the article and the update date changes. Read our editorial policy.