What is RAG (retrieval-augmented generation)?

What RAG is

A language model knows what was in its training data. It doesn't know your refund policy, last week's price change or the name of your support lead. RAG closes that gap without retraining the model. Your documents are split into short pieces (chunks) and indexed. When someone asks a question, a retriever finds the pieces that match, and those pieces are put into the model's context next to the question. The model then answers from them.

The name comes from Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, submitted by Patrick Lewis and co-authors in May 2020. Their system combined what the model had learned with passages fetched from a Wikipedia index. Today the same pattern sits behind most "chat with your documents" features: a Claude Project with files, a ChatGPT project with uploads, a help-center bot, or a company search assistant.

Two details decide whether it works. First, chunking: Microsoft's Azure AI Search guide explains that documents are split into chunks because models and embeddings have input limits, and the split changes what can be found. Second, context: Anthropic's write-up on contextual retrieval points out that a chunk such as "revenue grew 3% over the previous quarter" loses the company and the quarter it was about once it's cut out of its document.

Why a small team should care

• Answers from your facts. An assistant that reads your price sheet quotes this month's price, not a guess.
• Answers you can check. RAG answers can point to the passage they used, so a teammate can verify before sending.
• No engineering needed to start. Uploading files to a Claude or ChatGPT project already gives you the pattern. The work is choosing and cleaning the files.
• The common failure is boring. Two versions of the same policy, one from 2024. The AI finds both and blends them. No model fixes that; tidying the folder does.

A worked example

Illustration: a made-up business. Corner Desk is a six-person company that rents standing desks to offices. Its support inbox gets the same twenty questions about delivery, damage and returns. The founder uploads the whole shared drive to a project and asks, "What do we charge for a late return?" The assistant answers "£15 a week", from a 2024 price sheet. The current fee is £10 a day, in a doc called "Returns v3 FINAL".

The fix is in the sources, not the prompt. The team lists the 40 files, names one owner per topic, deletes the stale copies, gives each file a date and a title that says what it covers, and splits the returns doc so each fee sits under a heading that names it. Then it writes ten test questions with the right answers. On the second try, nine of ten are right and the tenth shows a gap: no document covered damage during delivery. Someone writes that paragraph.

How to start this week

1. Pick one job, such as answering delivery questions, not "everything we know".
2. List the documents for it with an owner and a last-updated date each. Remove duplicates and old versions.
3. Make each section stand on its own: a heading that names the topic, the product and the date inside the text.
4. Write 10 to 20 test questions with the answer and the document it should come from.
5. Upload, ask the test questions, and fix the documents behind each wrong answer before adding more files.

What RAG isn't

• Not training. The model doesn't change; it reads your passages each time. Remove a document and the answers change at once.
• Not a fix for questions about the whole collection. Microsoft Research found that plain RAG struggles with questions such as "what are the main themes across all of these?", because no single passage holds the answer. That is what GraphRAG was built for.
• Not the same as instructions. RAG supplies facts. How the assistant should behave (tone, what to refuse) belongs in its instructions; see context engineering.

Questions

What does RAG stand for?

Retrieval-augmented generation. The AI first retrieves passages from a set of documents, then generates its answer using them. The term comes from a 2020 paper by Patrick Lewis and co-authors.

Is a Claude Project or a ChatGPT file upload RAG?

It works on the same idea: the AI searches files you gave it and answers from them. Whether the product uses a search index or reads the whole file depends on the product and the file size; the advice on preparing documents is the same.

Does RAG stop an AI from making things up?

It reduces it when the right passage is found, and it lets the answer cite its source. It does not help when the documents are out of date, contradict each other, or don't contain the answer, so preparing the documents matters more than the tool.