Request a call

Retrieval-augmented generation (RAG)

Definition

Retrieval-augmented generation is a method where a system first searches your own documents for relevant passages, then gives them to a language model to answer from.

Updated

Also called grounded generation.

How it works

The term comes from a 2020 paper by Lewis and colleagues at Facebook AI Research. A RAG system has 2 halves. Ahead of time, your documents are split into passages (chunking), turned into embeddings and stored in a search index. At question time, the system finds the passages most relevant to the question, places them in the prompt and asks the model to answer using only that material, with a citation for each claim.

The model supplies the language. Your documents supply the facts. When a policy changes, you update the document and the answers change the same day, with no retraining.

How businesses use it

RAG is the standard pattern for any assistant that must answer from company knowledge:

  • A support assistant that answers from help articles and resolved tickets
  • An internal policy assistant for HR, IT and finance questions
  • Sales enablement over product specs and pricing rules
  • Contract and proposal review against your own templates

Our RAG knowledge systems builds cover document ingestion, permissions, search tuning, answer testing and the interface people use. The support knowledge assistant blueprint shows one in detail, and our guide to building a RAG knowledge assistant walks through the decisions in order.

Where it goes wrong

Most RAG failures are search failures. If the right passage never reaches the model, the answer is wrong whichever model you use. Typical causes are scanned PDFs that were never read properly, chunks that cut a table in half, and pure vector search missing exact part numbers (hybrid search fixes that one).

Permissions are the next failure. An assistant that indexes every SharePoint folder will happily quote a salary spreadsheet to anyone who asks, so source permissions have to be checked at query time.

The last is measurement. Keep a set of 50 to 100 real questions with known answers and score every change against it. Without that set, nobody can tell whether last week's tweak helped.

Related terms

  • Embeddings

    Embeddings are lists of numbers that capture the meaning of text or images, so software can measure how closely related one item is to another.

  • Vector database

    A vector database stores embeddings, the numeric fingerprints of text or images, and quickly finds the stored items whose meaning is closest to a query.

  • Chunking

    Chunking is splitting documents into smaller passages before they are embedded and indexed, so a retrieval system can find and return the exact part that answers a question.

  • Hybrid search

    Hybrid search runs keyword search and semantic search on the same query and merges the results, so it catches exact terms and paraphrased meaning together.

  • AI hallucination

    An AI hallucination is output from a language model that is fluent and confident but false, such as an invented citation, policy, figure or product detail.

All 30 definitions are in the AI and search glossary.

Find out where this fits in your own systems

Bring one process or page to a 30-minute call.

Request a 30-minute scoping call

We reply within 1 working day and suggest times that suit you.