AI glossary for businesses
Retrieval-augmented generation (RAG)
In short
Retrieval-augmented generation (RAG) is a method in which an AI system first searches a chosen set of documents for relevant information and then uses that information to write its answer. It lets a language model answer from your own policies, catalogues or manuals instead of relying only on what it learned in training, and it can show its sources.
How RAG works
Think of an open-book exam. A student who must answer from memory may guess. A student allowed to look up the right page in the textbook gives a grounded answer and can point to the page. RAG gives an AI model that textbook.
It runs in two steps:
Retrieval: when someone asks a question, the system searches your documents for the most relevant passages. This search usually relies on embeddings, which match by meaning rather than exact words, so "can I send it back?" finds the returns policy.
Generation: the system passes those passages to the language model along with the question and an instruction such as "answer only from these texts".
The model itself does not change. You update the answers by updating the documents.
In practice for a small business
A small hotel group has house rules, booking terms, a breakfast menu with allergens and a list of local partners, spread across PDFs and Word files. Reception staff answer the same guest questions many times a day. With RAG, staff can ask an internal assistant "do we allow pets in the sea-view rooms?" and get a short answer with a link to the source document.
You may not need to build anything. ChatGPT custom GPTs and projects with uploaded files, Claude projects, Google's NotebookLM and Microsoft 365 Copilot over SharePoint all use retrieval in some form. A custom build makes sense later, for example when you need it inside your own website or CRM. See the AI knowledge-base assistant use case for a step-by-step approach.
Why it matters and what to watch out for
Garbage in, garbage out: if the documents are outdated or contradict each other, the answers will be too. Clean up and name one owner per document.
Fewer errors, not zero: RAG reduces hallucinations, but the model can still misread a passage. Test with 20–30 real questions before launch.
Access rights: an assistant should not show HR files to everyone. Check who can see which documents.
Personal data: avoid indexing files with customer or employee data unless needed. If you do it at scale, plan a DPIA and check with your DPO.