A model has never read your firm's handbook. Ask it anyway and it will guess.
Example Advisors is a made-up wealth management firm. Its handbook, on the left, holds the real answers to clients' questions: what wires cost, when tax forms arrive, how to roll over a 401(k).
No language model was trained on this document. Asked "how much is an international wire?", a model would produce a plausible-sounding number from what other firms charge, and state it just as confidently as a fact it actually knows. That is a hallucination, and in finance or law it is a real liability.
The fix
Don't rely on memory: look it up. Search the document for the passages that answer the question, paste them into the prompt, and tell the model to answer only from them. That is RAG. The question bar at the top drives every step below.
First, cut the documents into small passages that can be searched and quoted.
Real document collections run to thousands of pages, far too much to paste into a prompt. So they are cut into chunks of a few dozen to a few hundred words. Each chunk is indexed for search, and only the best few are used for any question.
Neighbouring chunks usually overlap a little, so a sentence near a boundary still appears whole in at least one chunk.
Getting the size right
Drag the chunk size and watch the search results in the next step change. Too small and an answer is split across pieces, losing its context; too large and every chunk mentions everything, so search gets less precise and prompts get long and expensive.
Score every chunk against the question and keep the top three.
Two kinds of search work together:
Keyword search (BM25, the classic search-engine formula) rewards chunks that share rare words with the question. Precise for names, numbers and jargon, but blind to synonyms.
Meaning search compares embeddings, as in module 05. "Send money abroad" finds the chunk about "international wires" even though they share no words.
Most production systems use both, a hybrid, often followed by a second model that re-ranks the top results. Switch methods and compare: for "How do I move my old 401k?" keywords win, because "401k" is a rare, exact term. Then try "Do you invest in bitcoin?". The handbook does answer it (it says "cryptocurrency"), but the word bitcoin never appears, and the embedding model on this page, from 2014, has never heard of it, so both methods miss. A modern embedding model would connect the two.
Paste the passages into the prompt and tell the model to answer only from them.
The retrieved chunks are numbered and placed in the prompt, followed by the question and an instruction: answer only from these sources, cite them, and say so if the answer is not there. The language model then writes the reply, grounded in your document instead of its memory.
This page has no large language model, so instead it highlights the single sentence from the retrieved chunks that best matches the question, with its source. A real system would have an LLM write a fluent answer from the same passages, which also lets it combine facts, for example working out the fee on a $2 million account from the fee tiers.
Why citations matter
Because every answer points back to a numbered passage, a person can check it. For regulated work like financial advice, that audit trail is often the whole point.
The hardest part is noticing when the answer is not in the documents.
Search always returns something: the least-bad chunks. Ask "Who won the World Series?" and the top result is still some passage from the handbook. If the model is handed that and asked to answer, it may invent a connection.
So real systems set a confidence bar: if the best match scores below it, reply "I could not find that in the handbook" instead of answering. Set it too high and real questions get turned away; too low and off-topic questions get confident nonsense. Try the preset questions at different settings.
Notice the bitcoin question: at the default bar it gets through, attached to a passage about opening accounts that has nothing to do with crypto. A model handed that passage could produce a wrong answer with a real-looking citation. Catching cases like this is what RAG evaluation is for.
Other things that go wrong
Out-of-date documents give out-of-date answers. Two sources can contradict each other. Chunks can lose context ("it costs $40", but what does?). And a question can need facts spread across many passages. Most of the work in a real RAG system is in data quality, chunking and evaluation, not in the model.
Try it on real text: the retrieval workbench runs this on any document you paste or upload, compares keyword, meaning and hybrid search, and scores each setup against questions with known answers.
Next in the course: how agents work, where the model decides for itself when to search.