Build your first RAG assistant in an afternoon
Retrieval-augmented generation sounds complicated. It is mostly search plus a good prompt. Here is the shape of it, step by step.

A large language model knows a lot about the world, but it knows nothing about your documents: your company handbook, your course notes, your product manual. Ask it about them and it will either refuse or, worse, invent a confident answer.
Retrieval-augmented generation, or RAG, fixes that with a simple idea: before asking the model, find the passages that are most likely to contain the answer, and hand them over together with the question. The model's job becomes reading and summarising, which it is very good at.
Step 1: split and embed your documents
Long documents are split into small chunks, a few paragraphs each. Every chunk is turned into an embedding: a list of numbers that captures its meaning, so that chunks about similar things end up close together.
chunks = split_into_chunks(handbook, size=800, overlap=100)vectors = [embed(chunk) for chunk in chunks]index.add(vectors, chunks)Step 2: retrieve the closest chunks
When a question arrives, it is embedded in the same way, and the index returns the chunks whose embeddings are closest to it. Three to five chunks are usually enough.
question = "How many days of leave do new staff get?"hits = index.search(embed(question), k=4)Step 3: ask the model to answer from them
The prompt includes the retrieved chunks and a clear instruction: answer only from this context, and say so when the answer is not there. That last part matters more than anything else in the prompt.
Answer the question using only the context below.If the answer is not in the context, say "I don't know". Context:{chunks} Question: {question}Most of the quality comes from retrieval, not from the model.
Measuring whether it works
A demo that answers three questions well proves very little. In the AI path, trainees write twenty real questions with known answers, run them through their assistant and count how many it gets right, how many it gets wrong and how many it correctly refuses.
- Wrong answers usually mean retrieval found the wrong chunks: try smaller chunks or better splitting.
- Refusals on answerable questions usually mean too few chunks were retrieved.
- Invented answers mean the prompt is not strict enough.
Where to go next
Once the basic loop works, there is plenty to improve: hybrid search that mixes keywords with embeddings, re-ranking the retrieved chunks, or citing the source of each answer. But the core stays the same three steps, and you can build them in an afternoon.






