RAG Fundamentals: Retrieval-Augmented Generation Explained

RAG in one picture

What is RAG?

Retrieval-Augmented Generation gives a language model access to an external knowledge source at query time. Instead of relying only on what the model memorised during training, you:

1. Retrieve the most relevant pieces of your own documents, then
2. Generate an answer grounded in those pieces.

The model stays general; your knowledge stays in a database you control.

Why not just fine-tune?

Fine-tuning bakes knowledge into the weights. That is great for changing how a model behaves, but a poor fit for facts that change or are too large to memorise.

The RAG pipeline

Two phases: an offline indexing phase (documents → chunks → vectors) and an online query phase (question → retrieve → generate).

In code

# 1. Index: chunk documents and store embeddings
for doc in documents:
    for chunk in split(doc, size=500, overlap=50):
        db.upsert(id=chunk.id, vector=embed(chunk.text), text=chunk.text)

# 2. Query: retrieve relevant chunks, then generate
query_vec = embed(user_question)
chunks = db.search(query_vec, top_k=5)
context = "\n\n".join(c.text for c in chunks)

answer = claude.messages.create(
    model="claude-opus-4-8",
    messages=[{
        "role": "user",
        "content": f"Context:\n{context}\n\nQuestion: {user_question}",
    }],
)

The retrieved context is injected into the prompt — the model answers from your data, not its memory.

Check your understanding

Takeaways

• RAG = retrieve relevant context, then generate a grounded answer.
• Use it when knowledge is large, private, or fast-changing.
• The building blocks: a chunker, an embedding model, a vector DB (e.g. PostgreSQL + pgvector), and an LLM.
• Next lessons: chunking strategies, embedding choice, and evaluating retrieval.