Ask a large language model about your own code base, your company's wiki or last week's release notes and it will do one of two things: say it doesn't know, or answer confidently from whatever it learned in training, which is usually out of date and sometimes invented. Retrieval-augmented generation, or RAG, is the standard fix. Before the model answers, you retrieve the few pieces of text that are relevant to the question and put them in the prompt. The model then answers from that text instead of from memory.
That is the whole idea. Everything else in this track is about doing each part of it well, and about proving that each change you make actually helps.
RAG in one picture
┌──────────── once, ahead of time ────────────┐
your documents ─► split into chunks ─► embed each chunk ─► vector store
└─────────────────────────────────────────────┘
question ─► embed ─► find the closest chunks ─► prompt = rules + chunks + question ─► LLM ─► answer
(retrieval) (generation)
Three ideas make it work:
- Chunks. You can't hand a model every document you own, so you cut them into pieces small enough that a handful fit in one prompt and focused enough that each one is about one thing.
- Embeddings. An embedding model turns a piece of text into a list of numbers (a vector) such that texts with similar meaning get vectors that point in similar directions. Embed every chunk once; embed each question when it arrives; the chunks whose vectors are closest to the question's are the ones most likely to answer it.
- Grounding. The prompt tells the model to answer only from the chunks and to say so when they don't contain the answer. Without that instruction you have a search engine bolted to a model that still answers from memory.
When you need it, and when you don't
RAG is not the only way to give a model your data, and it is not always the right one. Three alternatives are worth ruling out first.
Just paste everything in
Current Claude models accept a million tokens of input. If your documents fit, putting all of them in the prompt is simpler than any retrieval system and often more accurate, because the model sees everything and nothing can be missed by a bad search. Prompt caching makes repeating a large, fixed prefix cheap. For a few hundred pages, start here.
It stops working when the corpus is bigger than the context window, when it changes faster than you want to re-send it, or when you are paying for a million input tokens on every question to answer from three paragraphs of them. The corpus in this track, the tutorials on this site, is roughly two million tokens, so it doesn't fit, and even if it did, a question about Python decorators doesn't need the SQL track in the prompt.
Plain search
If people want a list of matching documents rather than an answer, a search box is cheaper, faster and easier to trust. RAG earns its cost when the reader wants the answer itself: "how do I keep a long-running command alive after I disconnect from SSH?" is better answered with the three commands than with a link to a post that contains them somewhere.
Fine-tuning
Training a model on your documents teaches it style and vocabulary, but it is a poor way to teach facts: you can't see which document an answer came from, you can't remove a fact without retraining, and it goes stale the day you publish something new. RAG keeps knowledge in a store you can inspect, cite and update. If the problem is "the model doesn't know our content", RAG is almost always the first thing to try.
What we will build
Ask LoveMeSomeCoding: a question-answering assistant over every tutorial on this site. You ask "how do I validate the request body in a NestJS controller?" and it answers from the NestJS posts, quotes the exact passages it used, and links them. Ask it something the site doesn't cover, like training a GAN in PyTorch, and it says so in one sentence instead of improvising.
By the end it can also answer questions that aren't about what a post says, such as "what are the three most recent Spring Boot tutorials?", by choosing to query the list of posts rather than search their text.
The corpus is real, which matters more than it sounds. This site has 989 posts written over sixteen years: modern tracks with clean structure, older WordPress posts wrapped in layout markup, stub posts with one line of text, a post with a NUL byte in it, and two posts that contradict each other. Every one of those turned into a lesson. A tutorial built on a tidy toy dataset never meets them.
The stack
Everything runs on your machine except the call to Claude:
Piece Choice Why
--------------- --------------------------------------- ------------------------------------------
Language Python 3.12 the ecosystem for embeddings and evals
Vector store Postgres + pgvector, in Docker one container; SQL for everything else
Keyword search Postgres full-text search no second engine to run
Embeddings bge-small / bge-base (sentence-transf.) local, free, offline
Reranker bge-reranker-base (cross-encoder) local, tried in lesson 7
LLM Claude (claude-opus-5-5) via the API answers, judging, an agent with tools
Test UI React + FastAPI try every setting in a browser
The whole project is in the
ai_repo_lovemesomecoding.com
repository: one script per lesson in lessons/, the reusable pieces in
ask_lmsc/, the evaluation data in eval/, and the browser console in
api/ and web/. Each lesson's code is quoted from that repository, and a
check in this site's build compares the quotes with the files so they can't drift apart.
Cost: the embeddings and the database cost nothing. Claude calls cost about two cents per answered question. Building and evaluating everything in this track, including the expensive experiments, cost about seventeen dollars in API credit. Each lesson that spends money says how much before it does.
The rule this track follows: measure every change
RAG systems are easy to build and easy to fool yourself about. You change the chunk size, try three questions, the answers look a bit better, and you move on. Three questions prove nothing, and "looks better" usually means "looks different".
So from the fifth lesson on, every change is scored against a golden set: questions written by hand, each with the posts that answer it, plus questions the site deliberately doesn't cover. Retrieval gets a number (how often the right post is in the top five), answers get a number (how many claims are supported by the sources), and a change is kept only if its number goes up.
That rule changed the outcome more than once. Hybrid search, which every RAG guide recommends, made our best setup worse. A reranker looked like a regression until we found the golden set was missing correct answers. Contextual retrieval, a well-known technique, cost two dollars and did nothing measurable. A bigger embedding model, which we had decided against, turned out to be the single best change once we measured it properly. Those reversals are in the lessons, with the numbers, because they are the most useful part.
The lessons
1 RAG – what it is and when you need it (this post)
2 Project setup and getting the data Docker, pgvector, exporting 989 posts
3 A naive pipeline in 100 lines the baseline everything improves on
4 Chunking by structure headings, whole code blocks, titles
5 pgvector tables, HNSW, a bigger model's cost
6 Measuring retrieval with a golden set hit@1, hit@5, MRR
7 Hybrid search, reranking and pooled judging what helped, what didn't
8 Grounded answers and citations "not covered" without padding
9 Search small, answer big neighbouring chunks
10 Contextual retrieval measured, and not adopted
11 Evaluating answers groundedness and correctness
12 Keeping the index current incremental re-indexing
13 An agent that chooses between search and SQL the capstone
You will need Python 3.12, Docker, about 2 GB of disk for models, and an Anthropic API key from lesson 3 on. No machine-learning background is assumed: where a concept matters (embeddings, cosine similarity, cross-encoders), the lesson that first needs it explains it.
Before you start
Two habits will save you time in every lesson:
- Look at what retrieval returned before you read the answer. Most bad answers
are bad retrieval. Every script in the repository has a
--retrieve-onlyflag that prints the chunks without calling Claude, and it's free. - Read your failures one by one. A score tells you that something changed; only the individual misses tell you why. Several of the biggest fixes in this track came from reading five misses, not from a metric.