The Core Idea: RAG systems did not appear fully formed. They grew in three steps. Naive RAG is the simple pipeline: index, retrieve, generate. Advanced RAG adds improvements before, during and after retrieval. Modular RAG turns the pipeline into interchangeable parts that can be rearranged, repeated or skipped. Knowing this path tells you what to build first and what to add only when a measured problem demands it.
1. Beginner: Naive RAG
Naive RAG is the pipeline from the first lessons, with no extras: split documents into chunks, embed and store them, embed the question, fetch the top-k chunks, put them in a prompt, and let the model answer.
Index chunks→Embed the question→Fetch top-k→Build prompt→Generate
Strengths: simple to build, easy to understand, and surprisingly useful for clean documents and direct questions. It is the right starting point and the baseline that every improvement must beat.
2. Intermediate: Where Naive RAG Fails
Each weakness of naive RAG maps to a technique you have met in this course.
| Symptom | Likely cause | Fix, and where you learned it |
|---|---|---|
| Nothing found for a clear question | Query wording differs from documents | Query transformation |
| Wrong or outdated answer | Old versions and unrelated chunks are retrieved | Metadata filtering and versioning |
| Right chunk retrieved but ranked low | Rough first-stage scoring | Reranking |
| Exact codes and names missed | Embedding-only search | Hybrid retrieval |
| Question needs several facts | Only one search is done | Multi-step retrieval |
| Confident answer with no support | No check on retrieval or output | Grounding and verification |
3. Intermediate: Advanced RAG
Advanced RAG keeps the same skeleton but upgrades each phase.
Before retrieval
Better chunking, rich metadata, query rewriting, expansion and decomposition.
During retrieval
Hybrid dense and sparse search, metadata filters, tuned top-k.
After retrieval
Reranking, deduplication, compression, careful ordering and grounded prompts.
The pipeline is still mostly a fixed straight line. It is stronger than naive RAG, but every question takes the same path, and every improvement is hard-wired into it.
4. Intermediate: Modular RAG
Modular RAG treats each capability as a module with a clear input and output: query rewriter, retriever, reranker, memory, router, verifier, and so on. An orchestrator decides which modules run, in what order, and how many times.
| Pattern | What happens | Example |
|---|---|---|
| Linear | Modules run in a fixed order | Normalize, retrieve, rerank, generate |
| Conditional | A router picks the path | Simple question skips retrieval; hard one uses multi-step |
| Looping | Steps repeat until a check passes | Retry retrieval if the answer is unsupported |
| Parallel | Several branches run and are merged | Search documents and a database together |
Why it matters: modules can be swapped, tested alone and removed. This turns system design into experiments: take one module out and measure what you lose.
5. Advanced: Choosing an Architecture
More machinery means more cost, delay and failure modes. A sound approach is to start simple and add parts only in response to measured failures.
1. Build naive RAG and an evaluation set of real questions.
2. Find the most common failure and add the one module that addresses it.
3. Re-measure. Keep the module only if quality improves enough to justify its cost.
4. Repeat, and move toward a modular design once different questions clearly need different paths.
| Situation | Reasonable choice |
|---|---|
| Small, clean knowledge base, direct questions | Naive RAG, perhaps with hybrid search |
| Mixed question types and measurable failures | Advanced RAG with targeted upgrades |
| Many sources, tools, or questions needing different paths | Modular RAG with routing and checks |
A warning: modularity is not a goal in itself. A system with ten modules and no evaluation is harder to trust than a simple one that you can measure.
6. Visual Explanation: Three Generations of RAG
Panel A places the three generations side by side. Panel B shows measured results from the code below. Panel C is a symptom explorer.
Panel A · Naive, Advanced, ModularEach generation adds control
Generation 1
Naive RAG
Index, retrieve, generate in one straight line.
Generation 2
Advanced RAG
Upgrades before, during and after retrieval.
Generation 3
Modular RAG
Swappable modules with routing and loops.
Panel B · Hit Rate on Four QuestionsFrom the code in section 7
Naive RAG
1 of 4 questions answered by the right document
Without normalize
2 of 4
Without active filter
3 of 4
Advanced and modular, both modules
4 of 4
Panel C · Symptom ExplorerClick a symptom to open it
Click to see why you should not build everything at once▼
Each module adds latency, cost and new ways to fail. Without a baseline and a test set you cannot tell which additions help, so grow the system one measured step at a time.
How to read it: Panel A is the family tree, Panel B shows that each module earns its place on test questions, and Panel C maps problems to additions.
7. Python Implementation: Naive vs Advanced vs Modular with Ablations
The program defines five tiny documents, one of which is an outdated vacation policy, and four test questions with known answers. Naive RAG searches once with the raw question. The advanced and modular pipelines are built from two modules: a normalizer that expands abbreviations such as PTO and WFH, and a filter that keeps only active documents. Because the modules are plain functions in a list, the program can remove one at a time and measure the result.
rag_evolution.pyPython 3.10+ · Standard library only
rewrites = dict([("pto", "paid time off"), ("wfh", "work from home"), ("2fa", "two factor authentication")])stop_words = set(["how", "many", "do", "to", "when", "what", "is", "a", "the"]) documents = [ ("d1", "active", "vacation policy 2025 employees receive 20 days of paid time off"), ("d2", "superseded", "vacation policy 2023 employees get 15 days of paid time off"), ("d3", "active", "work from home policy remote work two days per week"), ("d4", "active", "security policy enable two factor authentication"), ("d5", "active", "expense policy submit receipts within 30 days"),] tests = [ ("how many PTO days do employees get", "d1"), ("WFH rules", "d3"), ("2FA requirements", "d4"), ("when to submit receipts", "d5"),] def words(text): return "".join(ch if ch.isalnum() or ch == " " else " " for ch in text.lower()).split() def top1(query, docs): q = set(w for w in words(query) if w not in stop_words) scored = [(len(q & set(words(text))), doc_id) for doc_id, status, text in docs] scored.sort(key=lambda s: (-s[0], s[1])) return scored[0][1] if scored and scored[0][0] > 0 else None # Naive RAG: one search with the raw questiondef naive(query, docs): return top1(query, docs) # Modules: each takes (query, docs) and returns (query, docs)def normalize(query, docs): return " ".join(rewrites.get(w, w) for w in words(query)), docs def only_active(query, docs): return query, [d for d in docs if d[1] == "active"] def run(modules, query, docs): for module in modules: query, docs = module(query, docs) return top1(query, docs) def score(label, fn): hits = sum(1 for question, gold in tests if fn(question) == gold) print(label.ljust(34), "hit rate:", hits, "/", len(tests)) score("Naive RAG", lambda q: naive(q, documents))score("Advanced: normalize + filter", lambda q: run([normalize, only_active], q, documents))print("Modular ablations (remove one module):")score(" without normalize", lambda q: run([only_active], q, documents))score(" without active filter", lambda q: run([normalize], q, documents))score(" both modules", lambda q: run([normalize, only_active], q, documents))Expected output
Naive RAG hit rate: 1 / 4Advanced: normalize + filter hit rate: 4 / 4Modular ablations (remove one module): without normalize hit rate: 2 / 4 without active filter hit rate: 3 / 4 both modules hit rate: 4 / 4Naive RAG answers one question out of four. Each module fixes a different failure: normalization rescues the questions that use abbreviations, and the active filter prevents the outdated vacation policy from winning. Removing either module lowers the score, which is exactly the experiment that modular design makes easy. Advanced and modular pipelines score the same here, because the benefit of modularity is flexibility and testing, not magic.
Knowledge Check
Q1. What is the main difference between advanced and modular RAG?
Q2. A team sees outdated policies in answers. Which addition addresses this most directly?
Q3. What is a sound way to evolve a RAG system?
Key Points
✓
Naive RAG indexes, retrieves and generates in a straight line and is the baseline for every improvement.
✓
Naive RAG fails on poor queries, noisy or outdated chunks, weak ranking, exact terms, multi-fact questions and unsupported answers.
✓
Advanced RAG upgrades the phases before, during and after retrieval.
✓
Modular RAG builds the system from swappable modules arranged linearly, conditionally, in loops or in parallel.
✓
Modules make ablation tests possible: remove one and measure the loss.
✓
Add complexity only in response to measured failures, since each module costs latency and money.
✓
Match the architecture to the situation: simple for clean small data, modular for varied questions and sources.
The Big Picture
The three generations are less a ladder to climb than a toolbox to grow into. The right design is the simplest one that meets your quality goals on real questions, and the habit of measuring each change is what lets you stop at the right place.
Naive baseline → Measure failures → Add targeted modules → Orchestrate → Re-measure
Up Next: Corrective, Self and Adaptive RAG. You will study RAG systems that check their own retrieval and answers: corrective RAG, self-reflective RAG, adaptive routing and retry loops.