The Core Idea: Every RAG system is really two pipelines joined by one shared store. An indexing pipeline runs ahead of time and prepares your documents so they can be searched. A query pipeline runs every time a user asks a question: it searches the prepared store, builds a prompt from what it finds, and asks the model to answer. Once you can picture these two pipelines, every later lesson is just a closer look at one box.
1. Beginner: One System, Two Pipelines
The meeting point: the index. The indexing pipeline writes to it and the query pipeline reads from it. Keeping the heavy work on the writing side is what makes answering quick.
2. Beginner: The Indexing Pipeline (Offline)
Why do it offline? Embedding thousands of chunks takes time and money. Doing it once, and again only for new or changed documents, means no user ever waits for it.
3. Beginner: The Query Pipeline (Online)
One rule you must never break: the question and the documents must be embedded by the same embedding model. Embeddings from two different models live in different number spaces, so comparing them is meaningless.
4. Intermediate: Offline vs Online Processing
| Aspect | Indexing (offline) | Query (online) |
|---|---|---|
| When it runs | Before launch, then on a schedule or when documents change | Every time a user asks |
| Main goal | Quality and completeness of the index | Low latency and good answers |
| Speed pressure | Low: can take minutes or hours | High: users wait |
| Typical cost | Embedding every chunk | One query embedding, one search, one LLM call |
| If it fails | Index is missing or stale | User sees an error or a bad answer right now |
| Touches the index | Writes | Reads |
Design consequence: any step that can be moved from online to offline usually should be. Pre-computing embeddings, cleaning text and attaching metadata all happen in indexing so the query path stays short.
5. Advanced: The Context Budget and Where the Pipeline Grows
| Added stage | Where it goes | What it improves |
|---|---|---|
| Query rewriting | Before retrieval | Vague or messy questions |
| Metadata filtering | During retrieval | Relevance, permissions, freshness |
| Hybrid search and reranking | Retrieval and after it | Ranking quality |
| Citations | Generation and output | Trust and checking |
| Evaluation and monitoring | Around the whole system | Finding and fixing failures |
6. Visual Explanation: Two Lanes, One Index, and Where Things Break
| Stage | What the data looks like |
|---|---|
| Load | handbook.txt, a whole file of text |
| Chunk | Employees get 20 vacation days per year. |
| Embed and store | A list of numbers, saved with the text and source name |
| Question | How many vacation days do employees get? |
| Retrieve | The best chunks with similarity scores |
| Prompt | Instruction, then context chunks, then the question |
| Answer | 20 vacation days per year, source: handbook.txt |
Click to see the golden debugging ruleExplain ▼
7. Python Implementation: A Complete Two-Pipeline RAG System
build_index is the offline pipeline: it chunks documents by sentence, vectorizes each chunk with word counts, and stores the source name next to it. retrieve, build_prompt and generate form the online pipeline. A real system would send the prompt to an LLM; here generate is a simple stand-in that reads the first context line from the prompt, so the whole program runs with the standard library only.import mathfrom collections import Counter documents = [ ("handbook.txt", "Employees get 20 vacation days per year. Unused days can be carried over for 3 months. Remote work is allowed on Tuesdays and Thursdays."), ("pricing.txt", "The Basic plan costs 10 dollars per month. The Pro plan costs 25 dollars per month. Annual billing gives a 15 percent discount."), ("support.txt", "Support is available Monday to Friday. Premium customers get a reply within 4 hours. Standard customers get a reply within 24 hours."),] stop_words = set(["the", "is", "a", "of", "how", "many", "much", "does", "do", "per", "to", "can", "for", "on", "get", "gets"]) # Shared tools: used by BOTH pipelines (same vectorizer for docs and queries)def vectorize(text): cleaned = "".join(ch for ch in text.lower() if ch.isalnum() or ch == " ") return Counter(w for w in cleaned.split() if w not in stop_words) def cosine(a, b): dot = sum(a[w] * b[w] for w in set(a) & set(b)) norm_a = math.sqrt(sum(v * v for v in a.values())) norm_b = math.sqrt(sum(v * v for v in b.values())) if norm_a == 0 or norm_b == 0: return 0.0 return dot / (norm_a * norm_b) # ===== OFFLINE PIPELINE: chunk, embed, store =====def chunk(text): return [s.strip() + "." for s in text.split(".") if s.strip()] def build_index(documents): index = [] for source, text in documents: for piece in chunk(text): index.append((source, piece, vectorize(piece))) return index # ===== ONLINE PIPELINE: retrieve, build prompt, generate =====def retrieve(question, index, k=2): q = vectorize(question) scored = [(cosine(q, vec), source, piece) for source, piece, vec in index] scored.sort(reverse=True) return scored[:k] def build_prompt(question, hits): lines = ["Answer using ONLY the context.", "Context:"] for score, source, piece in hits: lines.append("- " + piece + " (" + source + ")") lines.append("Question: " + question) return lines # Stand-in for the LLM call: it just reads the first context linedef generate(prompt): first = prompt[2] text, source = first[2:].rsplit(" (", 1) return text + " [source: " + source.rstrip(")") + "]" # Run the offline pipeline onceindex = build_index(documents)print("Chunks in index:", len(index))print("") # Run the online pipeline for each questionquestions = [ "How many vacation days do employees get?", "How fast do Premium customers get a reply?",] for question in questions: hits = retrieve(question, index, k=2) print("Question:", question) for score, source, piece in hits: print(" hit", round(score, 3), source, "|", piece) prompt = build_prompt(question, hits) print(" Prompt:") for line in prompt: print(" " + line) print(" Answer:", generate(prompt)) print("")Chunks in index: 9 Question: How many vacation days do employees get? hit 0.775 handbook.txt | Employees get 20 vacation days per year. hit 0.218 handbook.txt | Unused days can be carried over for 3 months. Prompt: Answer using ONLY the context. Context: - Employees get 20 vacation days per year. (handbook.txt) - Unused days can be carried over for 3 months. (handbook.txt) Question: How many vacation days do employees get? Answer: Employees get 20 vacation days per year. [source: handbook.txt] Question: How fast do Premium customers get a reply? hit 0.612 support.txt | Premium customers get a reply within 4 hours. hit 0.408 support.txt | Standard customers get a reply within 24 hours. Prompt: Answer using ONLY the context. Context: - Premium customers get a reply within 4 hours. (support.txt) - Standard customers get a reply within 24 hours. (support.txt) Question: How fast do Premium customers get a reply? Answer: Premium customers get a reply within 4 hours. [source: support.txt]build_index ran once and was never called again, while retrieve, build_prompt and generate ran for every question. That is the offline and online split in code form. Also notice that the metadata (the file name) travels with every chunk from the index into the prompt and finally into the answer, which is what makes citations possible.Knowledge Check
Key Points
The Big Picture
Up Next: Data Sources and Document Processing. You will start at the very beginning of the offline lane: how to load PDFs, web pages, Markdown, HTML, CSV, JSON and databases, and how to extract clean text from them.