The Core Idea: Retrieval hands you a pile of chunks. Context construction decides which ones the model actually reads, in what order, and with what instructions. Grounded generation means the model answers only from that context, admits when it lacks information, and handles disagreement between sources. This last step often decides whether a RAG answer is trusted.
1. Beginner: From Chunks to a Prompt
A RAG prompt has three parts, and the middle one is the retrieved context.
Instructions
Rules for the model: use only the context, cite sources, say when you do not know.
Context
The selected, ordered and labeled passages from retrieval.
Question
The user question, placed clearly after the context.
Grounded generation is the practice of tying every statement in the answer to that context. A grounded answer can be checked against the passages; an ungrounded one relies on the model memory, which is exactly what RAG tries to avoid.
2. Intermediate: Selecting Context
More context is not better context. Every passage costs tokens, adds delay, and gives the model more to be distracted by. Selection applies a few simple rules in order.
Threshold. Drop chunks with a relevance score below a minimum.
Deduplicate. Remove repeated or near-identical chunks, which waste space and overweight one idea.
Diversify. Avoid filling the context with several chunks that say the same thing. A technique called maximal marginal relevance balances relevance against similarity to chunks already chosen.
Fit the budget. Add chunks in order of score until the prompt budget is used.
The prompt budget is whatever remains of the model window after the instructions, the question and the space reserved for the answer.
3. Intermediate: Ordering and Compression
Order matters because models do not use all positions equally. Research on long prompts has found that information in the middle can be used less reliably than information at the start or end. So a common choice is to put the strongest passages at the beginning and the end, and weaker ones in the middle.
| Ordering | How | Use when |
|---|---|---|
| Best first | Highest score at the top | A short context with a clear winner |
| Edges first | Best chunks at the start and the end | Longer contexts with many chunks |
| Document order | Chunks in their original reading order | Passages that must be read as a story |
Compression trims each chunk to the sentences that answer the question, so more distinct evidence fits in the budget. Remember the risk from the reranking lesson: compression may remove a sentence that qualifies the answer.
4. Intermediate: Building a Prompt for Grounded Answers
The instructions are where grounding is requested. A reliable template contains these elements.
Role and scope: answer only from the context below.
Fallback: if the context does not contain the answer, say that you do not know.
Conflict rule: if sources disagree, prefer the newest and mention the older value.
Format: give a short answer, then cite source ids.
Labels: wrap each passage with an id, its source and date, so it can be cited.
Safety tip: retrieved text is data, not instructions. Separate it clearly from your own instructions, and tell the model to ignore commands that appear inside the context. A later lesson covers prompt injection in detail.
5. Advanced: Missing and Conflicting Information
Two situations separate careful systems from careless ones.
Missing information
If nothing in the context scores above the threshold, do not send weak chunks and hope. Either skip generation and answer that nothing was found, or send an empty context with an instruction to say so. Test with questions the documents do not cover.
Conflicting information
Two sources can disagree, for example an old and a new policy. Resolve by metadata when you can: newest wins, or the more authoritative source wins. When you cannot, show both values and say they differ.
Test it deliberately: keep a test set with unanswerable questions and with known conflicts, and check that the system refuses or flags them instead of producing a confident but unsupported answer.
6. Visual Explanation: Building the Context Step by Step
Panel A shows the selection funnel. Panel B shows the chunks from the code below and what happened to each one. Panel C lists edge cases.
Panel A · The Selection FunnelFive chunks in, a clean context out
Retrieved chunks→Drop weak scores→Remove duplicates→Fit the budget→Order and label
Panel B · What Happened to Each ChunkFrom the code in section 7
| Chunk | Score | Outcome |
|---|---|---|
| c1, handbook 2024 | 0.91 | kept, but conflicts with c2 |
| c2, policy update 2025 | 0.89 | kept, newest source |
| c3, copy of c2 | 0.88 | dropped, duplicate |
| c4, FAQ | 0.74 | kept |
| c5, newsletter | 0.31 | dropped, below threshold |
Panel C · Edge CasesClick a case to open it
Click to see why labels are put on every chunk▼
Labels such as an id, a source and a date let the model cite what it used, let you check the answer, and let a conflict rule such as prefer the newest actually work.
How to read it: Panel A is the process, Panel B applies it to real chunks, and Panel C is the checklist for unusual situations.
7. Python Implementation: Select, Resolve Conflicts, Order and Build the Prompt
The program takes five retrieved chunks with scores and dates. It drops chunks below a score threshold, removes duplicates and respects a word budget. It then detects conflicting statements, applies the newest-wins rule, orders the chunks with the strongest at the edges, and assembles the grounded prompt with ids, sources and dates. The final line shows the refusal path when nothing strong enough exists.
context_builder.pyPython 3.10+ · Standard library only
chunks = [ ("c1", 0.91, "2024-03-01", "handbook", "Employees receive 15 vacation days per year."), ("c2", 0.89, "2025-06-10", "policy update", "Employees receive 20 vacation days per year."), ("c3", 0.88, "2025-06-10", "policy update", "Employees receive 20 vacation days per year."), ("c4", 0.74, "2025-01-15", "faq", "Unused vacation days carry over for 3 months."), ("c5", 0.31, "2023-02-01", "newsletter", "The office picnic is in July."),] BUDGET = 22THRESHOLD = 0.5 def count(text): return len(text.split()) def select(chunks): seen = set() picked = [] used = 0 for chunk in sorted(chunks, key=lambda c: -c[1]): chunk_id, score, date, source, text = chunk if score < THRESHOLD or text in seen: continue if used + count(text) > BUDGET: continue seen.add(text) picked.append(chunk) used += count(text) return picked, used def find_conflict(picked): numbers = {} for chunk_id, score, date, source, text in picked: key = " ".join(text.split()[:2]) numbers.setdefault(key, []).append((date, source, text)) return [v for v in numbers.values() if len(set(t for d, s, t in v)) > 1] def order_edges(picked): # put the strongest chunks at the start and the end of the context first = picked[0::2] last = picked[1::2][::-1] return first + last picked, used = select(chunks)print("Kept", len(picked), "of", len(chunks), "chunks using", used, "of", BUDGET, "words")for chunk_id, score, date, source, text in picked: print(" ", chunk_id, score, date, source) conflicts = find_conflict(picked)print("Conflicting statements found:", len(conflicts))for group in conflicts: for date, source, text in sorted(group): print(" ", date, source, "->", text) print(" Prefer the newest:", sorted(group)[-1][0]) prompt = ["Answer only from the context. If it is missing, say you do not know.", "If sources disagree, prefer the newest and mention the older value.", "Context:"]for chunk_id, score, date, source, text in order_edges(picked): prompt.append("[" + chunk_id + "] (" + source + ", " + date + ") " + text)prompt.append("Question: How many vacation days do employees get?")print("")for line in prompt: print(line) print("")no_context = [c for c in chunks if c[1] >= 0.95]print("Strong matches for a question the documents do not cover:", len(no_context), "-> answer: I do not know")Expected output
Kept 3 of 5 chunks using 22 of 22 words c1 0.91 2024-03-01 handbook c2 0.89 2025-06-10 policy update c4 0.74 2025-01-15 faqConflicting statements found: 1 2024-03-01 handbook -> Employees receive 15 vacation days per year. 2025-06-10 policy update -> Employees receive 20 vacation days per year. Prefer the newest: 2025-06-10 Answer only from the context. If it is missing, say you do not know.If sources disagree, prefer the newest and mention the older value.Context:[c1] (handbook, 2024-03-01) Employees receive 15 vacation days per year.[c4] (faq, 2025-01-15) Unused vacation days carry over for 3 months.[c2] (policy update, 2025-06-10) Employees receive 20 vacation days per year.Question: How many vacation days do employees get? Strong matches for a question the documents do not cover: 0 -> answer: I do not knowThree of five chunks survive. The duplicate and the weak newsletter chunk are removed, and the conflict between 15 days in 2024 and 20 days in 2025 is detected, with the newer value preferred. The prompt carries that rule in its instructions, so the model has both the facts and the policy for handling disagreement. The last line proves that the pipeline can decline to answer instead of inventing something.
Knowledge Check
Q1. Why deduplicate retrieved chunks before building the prompt?
Q2. What should a RAG system do when nothing relevant is retrieved?
Q3. Why put retrieved text clearly apart from your own instructions?
Key Points
✓
A RAG prompt has instructions, context and the question; grounded generation ties the answer to the context.
✓
Select context by threshold, deduplication, diversity and the token budget.
✓
Models can use the middle of a long prompt less reliably, so consider putting the best chunks at the edges.
✓
Compression saves space but may remove a qualifier, so test it.
✓
Prompts should say: answer only from context, say you do not know, resolve conflicts, cite ids.
✓
Handle missing information with an honest refusal and conflicts with newest-wins or by showing both values.
✓
Treat retrieved text as data, keep it apart from instructions, and label every chunk.
The Big Picture
Context construction is where retrieval turns into an answer. Choosing, ordering and labeling the evidence, and telling the model how to behave when evidence is missing or contradictory, is what makes the final output reliable instead of merely fluent.
Retrieve → Select → Resolve conflicts → Order → Label → Instruct → Generate
Up Next: Citations and Source Attribution. You will learn how to track sources through the pipeline, generate citations, check that each claim is supported, and prevent unsupported statements.