The Core Idea: Computers cannot compare meanings directly, but they can compare numbers. An embedding is a list of numbers that represents the meaning of a piece of text, learned so that texts with similar meaning get similar numbers. RAG uses embeddings to find passages that mean the same thing as the question, even when they share none of its words.
1. Beginner: From Words to Coordinates
Think of a map. Every city has coordinates, and cities that are close together have similar coordinates. You can find the nearest city to any point just by comparing numbers. An embedding does the same for text: it gives every sentence a position in a space of meaning, and sentences about similar things sit near each other.
A map needs 2 numbers per place. An embedding needs many more, often hundreds or thousands, because meaning has far more directions than north-south and east-west.
Key point: an embedding is not a summary and not a code you can read. It is a position. Its only job is to make similar meanings close and different meanings far apart.
2. Beginner: Meaning Beyond Keywords
Keyword search looks for the same words. Embedding search looks for the same meaning. Consider a user asking about a puppy while the document talks about a young dog: the two texts share no important word, yet they are about the same thing.
| Question | Document | Keyword search | Embedding search |
|---|---|---|---|
| How do I train a puppy? | Teaching a young dog basic commands | Weak match, few shared words | Strong match by meaning |
| car will not start | automobile engine troubleshooting | No shared words | Strong match by meaning |
Embedding models learn this from huge amounts of text. They are trained so that texts used in similar contexts, or that answer one another, end up with nearby vectors.
3. Intermediate: Embedding Space and Dimensions
The number of values in an embedding is its dimension. Common models produce vectors with a few hundred to a few thousand dimensions. You cannot read a single dimension as a human concept like animal or vehicle. Meaning is spread across all of them together.
More dimensions can store finer distinctions, but they cost memory and search time. Storage is easy to estimate. If each number takes 4 bytes (a 32-bit float), then chunks of dimension need about:
For example, 1,000,000 chunks with 768 dimensions need bytes, about 3 gigabytes, before any index structures. This is why dimension and storage format matter in production.
4. Intermediate: Similarity Intuition
Once texts are vectors, similarity is geometry. Two vectors pointing in nearly the same direction represent similar meanings. The most common score, cosine similarity, measures the angle between vectors: 1 means same direction, 0 means unrelated, negative values mean opposite directions.
High score
The texts are about the same thing and are good retrieval candidates.
Middling score
Related but not the same, or only partly overlapping topics.
Low score
Probably unrelated. The retriever should not return this text.
Careful: a similarity score is not a probability or a measure of correctness. A score of 0.8 from one model can mean something different from 0.8 from another, so thresholds must be tuned for each model.
5. Advanced: Query vs Document Embeddings and Their Limits
Questions and passages look different. A question is short and asks, a passage is long and states. Many retrieval models are trained for this asymmetry, and some expect a short prefix or instruction that tells them whether the text is a query or a document. Always follow the model documentation, and always use the same model for both sides.
Embeddings also have known weaknesses you should test for:
| Limitation | Example | What to do |
|---|---|---|
| Negation | Refunds allowed and Refunds not allowed look similar | Verify with the generator or a reranker |
| Exact identifiers | Order codes and rare product names | Add keyword search (hybrid retrieval) |
| Numbers and dates | 30 days and 60 days embed almost the same | Use metadata filters |
| Long inputs | Text beyond the model limit is cut off | Chunk before embedding |
6. Visual Explanation: A Map of Meaning
Panel A draws a tiny two-dimensional map of meaning. Panel B shows what happens to a query. Panel C is an explorer of common misunderstandings.
Panel A · The MapClose together means similar meaning
cat
kitten
dog
car
truck
automobile
banana
apple
your query: puppy
The query lands near the animals, so animal passages win, even without shared words
Panel B · Embedding a QuerySame model for queries and documents
Question→Embedding model→Vector of numbers→Nearest document vectors
Panel C · MythsClick a myth to open it
Click to see why the original text is stored too▼
The vector is only used to find the chunk. The model needs the actual words to read, so the vector store keeps the chunk text and metadata next to every vector.
How to read it: Panel A is the idea, Panel B is the mechanism, and Panel C removes the most common wrong assumptions.
7. Python Implementation: Toy Embeddings and Semantic Search
Real embeddings come from a trained neural network. To see the idea without any download, this program uses a tiny hand-made word vector table with four dimensions (animal, vehicle, food, technology). A sentence embedding is the average of its known word vectors. The search then ranks documents by cosine similarity. The last lines recompute the storage estimate from the lesson.
toy_embeddings.pyPython 3.10+ · Standard library only
import math word_vectors = dict([ ("cat", [0.9, 0.0, 0.0, 0.0]), ("kitten", [0.85, 0.0, 0.0, 0.0]), ("dog", [0.9, 0.0, 0.05, 0.0]), ("puppy", [0.85, 0.0, 0.05, 0.0]), ("car", [0.0, 0.9, 0.0, 0.1]), ("automobile", [0.0, 0.9, 0.0, 0.1]), ("engine", [0.0, 0.7, 0.0, 0.3]), ("banana", [0.0, 0.0, 0.9, 0.0]), ("apple", [0.0, 0.0, 0.9, 0.05]), ("laptop", [0.0, 0.1, 0.0, 0.9]), ("phone", [0.0, 0.1, 0.0, 0.9]),]) def embed(text): known = [word_vectors[w] for w in text.lower().split() if w in word_vectors] if not known: return [0.0, 0.0, 0.0, 0.0] return [sum(v[i] for v in known) / len(known) for i in range(4)] def cosine(a, b): dot = sum(x * y for x, y in zip(a, b)) na = math.sqrt(sum(x * x for x in a)) nb = math.sqrt(sum(y * y for y in b)) if na == 0 or nb == 0: return 0.0 return dot / (na * nb) documents = [ "the kitten sleeps", "an automobile engine", "a ripe banana", "new laptop and phone",] for query in ["my puppy", "car trouble", "apple pie"]: q = embed(query) scored = sorted(((cosine(q, embed(d)), d) for d in documents), reverse=True) best_score, best_doc = scored[0] shared = set(query.split()) & set(best_doc.split()) print(query.ljust(12), "->", best_doc.ljust(24), round(best_score, 2), "shared words:", len(shared)) chunks = 1000000dimensions = 768print("")print("Storage for", chunks, "chunks:", chunks * dimensions * 4 / 1000000000, "GB")Expected output
my puppy -> the kitten sleeps 1.0 shared words: 0car trouble -> an automobile engine 0.99 shared words: 0apple pie -> a ripe banana 1.0 shared words: 0 Storage for 1000000 chunks: 3.072 GBThe query my puppy finds the kitten sentence although the two share no word, and car trouble finds the automobile sentence for the same reason. That is semantic search in miniature. Real models do this with hundreds of dimensions learned from data, and they cope with far more words and phrasing than any table could. The storage line confirms the roughly 3 gigabyte estimate.
Knowledge Check
Q1. What is an embedding?
Q2. A query and a document share no words yet are close in embedding space. What does this show?
Q3. Why must queries and documents use the same embedding model?
Key Points
✓
An embedding is a list of numbers that places text in a space of meaning.
✓
Similar meanings get nearby vectors, which lets retrieval work even when wording differs.
✓
Dimensions are not individually interpretable, and more dimensions cost more storage.
✓
Storage is roughly chunks times dimensions times 4 bytes for 32-bit numbers.
✓
Cosine similarity compares directions; its values are not probabilities and thresholds depend on the model.
✓
Queries and documents must use the same model, following its query and document conventions.
✓
Embeddings struggle with negation, exact identifiers, numbers and very long input, so combine them with other tools.
The Big Picture
Embeddings are the bridge between human language and fast computation. Everything in retrieval, from vector databases to reranking, assumes that you can ask how close two meanings are. Now you know what is being measured and where the measurement can fail.
Text → Embedding model → Vector → Compare with other vectors → Nearest meanings
Up Next: Embedding Models and Similarity. You will compare the main types of embedding models, work with cosine similarity, dot product and Euclidean distance, and learn how to choose a model for your data.