The Core Idea: A base model knows a lot but does not behave like an assistant. Fine-tuning teaches it a behavior with a small, high-quality dataset. LoRA makes this cheap by freezing the model and training only tiny adapter matrices.
- Beginner: Why Fine-Tune a Model That Already Knows So Much?
After pre-training, a base model knows a huge amount about language and the world, but it behaves like a very well-read autocomplete. Ask it What is the capital of France? and it might continue with What is the capital of Germany? because that is what often follows a question in its training text.
Fine-tuning continues training the model on a much smaller, carefully chosen dataset so that it learns a behavior: follow instructions, answer politely, use a company's writing style, speak medical language, or output clean JSON.
Analogy: Pre-training is like going through school: broad general knowledge. Fine-tuning is the job training: learning how to apply that knowledge in one specific role.
- Beginner: Supervised Fine-Tuning (SFT) and Instruction Data
The most common first step is supervised fine-tuning (SFT), also called instruction tuning. The training data is a set of examples written as instruction and ideal answer pairs:
| Instruction (prompt) | Ideal response (target) |
|---|---|
| Explain gradient descent to a 10-year-old. | Imagine walking down a foggy hill by feeling the slope under your feet... |
| Convert this to JSON: name Ana, age 31 | a JSON object with the name and age fields |
| Summarize this email in one sentence. | A one-sentence summary written in a clear and polite tone. |
The training loop is the same next-token loss as pre-training, but usually computed only on the response part. The model is not rewarded for repeating the question, only for producing a good answer.
Quality over quantity: A small set of high-quality, varied examples often beats a huge noisy set. Many successful assistants were tuned with only thousands of carefully written examples.
- Medium: Full Fine-Tuning and Its Problem
The simplest approach is full fine-tuning: update every weight of the model. It is powerful, but it has serious costs:
| Problem | What it means for a 7B model |
|---|---|
| Memory | Weights, gradients and optimizer states together need roughly 16 bytes per parameter, so about 112 GB for 7 billion parameters. Far beyond a normal GPU. |
| Storage | Every fine-tuned version is a full copy of about 14 GB (16-bit weights). |
| Forgetting | Changing all weights can erase some of the original skills. This is called catastrophic forgetting. |
The big question: Do we really need to change all 7 billion numbers to teach a new style? Research says no. The needed change is usually simple, and that is what LoRA exploits.
- Medium: LoRA (Low-Rank Adaptation)
LoRA keeps the original weight matrix frozen and learns only a small correction next to it. The correction is the product of two thin matrices, and :
If is a table, then is and is , where the rank is tiny, such as 8. So instead of training numbers we train only , about 0.39%.
Why it works: Fine-tuning changes are usually low-rank: they live in a small number of directions. A thin pair of matrices is enough to capture them, so quality stays close to full fine-tuning.
- Advanced: QLoRA, Forgetting and Choosing the Right Tool
QLoRA goes one step further. It stores the frozen base model in 4-bit numbers (about 0.5 byte per parameter, so a 7B model shrinks from about 14 GB to about 3.5 GB) and trains 16-bit LoRA adapters on top. This lets one consumer GPU fine-tune models that would otherwise need a data-center.
| Method | Trains | Rough memory for a 7B model | Best when |
|---|---|---|---|
| Prompting / RAG | Nothing | Just the model for inference | Knowledge changes often, or you only need a better prompt |
| Full fine-tuning | All weights | About 112 GB | You have a big budget and need maximum change |
| LoRA | Small adapters | About 14 GB for the 16-bit base, plus a little | Style, format or domain adaptation on one good GPU |
| QLoRA | Small adapters on a 4-bit base | About 4 to 6 GB in practice | Limited hardware, such as a single consumer GPU |
Next stage: alignment. After SFT, many assistants are refined with preference data, where humans or AI judges say which of two answers is better. Methods such as RLHF and DPO use this to make answers more helpful and safe. That is a topic of its own.
- Visual Explanation: The Fine-Tuning Pipeline and the Method Explorer
This is the path from a base model to a specialist assistant, followed by a quick explorer that compares the four tools from this lesson for a 7B model.
- Python Implementation: Training a LoRA Adapter From Scratch
This script shows LoRA working on a small scale. The big weight matrix W is frozen. The new skill is a low-rank change, and we recover it by training only the thin matrices A and B, which hold just a quarter of the parameters of W in this tiny example and less than half a percent in a real layer.
import numpy as np rng = np.random.default_rng(0)d, r, n = 16, 2, 200 W = rng.normal(size=(d, d)) # frozen pretrained weightsdelta = rng.normal(size=(d, r)) @ rng.normal(size=(r, d)) / d # the new skill is a low-rank changeX = rng.normal(size=(n, d))Y = X @ (W + delta).T # what the fine-tuned model should output A = rng.normal(size=(r, d)) * 0.1 # LoRA matrices: the ONLY things we trainB = np.zeros((d, r)) # B starts at zero, so we begin exactly at the pretrained modellr = 1.0 def current_loss(): return np.mean((X @ (W + B @ A).T - Y) ** 2) print("Trainable parameters: LoRA =", A.size + B.size, "| full fine-tuning =", W.size)print("Loss before training:", round(float(current_loss()), 5)) for step in range(601): E = X @ (W + B @ A).T - Y G = (2 / (n * d)) * E.T @ X # gradient with respect to the combined update B @ A dB, dA = G @ A.T, B.T @ G B -= lr * dB A -= lr * dA if step in (100, 300, 600): print("step", step, "| loss", format(current_loss(), ".2e")) # Same idea at real scale: one 4096 x 4096 layer with rank 8full, lora = 4096 * 4096, 2 * 4096 * 8print("One real layer: full =", format(full, ","), "| LoRA =", format(lora, ","), "|", round(100 * lora / full, 2), "% of the size")The loss before training is about 0.16 (the model lacks the new skill). It collapses toward zero within a few hundred steps even though only 64 of 256 numbers were trained. In a real 4096 × 4096 layer the same trick trains only 0.39% of the numbers.
Interactive Knowledge Check
Key Points
The Big Picture
Base model + a few good examples + small adapters (LoRA) = a specialist assistant at a fraction of the cost
You now understand the full journey of an LLM: how attention and positions work, how a modern block is built, how pre-training creates a base model, and how fine-tuning turns it into something useful. The same ideas appear in every model you will use.
Up Next: Alignment methods such as RLHF and DPO, plus prompting techniques, build on everything in this series. They are the next topics in Prompting and Alignment.