Emerging LLM Techniques: Test-Time Compute, Distillation and Beyond

A map of current ideas in LLMs: test-time compute, self-consistency and best-of-N, reasoning models with verifiable rewards, distillation, multimodality, agents, synthetic data and new architectures.

28 minAdvancedCode Examples

The Core Idea: Many recent gains come from using models smarter, not just making them bigger: thinking longer at answer time, voting or verifying several attempts, distilling big models into small ones, and giving models tools and senses.

  1. Beginner: What Is Changing in LLMs Right Now?

LLMs are one of the fastest-moving areas of technology. This lesson covers ideas that are shaping current models rather than specific products, because product names change quickly while the underlying ideas last. Details here are simplified, and the field moves fast, so treat this as a map and not the final word.

TrendOne-line idea
Reasoning and test-time computeLet the model think longer or try several times on hard problems
Distillation and small modelsTeach a small model to imitate a big one
Multimodal modelsUnderstand text, images, audio and video together
AgentsModels that plan, use tools and act over many steps
Synthetic dataUse models to create training data for other models
New architecturesAlternatives and additions to plain attention

Why it matters: Many of the biggest recent gains came not from making models larger but from using them smarter.

  1. Beginner: Thinking Longer Instead of Growing Bigger

For a long time, the main way to improve a model was to train a bigger one. A newer idea is to spend extra effort at answer time, called test-time compute. For hard questions such as math, logic and coding, the model can produce a long chain of reasoning, check its work, or try several solutions before answering.

Train-time computePay once
Idea: Make the model itself stronger.
Cost: Huge, paid before release.
Test-time computePay per question
Idea: Let the model think more on hard problems.
Cost: Paid at every answer, only when needed.

Trade-off: More thinking means slower and more expensive answers, so it is best used for questions that really need it.

  1. Medium: Self-Consistency and Best-of-N

Two simple ways to use extra compute need no new training:

1
Self-consistency: Ask the same question several times with randomness, then take the most common final answer (a majority vote).
2
Best-of-N: Generate N answers and let a verifier (a checker or a scoring model) pick the best one.

Both work because errors are usually scattered while the right answer tends to repeat. If a single answer is right with probability p>0.5p > 0.5 and the tries are independent, a majority vote gets more reliable as you add tries.

Real example from the code: If one answer is right 70% of the time, a majority vote of 21 tries is right about 97% of the time.

Limits: Votes help only when the model is usually right. If it is usually wrong in the same way, more tries just repeat the same mistake.

  1. Medium: Reasoning Models and Verifiable Rewards

Some recent models are trained to produce long reasoning traces before the final answer. A key idea behind them is reinforcement learning with verifiable rewards: for problems where the answer can be checked automatically (a math result, passing unit tests), the model gets a reward for correct final answers, and learns which kinds of reasoning lead to success, without needing human-labeled preferences for every step.

IngredientRole
Problems with checkable answersGive a trustworthy reward signal
Many sampled attemptsThe model learns from its own successes and failures
Long reasoning before answeringSpace to plan, check and correct itself
A cost limitControls how long the model may think

Caution: Reasoning traces can be long, slow and costly, and a confident-sounding chain of thought is not proof that the answer is right. Verify important results.

  1. Advanced: Distillation, Multimodality and Beyond

Distillation trains a small student model to imitate a large teacher. Instead of learning only from hard labels, the student matches the teacher's full probability distribution, which carries rich hints about how similar the options are. A temperature T>1T > 1 softens the teacher's probabilities to make these hints easier to see.

DirectionWhat it isWhy it matters
DistillationA student copies a teacher's outputsSmall, cheap models that keep much of the quality
MultimodalOne model reads text, images and audioAssistants that can see and listen
AgentsPlan, call tools, observe, repeatModels that finish tasks, not just answer
Synthetic dataModels generate training examplesMore data where real data is scarce (needs careful filtering)
State-space and hybrid modelsFixed-size memory mixed with attentionEfficient long sequences
On-device modelsSmall quantized models on phones and laptopsPrivacy and no network needed
How to keep up
Follow ideas rather than names: read model announcements for what changed (data, training method, architecture, inference), and test any new technique on your own task with a proper evaluation set.

  1. Visual Explanation: Spending Compute Where It Counts

The figure contrasts spending compute during training with spending it at answer time, then shows how majority voting improves with more tries.

Part 1 · Two Ways to Spend ComputeBefore release or at answer time
Train-timeBigger or better trained
More parameters or more data
↓
Longer, better training
↓
Smarter model for every question
Test-timeThink more on this question
Long reasoning, or N tries
↓
Vote or verify the answers
↓
Better answers when it matters
Part 2 · Majority Voting Gets StrongerMore tries, more reliable (if p is above 0.5)
1 tryp = 0.70
One answer, 70% chance it is right
↓
70% correct
5 triesMajority vote
Take the most common answer
↓
About 84% correct
21 triesMajority vote
Take the most common answer
↓
About 97% correct
Click to see when voting does not helpCompare ▼
If the model is right less than half the time on a problem, the wrong answer tends to win the vote and more tries make things worse. Voting also assumes tries are somewhat independent, which is not always true. In the code below, p = 0.6 gives only 83% at 21 tries, while p = 0.7 gives 97%.

Part 1 compares the two places to invest effort. Part 2 uses the numbers computed in the code for a model that is right 70% of the time. Open the bar to learn when voting fails.

  1. Python Implementation: Majority Voting, Best-of-N and Distillation

This script calculates the accuracy of majority voting and best-of-N sampling with exact probabilities, then computes the soft teacher probabilities used in distillation and measures how close three different students are to the teacher with a KL distance.

test_time_compute.pyPython 3.10+ · NumPy and the standard library
import numpy as npfrom math import comb # 1. Self-consistency: ask several times, take the majority answerdef majority_correct(p, n):    # chance that more than half of n independent answers are right (n is odd)    return sum(comb(n, k) * p ** k * (1 - p) ** (n - k) for k in range(n // 2 + 1, n + 1)) for p in [0.6, 0.7]:    print("single answer right", p, "-> majority of 5:", round(majority_correct(p, 5), 3), "| of 21:", round(majority_correct(p, 21), 3)) # 2. Best-of-n with a verifier: chance that at least one of n samples is correctfor p in [0.1, 0.3]:    print("one sample right", p, "-> best of 8 (perfect verifier):", round(1 - (1 - p) ** 8, 3)) # 3. Distillation: the student learns from the teacher's soft probabilitiesdef softmax(z, T=1.0):    z = np.array(z) / T    e = np.exp(z - z.max())    return e / e.sum() teacher_logits = [4.0, 2.0, 0.5]for T in [1.0, 3.0]:    print("teacher probabilities at temperature", T, "->", np.round(softmax(teacher_logits, T), 3)) def kl(p, q):    return float(np.sum(p * np.log(p / q))) teacher = softmax(teacher_logits, 3.0)for name, logits in [("student far off", [0.0, 0.0, 0.0]), ("student close", [3.5, 1.8, 0.7]), ("student identical", teacher_logits)]:    print(name.ljust(18), "-> KL distance to teacher:", round(kl(teacher, softmax(logits, 3.0)), 4))
EXPECTED OUTPUT (your numbers may differ slightly)
single answer right 0.6 -> majority of 5: 0.683 | of 21: 0.826
single answer right 0.7 -> majority of 5: 0.837 | of 21: 0.974
one sample right 0.1 -> best of 8 (perfect verifier): 0.57
one sample right 0.3 -> best of 8 (perfect verifier): 0.942
teacher probabilities at temperature 1.0 -> [0.858 0.116 0.026]
teacher probabilities at temperature 3.0 -> [0.548 0.281 0.171]
student far off -> KL distance to teacher: 0.1105
student close -> KL distance to teacher: 0.0038
student identical -> KL distance to teacher: 0.0

A majority of 21 tries lifts a 70% model to about 97%, but a 60% model only to about 83%. Best-of-8 with a perfect checker lifts a 30%-per-try model to about 94%. For distillation, a higher temperature spreads the teacher's probabilities, and the KL distance shrinks to zero as the student matches the teacher.

Interactive Knowledge Check

Question 1: What is test-time compute?
Question 2: When does majority voting help a lot?

Key Points

✓Test-time compute spends extra effort per question instead of making the model bigger.
✓Self-consistency (majority vote) and best-of-N with a verifier improve results without retraining, if single answers are usually right.
✓Reasoning models are trained with long reasoning and rewards that can be checked automatically.
✓Distillation trains a small student to match a big teacher's soft probabilities.
✓Multimodality, agents, synthetic data and new architectures are major directions.
✓Follow ideas rather than product names and always test new techniques on your own evaluation set.

The Big Picture

Emerging Techniques in One Line

Think longer + check more + shrink what you learned + give models tools = better answers at lower cost

This series took you from tokens and attention to serving, evaluation and the frontier. The same fundamentals will help you understand whatever comes next.

Up Next: You have finished the LLM series. A good next step is to build a small project that combines prompting, structured outputs, retrieval and evaluation, then apply these ideas in RAG and AI Agents.