Large language models, pretraining, fine-tuning, alignment, and generative architectures.
A beginner-friendly introduction to Large Language Models: what they are, why they are called large, how they differ from rule-based code, what they learn, training vs. inference, and where they fit in modern Generative AI.
Learn how Large Language Models convert human language into tokens and token IDs, including vocabulary, subword tokenization, special tokens, BPE, WordPiece, SentencePiece, context length, chat templates, and why tokenization matters.
Learn how token IDs become dense numerical vectors, why embeddings represent semantic information, how embedding tables work, positional information, similarity, dimensions, and how representations flow into Transformer layers.
Understand how an LLM turns a prompt into probabilities, selects the next token, and repeats the process to generate text.
Understand the architecture of decoder-only Transformers used by modern LLMs, from token embeddings and attention to feed-forward networks, residual connections, normalization, and next-token prediction.
Understand how causal masking keeps a language model from seeing the future, and how the KV cache lets it generate text without redoing old work.
Understand why attention cannot see word order, how older methods added position, and how Rotary Position Embedding (RoPE) lets modern LLMs focus on the distance between tokens, plus how context windows get stretched.
Learn what a modern large language model is built from: the decoder-only design, pre-norm RMSNorm, SwiGLU feed-forward layers, RoPE, grouped-query attention and Mixture of Experts, and count the parameters of a 7B model in code.
Learn how a random neural network becomes a language model: the next-token objective, how training data is collected and cleaned, loss and perplexity, scaling laws, the 6ND compute rule and how training is spread over thousands of GPUs.
Learn how an LLM turns probabilities into text: greedy decoding, temperature, top-k and top-p sampling, repetition penalties, seeds and stop sequences, with every method built from scratch in code.
Learn what a context window really is, how to budget tokens, what to do when a conversation gets too long, why very long prompts cost more and can be less accurate, and how the KV cache and prompt caching affect memory and price.
Learn how much GPU memory an LLM needs, what float16, bfloat16, int8 and int4 mean, why training needs far more memory than inference, and how quantization shrinks a model with only a small loss in quality.
Learn how to call an LLM from code: messages and roles, token pricing, API key safety, retries with backoff, and how to choose between hosted APIs and open-weight models that you run yourself.
See why long inputs are expensive and how FlashAttention, sliding-window and sparse attention, GQA, ring attention and linear-time architectures make long context practical, with online softmax built from scratch.
Learn how Mixture of Experts models run only part of the network for each token: the router, top-k expert selection, load-balancing loss, capacity limits, and the difference between total and active parameters.
Learn how production systems serve LLMs fast and cheaply: prefill vs. decode, latency and throughput metrics, continuous batching, PagedAttention, prefix caching, quantization and speculative decoding.
Learn how to measure LLM quality honestly: benchmarks and their pitfalls, accuracy, perplexity and pass@k, pairwise comparison and Elo, LLM-as-judge and its biases, and why confidence intervals matter.
A map of current ideas in LLMs: test-time compute, self-consistency and best-of-N, reasoning models with verifiable rewards, distillation, multimodality, agents, synthetic data and new architectures.
Learn how a pretrained base model becomes a helpful specialist: supervised fine-tuning with instruction data, why full fine-tuning is costly, how LoRA and QLoRA train tiny adapters, and when to fine-tune instead of prompting or using RAG.
Learn how language models are steered toward helpful, honest and safe behavior: preference data, reward models, RLHF with PPO, the simpler DPO loss, and common failure modes such as reward hacking and sycophancy.
Learn how to write clear prompts: role, task, context, format, constraints and examples, few-shot prompting, delimiters, step-by-step reasoning, testing and iterating, and basic protection against prompt injection.
Learn how to get reliable JSON from a model with schemas and constrained decoding, how tool calling lets a model request actions that your code runs, and how to validate and secure everything the model returns.