Writing
All posts
How GPT Works — Part 5: From Base Model to ChatGPT
The final step: how a raw next-token predictor becomes a helpful assistant. Part 5 covers pretraining, supervised fine-tuning, and RLHF, the difference between a base model and a chat model, plus the context window and KV-cache that govern inference.
How GPT Works — Part 4: Training & Generation
How a transformer learns and how it writes. Part 4 covers next-token cross-entropy training with an interactive loss-descent demo, then decoding strategies — greedy, temperature, top-k, and top-p sampling — you can reshape live.
How GPT Works — Part 3: The Transformer
How the attention mechanism becomes a working language model. Part 3 covers subword tokenization with live Byte-Pair Encoding, the transformer block (residuals, LayerNorm, MLP), the causal mask that makes a GPT decoder-only, and the full pipeline from text to next-token probabilities.
How GPT Works — Part 2: Attention
From the seq2seq bottleneck to the mechanism that replaced recurrence entirely. Part 2 builds attention from the ground up — soft alignment, scaled dot-product self-attention with Q/K/V, multi-head attention, and why a transformer needs positional encoding.
How GPT Works — Part 1: The Foundations
A visual, hands-on guide to how large language models work. Part 1 covers the only prerequisites you need — vectors, the dot product, matrix multiplication, and softmax — then the one idea the whole model is built on: next-token prediction.
Image Captioning with the 'Merge' Architecture
The 'merge' architecture generates image captions differently from the traditional 'inject' approach where image features are fed into the RNN. Here's how it works.
Attention Model for Machine Translation
Attention is one of the most powerful sequence-to-sequence ideas, powering machine translation, image captioning, and more. Here's how the attention model and mechanism work, plus a date-translation demo.
Proximal Policy Optimization (PPO)
PPO became an industry-standard RL algorithm after OpenAI's release. This post explains the clipped surrogate objective and why PPO is stable and effective.
Policy Gradient and the Actor-Critic Algorithm
Where Deep Q-Learning falls short, policy gradient methods step in. This post builds from the policy gradient up to the Advantage Actor-Critic (A2C) algorithm.
Deep Q-Learning and Advancements over Deep Q-Networks
Using deep neural networks for Q-learning to build an agent that plays Flappy Bird — plus key improvements to DQN like Double DQN, Dueling Networks, and Prioritized Experience Replay.
Monte Carlo and Temporal Difference Learning
When the full MDP is unknown, an agent must learn from experience. This post covers Monte Carlo and Temporal Difference methods for model-free reinforcement learning.
Policy Optimization in a Known MDP
When the MDP is fully known, an agent can compute an optimal policy directly. This post walks through policy iteration, value iteration, and related dynamic-programming techniques.
LSTM — Long Short Term Memory Networks
LSTMs are a special kind of RNN capable of learning long-term dependencies. This post demystifies the cell state, the gates, and how LSTMs remember.
Word Embeddings in Natural Language Processing
To use words in NLP models we must turn them into numbers. This post explores word embeddings — from one-hot vectors to Word2Vec and GloVe — and why they work.
No posts match this topic.