PyTorch

Learn PyTorch from the first tensor to a model you can hand over. Build and train models with autograd, modules, data loaders, and optimizers, evaluate them without leakage, save and restore them safely, and make inference faster with batching, mixed precision, compilation, and export, proving every change with a measurement.

Beginner → advanced
20 units · approximately 20 hours

How to work through this course

Before you begin: Comfortable Python: functions, classes, lists, and dictionaries. No machine learning background is needed; matrix products, derivatives, and averages are introduced with small worked examples. The lesson code runs on a laptop CPU with Python 3.10 or later and PyTorch 2.14.0, and the few steps that need a GPU or several processes say so.

By the end: Train, evaluate, and checkpoint a classifier that someone else can reproduce, prove that restoring, batching, and exporting it leave its outputs unchanged, and accept or reject each speed change with a measured comparison on a fixed workload.

Understand the idea. Read each section in order and study its diagrams. When a pause-and-predict question appears, commit to an answer before you reveal it. Look up unfamiliar terms in the glossary below.
Try it yourself. Run the code examples and compare your output with the lesson's. Use the browser labs where available; steps that need a GPU are optional and say so.
Check your understanding. Answer the knowledge check at the end of each lesson and make sure you can say why the other options are wrong. Complete all units to unlock the final quiz; the passing score is 80%.

The optional project brings the lessons together. Keep notes on your choices, results, and any failures so you can explain what you learned.

Course orientation

What is PyTorch?

PyTorch is a Python library for computing with tensors, multidimensional arrays of numbers, on CPUs and GPUs. It records the operations you run so it can compute gradients automatically, and it provides layers, loss functions, optimizers, data loading, mixed precision, a compiler (torch.compile), and a way to capture a model as a standalone graph (torch.export). Most open models, and the engines that serve them, are built on it.

The problem it solves

Model code usually fails without an error. A tensor with the right shape can hold values in the wrong places, a training loop with a step missing still prints a falling loss, a checkpoint can restore the weights but not the preprocessing, and a faster version can quietly change predictions. This course teaches you to catch each of these with small, concrete tests: values traced by hand, parameters that must move, logits compared after every restore, batch, and export, and timings measured under stated conditions.

Model development workflow

01Tensors and autograd
Tensors hold data with a shape, a dtype, and a device. Autograd records the operations that produce a loss, and a call to backward() fills in the gradient of every parameter.
02Modules, data, and optimizers
nn.Module registers a model's parameters and buffers. Dataset and DataLoader turn records into batches, a loss function scores the predictions, and an optimizer turns gradients into weight updates.
03Training and evaluation
Each training step clears gradients, runs the model, computes the loss, runs backward, and updates the weights. Validation runs in evaluation mode without gradients, on data split so that nothing leaks from training.
04Checkpoints and inference
state_dict and torch.save store weights and training state. eval() and inference_mode run predictions, batching and mixed precision make them cheaper, and torch.compile and torch.export speed up or package the model for serving.

What it covers

Tensor math on CPUs and GPUs, with shapes, dtypes, strides, views, and broadcasting.
Automatic gradients, model building blocks, loss functions, optimizers, and data loading.
Saving and loading weights and full training state, and restoring models for inference.
Mixed precision, activation checkpointing, profiling, compilation, and distributed training with DDP and FSDP.
Capturing a model as a graph with torch.export so other runtimes can load it without your Python code.

Outside its scope

Running a service. PyTorch runs the model; request validation, batching across users, timeouts, and monitoring are code you build around it, or come from serving engines such as vLLM and SGLang.
Deciding whether a model is good enough. The data splits, metrics, and acceptance thresholds are your choices, and the course shows how to make them before you measure.
Identical results on every machine. Seeds make a run repeatable on one setup; different devices and kernels can differ in the last bits.
Running Python in the browser. The lessons explain and simulate in the browser, and the code runs on your own machine.

Key terms in this course

Tensor: A multidimensional array with a shape, a dtype such as float32 or bfloat16, and a device such as the CPU or a GPU.
Autograd: PyTorch's automatic differentiation: it records operations during the forward pass and applies the chain rule backward to compute gradients.
Parameter and buffer: Tensors registered on a module. Parameters are trained by the optimizer; buffers, such as normalization statistics, are saved and moved with the model but not trained.
state_dict: A dictionary of a module's parameters and buffers by name, used to save and restore weights.
DataLoader: The object that draws records from a Dataset, groups them into batches, and optionally shuffles them and loads them in worker processes.
Optimizer: An algorithm such as SGD or AdamW that updates parameters from their gradients, keeping state such as momentum between steps.
Evaluation mode: The mode set by model.eval(), in which dropout is off and batch norm uses its stored statistics. It does not turn off gradients.
Inference mode: The context torch.inference_mode(), which stops recording operations for autograd so predictions use less memory and time.
Checkpoint: A saved file with model weights, and for resuming training also the optimizer, scheduler, progress, and random state.
Mixed precision: Running most operations in bfloat16 or float16 while keeping sensitive ones in float32, usually with torch.autocast.
torch.compile: A compiler that traces a model into graphs and generates faster code, at the cost of compile time on the first calls.
DDP and FSDP: Distributed data parallel copies the whole model to every GPU; fully sharded data parallel splits parameters, gradients, and optimizer state across them.
torch.export: A tool that captures a model's tensor operations, weights, and accepted input shapes as one file that loads without the original Python class.

Syllabus

Optional field project

Train, restore, and optimize a small classifier

01Prove your setup, shapes, and gradients
02Train and select a baseline without leakage
03Restore the model and prove it behaves the same
04Measure, optimize, and package the model

View optional project brief