Hugging Face Transformers

Learn to load, run, adapt, and ship pretrained models with Hugging Face Transformers. Pin checkpoints to exact revisions, turn text, images, and audio into model inputs, read logits and generate text, fine-tune with Trainer and LoRA, and package a model that a serving engine such as vLLM can load, checking each step against a known answer.

Beginner → advanced
20 units · approximately 20 hours

How to work through this course

Before you begin: Comfortable Python and basic PyTorch: tensors, shapes, and what a training step does (the PyTorch course covers all three). The lesson code runs on a laptop CPU with Python 3.10 or later and pinned versions of torch, transformers, datasets, accelerate, and peft. Most examples download tiny test checkpoints of a few megabytes, a few use real models of a few hundred megabytes, and the steps that need a GPU or several processes say so.

By the end: Adapt a pretrained text classifier, show with a fixed held-out comparison that it beats the base checkpoint, and hand over a package with its tokenizer, labels, provenance, and golden outputs that reloads offline and gives the same answers when a serving engine runs it.

Understand the idea. Read each section in order and study its diagrams. When a pause-and-predict question appears, commit to an answer before you reveal it. Look up unfamiliar terms in the glossary below.
Try it yourself. Run the code examples and compare your output with the lesson's. Use the browser labs where available; steps that need a GPU are optional and say so.
Check your understanding. Answer the knowledge check at the end of each lesson and make sure you can say why the other options are wrong. Complete all units to unlock the final quiz; the passing score is 80%.

The optional project brings the lessons together. Keep notes on your choices, results, and any failures so you can explain what you learned.

Course orientation

What is Hugging Face Transformers?

Transformers is a Python library for loading, running, and adapting pretrained models. From a Hub repository it builds the configuration, the tokenizer or processor, and the model class, then adds pipelines for quick predictions, generate for text generation, Trainer for fine-tuning, and save_pretrained for packaging. Serving engines such as vLLM and SGLang read the same checkpoint files.

The problem it solves

A pretrained model gives the right answers only with the exact pieces it was trained with. A moved revision, another model's tokenizer, the wrong padding side, a swapped label map, a mislabeled sample rate, or a doubled image rescale changes the output without raising an error, and a training run can lower its loss while learning the wrong task. This course teaches you to catch each of these with small checks: token IDs and logits reproduced by hand, labels traced to the positions they score, packages reloaded offline in a fresh process, and adapted models compared with the base checkpoint on held-out data.

Model development workflow

01Checkpoints and the Hub
A repository holds config.json, the weights, the tokenizer files, and processor settings at a commit. from_pretrained resolves a revision, caches the files, and builds the class the configuration names; from_config builds the same architecture with random weights.
02Inputs
Tokenizers turn text into IDs with special tokens, padding, truncation, and attention masks. Chat templates render conversations, image processors turn pixels into normalized tensors, and feature extractors turn audio samples into features.
03Models and outputs
A base model returns hidden states, and a task head turns them into logits: one score per label, per token, or per vocabulary entry. generate repeats forward passes with a KV cache and decoding rules, and a pipeline wraps preprocessing, the model, and postprocessing in one call.
04Adaptation and handoff
Datasets and collators build batches, Trainer or a custom loop fine-tunes the model or a LoRA adapter, evaluation compares the result with the base checkpoint, and save_pretrained writes a package that a serving engine can load.

What it covers

Loading pinned checkpoints, configurations, tokenizers, processors, and task heads from the Hub or a local directory.
Tokenization, chat templates, and image and audio preprocessing into the tensors each model expects.
Forward passes, named model outputs, and text generation with decoding strategies, stopping rules, streaming, and a KV cache.
Fine-tuning with Trainer or a custom loop, parameter-efficient adapters such as LoRA, and Datasets and collators for the data.
Loading large models with device_map and quantization, multi-process training through Accelerate, and saving everything needed to reproduce inference.

Outside its scope

High-throughput serving. generate is a readable reference loop in one process; engines such as vLLM and SGLang add continuous batching, a paged KV cache, and an HTTP API while loading the same checkpoint files.
Deciding whether a model is good enough. Test data, metrics, and acceptance rules are your choices, and the course shows how to fix them before you measure.
Real accuracy from the tiny test checkpoints. Most examples use random test models so they run in seconds; they show the mechanics, and the lessons say where a real checkpoint changes the numbers.
Running Python in the browser. The lessons explain and illustrate in the browser, and the code runs on your own machine.

Key terms in this course

Checkpoint: A model's saved weights together with its configuration, usually in a Hub repository or a local directory.
Revision: A branch, tag, or 40-character commit hash that selects one version of a repository. Only a commit hash can never move.
Configuration: The config.json that names the architecture, sets its sizes, and stores label names such as id2label.
Auto class: A class such as AutoModelForSequenceClassification that reads the configuration and builds the matching model class.
Tokenizer: The vocabulary and rules that turn text into token IDs and back, including special tokens, padding, and truncation.
Attention mask: A tensor of ones and zeros that tells the model which positions are real tokens and which are padding.
Task head: The layers on top of a base model that turn hidden states into task scores, such as one logit per label or per vocabulary entry.
Logits: The raw scores a model returns before softmax or sigmoid turns them into probabilities.
generate: The method that runs a model repeatedly, choosing and appending one token per step until a stopping rule ends the output.
Chat template: A template stored with a checkpoint that renders a list of messages into the exact prompt text the model was trained on.
Collator: The function that pads the examples of one batch to a common length and stacks them into tensors.
Trainer: The Transformers class that runs a training loop from TrainingArguments: batching, optimization, evaluation, logging, and checkpoints.
LoRA adapter: A small set of low-rank matrices trained on top of a frozen base model; it only works with the exact base it was trained on.
Processor: A model-specific object that prepares images, audio, or text together, such as an image processor or a feature extractor.

Syllabus

Optional field project

Adapt and package a pretrained text classifier

01Pin the base model and trace one prediction by hand
02Save the base model and read what it predicts
03Fine-tune and compare against the base checkpoint
04Package, serve, and decide

View optional project brief