The inference engineering blog

Understand the mechanics. Run the engines. Read the evidence.

Practical guides to LLM inference, from the first generated token to the tradeoffs behind a serving system.

FoundationsSeptember 25, 20266 min read

LLM inference explained: from prompt to generated tokens

Understand LLM inference, prefill and decode, TTFT and TPOT, GPU memory, batching, and the measurements that matter for a serving system.

A request, from text to tokens

Memory & attentionSeptember 25, 20266 min read

Learn what a KV cache stores, calculate memory for grouped-query attention, and understand batching, prefix reuse, paging, and quantization limits.

DeploymentSeptember 25, 20266 min read

Run a pinned vLLM server with Qwen3, send an authenticated chat request, understand memory settings, and troubleshoot your first deployment.

Serving systemsSeptember 25, 20267 min read

Compare vLLM and SGLang using attributed benchmark results, latency tradeoffs, and a repeatable method for choosing an inference server.