Inference Engineering Academy
Sandipan Kundu
Practical education in building, understanding, and operating model inference systems.
Maintainer and author
Sandipan Kundu maintains Inference Engineering Academy and authors its technical guides. The curriculum covers Foundations, CUDA, PyTorch, Transformers, vLLM, SGLang, and NVIDIA Dynamo.
How to assess our technical content
- Courses identify their target software release. Check that release before copying commands into a different environment.
- Articles link primary documentation and research so readers can inspect the evidence.
- Directory entries distinguish documented support, conditional support, and missing evidence. Unknown support does not mean unsupported.
- Calculated memory requirements are estimates with explicit assumptions. They are separate from measured GPU performance.
- Published benchmark comparisons attribute their results to the original authors. A suggested experiment is not a claim that we ran it.
Report a correction with the page URL, software version, observed behavior, and a primary source or reproduction. Email contact@inference-engineering.xyz.
Articles by Sandipan Kundu
- LLM inference explained: from prompt to generated tokens
- LLM inference optimization: a practical guide to lower latency
- Mixture of experts (MoE) inference: memory, routing, and GPUs
- Prefill vs. decode: how LLM inference latency works
- Tensor parallelism vs. pipeline parallelism for LLM inference
- Prefix caching explained: faster LLM prompts with KV reuse
- FlashAttention vs PagedAttention: what each optimizes
- GPU memory for LLM inference: how to size and choose a GPU
- AI cloud infrastructure for LLM inference: a practical guide
- What is AI inference? How trained models answer in production
- How to learn inference engineering: a step-by-step roadmap
- Speculative decoding explained: faster LLM inference
- LLM quantization explained: FP8, INT8, INT4, AWQ, GPTQ
- Continuous batching explained: how LLM servers scale
- LLM inference metrics: TTFT, TPOT, ITL, and throughput
- KV cache explained: the formula, a diagram, and a memory example
- vLLM tutorial: serve your first model with Docker
- vLLM vs SGLang: a measured comparison and a fair benchmark plan