Faster Together: Train and Deploy a Speculative Decoding Model for Low-Latency LLM Inference

Dive deep into theory and practice of low-latency inference by deploying NVIDIA TensorRT-LLM with advanced speculative decoding techniques. You’ll train an Eagle-3 draft head to propose candidate tokens efficiently, serve it, and benchmark it using AIPerf to quantify how these strategies minimize latency.

Workshop Preparation: - Please bring your own laptop. - Please review the following document and prepare accordingly before the workshop: https://developer.nvidia.com/dli/getready