LLM Local Inference: Tools, Techniques, and Insights

Over the past few months, I’ve had the chance to work on a research and development project exploring the feasibility of adopting solutions powered by Small and Large Language Models—with one unique twist: inference had to be performed at the edge.

This journey led me to scratch the surface of the vast, fascinating, and rapidly evolving world of LLMs, with a particular focus on inference servers. Along the way, I encountered a fair share of head-scratching moments and valuable insights that I’m excited to share with you.

In this talk, we’ll cover the key concepts behind LLM inference, untangle the tricky jargon, and give you a glimpse into the primary tools and solutions you can leverage if you ever need or want to explore LLM local inference.