The State of Local AI in 2026

Cloud companies spent years getting enterprise to go off-premise; now we're going to spend years getting them to go on-premise. This talk provides a practical snapshot of the local AI landscape and what building with on-device AI means for developers in 2026. We'll survey the current ecosystem, from open-weight models and model architectures to runtimes, inference engines, quantization techniques, and deployment options. We'll compare today's leading on-device models, discuss which workloads are best suited for local inference, and examine how developers are combining local and cloud models into hybrid systems. We'll also explore the broader trends shaping the ecosystem: the rise of open weights, improvements in model quality and efficiency, and the growing importance of AI sovereignty. We'll also discuss where local AI still falls short, including hardware constraints, operational complexity, and the workloads where cloud inference continues to be the better choice. Whether participants are early in their exploration of local models or already deeply embedded with on-device AI, this session will provide an up-to-date map of the ecosystem, explain the major technologies driving it forward, and offer practical guidance for choosing the right tools and architectures for their next project.