World Congress 2025 • Aug 20, 2025 • Session details

Self-Hosted LLMs: From Zero to Inference

Cedric Clyburn , Roberto Carratalá

Why risk data privacy with third-party APIs when you can run models natively? Discover how to deploy, quantize, and scale self-hosted LLMs for completely secure, offline development.

Self-Hosted LLMs: From Zero to Inference thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

1:56 min

Running local open source models using Ollama

Aarno Aukia · LIVE

54 sec

Running generative AI models in local environments

Cedric Clyburn

Cedric Clyburn · World Congress 2024

3:14 min

Building a community-governed LAMP stack for open AI

Raffi Krikorian

Raffi Krikorian · World Congress 2026 Europe

1:24 min

Exploring open source inference tools for language model serving

Legare Kerrison

Legare Kerrison +1 · World Congress 2026 North America

1:45 min

Introduction to serving large language models locally

Patrick Koss

Patrick Koss · World Congress 2025

1:39 min

Addressing data privacy concerns with local language models

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing · AWS

Behind the Code: How Women Are Powering the Future of AI thumbnail

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel +2

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe +3

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler +3

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher +3

AI Sovereignty: What Does It Take? thumbnail

AI Sovereignty: What Does It Take?

Boris Hecker +3

Upcoming sessions on this topic

Decision models for developers who use LLMs

Anurag Bhandari

antelligent

CTO

Build a Zero-Cost AI Agent in Your Browser

Aishwarya Mathuria

Adobe

Computer Scientist
Nishant Thakur

Adobe

Senior Computer Scientist

Bypassing the GPU Tax: Smart Layer-7 Routing with Small Language Models (SLMs)

Sagar Utekar

CrowdStrike

SRE

Recursive Language Models: Scaling infinitely beyond the Context Window

Jayita Bhattacharyya

Domyn

Member of Technical Staff

Enterprise AI Memory at Scale

Aditya Joshi

Walmart

Senior Software Engineer

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

Purnima Kumari

DigitalOCean

Sr. Solutions Architect II

Related videos

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization thumbnail

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison +1

World Congress North America · 2026

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl · Microsoft

World Congress Europe · 2026

Efficient deployment and inference of GPU-accelerated LLMs thumbnail

Efficient deployment and inference of GPU-accelerated LLMs

World Congress · 2024

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

LIVE · Oct 2024

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss · STACKIT

World Congress · 2025

Building APIs in the AI Era thumbnail

Building APIs in the AI Era

Hugo Guerrero · Red Hat

World Congress · 2024

Related articles

LM

Luis Minvielle

What Are Large Language Models?

Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of...

about 3 years ago

What Are Large Language Models?

BB

Benedikt Bischof

MLOps And AI Driven Development

Welcome to this issue of the WeAreDevelopers Dev Talk Recap series. This article recaps an interesting talk by Natalie Pistunovic who spoke about the development of AI and MLOps. What you will lear...

over 4 years ago

MLOps And AI Driven Development

BB

Benedikt Bischof

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Bas Geerdink who gave advice on MLOps.‍About the speaker:‍Bas is a programmer, scientist, a...

about 4 years ago

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

DC

Daniel Cranney

A 5-Step Open-Source Setup for Agentic Engineering

If you’re a user of GitHub Copilot, then you’re likely already aware that GitHub recently switched all Copilot plans over to usage-based billing . Every interation with Copilot now burns credits by...

5 months ago

A laptop showing a leaderboard of LLMs

From learning to earning

Jobs that call for the skills explored in this talk.

about 1 month ago

Machine Learning Engineer

Valladolid, Spain

Remote Twilio AutoGen Pytorch

about 1 month ago • Verified

AI Software Engineer (Germany)

Berlin, Germany

Expert €70k–100k Remote REST TypeScript AI Frameworks

Machine Learning Engineer

Valencia, Spain

Remote Twilio AutoGen Pytorch

Machine Learning Engineer

Cantabria, Spain

Remote Twilio Chatbots Tensorflow

about 1 month ago

Engineering Manager, Google Cloud AI/ML

Portland, OR, United States

Expert $99k–232k LangChain Google Cloud Generative AI

5 days ago • Verified

Senior Data & AI Architect

reeeliance IM GmbH

Berlin, Germany

Expert Remote Python Microsoft SQL Server Cloud (AWS/Google/Azure)