WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2025 • Aug 20, 2025 • Session details

Self-Hosted LLMs: From Zero to Inference

Cedric Clyburn , Roberto Carratalá

Why risk data privacy with third-party APIs when you can run models natively? Discover how to deploy, quantize, and scale self-hosted LLMs for completely secure, offline development.

Pause
Mute Enter Fullscreen
#1 about 3 min

The growing trend of self-hosting AI models

Why developers are increasingly opting to self-host language models to reduce reliance on third-party web services.

#2 about 2 min

Why developers should run AI models locally

How self-hosting resolves organizational data privacy concerns and accelerates the inner loop of local application development.

#3 about 3 min

Open source tools for running and scaling models

How container-based execution engines assist in scaling and inferencing massive language models across varied hardware constraints.

#4 about 3 min

Selecting the right open source model for workloads

How to evaluate open source repository formats to locate models perfectly fine-tuned for specialized reasoning, language interpolation, or multimodal pipelines.

#5 about 3 min

Reducing memory footprints through AI model quantization

Using quantization techniques to compress active weighting constants into lower precision scales for highly performant execution on consumer components.

#6 about 2 min

Integrating local AI models with existing business data

How tightly coupling locally hosted open source models directly with active codebase logic and documentation solves generic contextual boundaries.

#7 about 3 min

Running an AI model locally using Podman AI Lab

How to pull and serve an instruct-tuned conversational model via an offline desktop containerized testing playground.

#8 about 4 min

Building local RAG architectures using the Anything LLM tool

How intertwining a lightweight document vector database with a locally served language model wholly eliminates false inferences during queries.

#9 about 5 min

Setting up a local AI code assistant workspace

How configuring an open source code extension rapidly generates functional Python endpoints using entirely private offline compute constraints.

#10 about 5 min

Developing agentic AI applications using Model Context Protocol

How implementing systemic communication protocols alongside a Python execution framework equips local models to resolve deep external calculation tasks.

#11 about 1 min

Replacing commercial AI APIs with self-hosted open source models

How exchanging proprietary cloud APIs for strictly self-hosted containerized infrastructure yields massive deployment flexibility and vendor autonomy.

Matching moments

1:56 min

Running local open source models using Ollama

Aarno Aukia · LIVE

54 sec

Running generative AI models in local environments

Cedric Clyburn Cedric Clyburn · World Congress 2024

3:14 min

Building a community-governed LAMP stack for open AI

Raffi Krikorian Raffi Krikorian · World Congress 2026 Europe

1:24 min

Exploring open source inference tools for language model serving

Legare Kerrison Legare Kerrison +1 · World Congress 2026 North America

1:45 min

Introduction to serving large language models locally

Patrick Koss Patrick Koss · World Congress 2025

1:39 min

Addressing data privacy concerns with local language models

More from World Congress 2025

GenAI Security: Navigating the Unseen Iceberg thumbnail
24:33
12 months ago

GenAI Security: Navigating the Unseen Iceberg

Maish Saidel-Keesing

Behind the Code: How Women Are Powering the Future of AI thumbnail
30:28
about 1 year ago

Behind the Code: How Women Are Powering the Future of AI

Alexandra Wudel, Laura Moritz, Madalina Florean

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise? thumbnail
28:12
about 1 year ago

The AI Hype Filter: What’s Real, What’s Investable, What’s Noise?

Justus Lumpe, Konstantin Vinogradov, Nils Eiteneyer, Stephan Wirris

AI in Leadership: How Technology is Reshaping Executive Roles thumbnail
28:19
about 1 year ago

AI in Leadership: How Technology is Reshaping Executive Roles

Bernd Datler, Jasmin Kaiser, Jeff Hausmann, Sonja Alvarez

The AI-Ready Stack: Rethinking the Engineering Org of the Future thumbnail
28:09
about 1 year ago

The AI-Ready Stack: Rethinking the Engineering Org of the Future

Alex Laubscher, Jan Oberhauser, Keno Dreßel, Mirko Novakovic

AI Sovereignty: What Does It Take? thumbnail
29:13
about 1 year ago

AI Sovereignty: What Does It Take?

Boris Hecker, Clemens Wasner, Robin Hermann, Sonja Alvarez

Upcoming sessions on this topic

Open session

Build a Zero-Cost AI Agent in Your Browser

  • Aishwarya Mathuria

    Adobe

    Computer Scientist

  • Nishant Thakur

    Adobe

    Senior Computer Scientist

Open session

How 8 GPUs Became 16: Rethinking LLM Inference Through Parallelism, Memory, and Runtime Design

  • Purnima Kumari

    DigitalOCean

    Sr. Solutions Architect II

Open session

Beyond File Dumps: Context Engineering for Coding Agents

  • Animesh Dutta

    Arm

    Senior Software Engineer

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

Teaching AI Coding Agents to Build It Right the First Time

  • Sajeetharan Sinnathurai

    Microsoft

    Principal Product Manager

Open session

Agentic Forecasting with sktime: Towards Autonomous Time Series Analysis

  • Franz Kiraly

    sktime

    Director

Related videos

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization thumbnail
26:27
World Congress 2026 North America • 22 days ago

Fast, Cheap, and Accurate: Optimizing LLM Inference with vLLM and Quantization

Legare Kerrison, Cedric Clyburn

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
World Congress 2026 Europe • 3 months ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
World Congress 2024 • about 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21
LIVE • about 2 years ago

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Unveiling the Magic: Scaling Large Language Models to Serve Millions thumbnail
31:25
World Congress 2025 • about 1 year ago

Unveiling the Magic: Scaling Large Language Models to Serve Millions

Patrick Koss

Building APIs in the AI Era thumbnail
32:28
World Congress 2024 • about 2 years ago

Building APIs in the AI Era

Hugo Guerrero

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.