WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 Europe • Jul 9, 2026 • Session details

Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source

Viktoria Semaan

Relying on proprietary LLMs is like hiring a PhD for an intern's task. Learn how fine-tuning compact open-source models maintains top-tier accuracy while slashing production costs by 86%.

Pause
Mute Enter Fullscreen
#1 about 2 min

The shift toward token efficiency and independent model layers

Decoupling the application layer from the model serving layer mitigates risks caused by the short life cycle of commercial models.

#2 about 2 min

Cursor case study on solving high task costs

Fine-tuning an open weight model explicitly for coding enabled an eighty-six percent reduction in token usage.

#3 about 3 min

Defining an AI moderation use case for evaluation setup

Establishing message classification and human review routing tasks demonstrates practical framework assessments.

#4 about 4 min

Trade-offs between proprietary, open weight, and fine-tuned models

Navigating model selection requires balancing organizational control, hosting responsibilities, and specific task complexities.

#5 about 5 min

Programmatic model evaluation and custom metrics via MLflow

Using an LLM as a judge alongside clear guidelines within MLflow creates an automated baseline for continuous improvement.

#6 about 3 min

Managing traffic and tracking costs with Databricks Unity Catalog

Implementing a centralized gateway controls API spending and enables intelligent traffic splitting between different parameter sizes.

#7 about 2 min

Analyzing evaluation dashboard results for optimal model selection

Comparing latency, benchmark accuracy, and operational costs reveals cases where smaller open-source models outclass proprietary peers.

#8 about 2 min

Identifying when narrow tasks require custom model fine-tuning

Determining when to transition from prompt engineering and retrieval-augmented generation to supervised fine-tuning reduces latency and API dependency.

#9 about 3 min

Executing LoRA fine-tuning using serverless Databricks AI runtimes

Utilizing low-rank adaptation on serverless GPUs minimizes infrastructure overhead while efficiently training smaller architectures like Qwen.

#10 about 2 min

Comparing resulting task performance across varied model classifications

While fine-tuned implementations dramatically lower classification costs, larger commercial deployments remain necessary for complex reasoning and context ranking.

#11 about 2 min

Key takeaways and accessing the Databricks developer toolkit

Adopting a standardized evaluation protocol prevents costly assumptions and is highly accessible via the provided community tier resources.

#12 about 2 min

Addressing system transparency when utilizing an LLM judge

Leveraging deep evaluation traces clarifies the logical steps and exact reasoning behind individual automated assessment scores.

Matching moments

3:53 min

Evaluating and observing large language model performance at scale

Krzysztof Cieślak Krzysztof Cieślak · World Congress 2024

3:26 min

Introducing LLMs as judges for automated testing

Sebastian Messingfeld Sebastian Messingfeld · World Congress 2026 Europe

2:36 min

Addressing AI observability and cost transparency with Langfuse

Hellmar Becker Hellmar Becker · World Congress 2026 Europe

3:22 min

Evaluating advanced artificial intelligence platforms for daily recruitment

Rudi Bauer Rudi Bauer +1 · Cappuccino with HR

1:37 min

Managing AI costs with open-source models

Laurie Voss Laurie Voss · Coffee With Developers

3:14 min

Balancing performance and costs with custom language models

Jemiah Sius Jemiah Sius · World Congress 2024

More from World Congress 2026 Europe

Building the Nervous System of AI - Michael Kagan (NVIDIA) thumbnail
29:17
3 months ago

Building the Nervous System of AI - Michael Kagan (NVIDIA)

Michael Kagan, Sead Ahmetović

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote thumbnail
19:41
3 months ago

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote

Christian Heilmann

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend thumbnail
32:06
3 months ago

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend

David vonThenen

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail
35:13
3 months ago

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail
24:23
3 months ago

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray

Angstfreude - AI and Corporate Culture - The Thrill and the Threat thumbnail
30:26
3 months ago

Angstfreude - AI and Corporate Culture - The Thrill and the Threat

Axel Ebert, Karin Krobath

Related videos

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated thumbnail
30:40
World Congress 2026 Europe • 3 months ago

Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated

Christin Pohl

Unlocking the Power of AI: Accessible Language Model Tuning for All thumbnail
31:50
World Congress 2024 • about 2 years ago

Unlocking the Power of AI: Accessible Language Model Tuning for All

Cedric Clyburn, @technicallylegare

AI That Fits Your Business, Not the Other Way Around thumbnail
25:56
World Congress 2026 Europe • 3 months ago

AI That Fits Your Business, Not the Other Way Around

Anshul Jindal, Cansu Kavili Örnek

Efficient deployment and inference of GPU-accelerated LLMs​ thumbnail
23:01
World Congress 2024 • about 2 years ago

Efficient deployment and inference of GPU-accelerated LLMs​

Adolf Hohl

Why LLMs Need Observability and How to Do It thumbnail
32:41
Europe 2026 Virtual • 3 months ago

Why LLMs Need Observability and How to Do It

Rahul Gupta

Localized Open Models in Production: What Builders Need to Know thumbnail
31:45
World Congress 2026 Europe • 3 months ago

Localized Open Models in Production: What Builders Need to Know

Ankit Patel, Pierre-Louis Cedoz, Jamie Madden, Stephen Batifol

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.