WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2023 • Oct 6, 2023

How building an industry DBMS differs from building a research one

Markus Dreseler

Building a planetary-scale DBMS requires radically different engineering than academic research. Learn why defensive programming, petabyte-scale telemetry, and obsessive reliability beat rapid prototyping in the real world.

Pause
Mute Enter Fullscreen
#1 about 3 min

Building a research database prototype from scratch

Developing an end-to-end open-source in-memory database enables unobstructed academic experimentation.

#2 about 4 min

Understanding decoupled compute and central storage architecture

Separating central storage from an independent compute layer scales resources efficiently without hardware constraints.

#3 about 3 min

Comparing textbook query planning to industry reality

Examining how commercial databases parse, optimize, and execute logical query plans mirrors academic prototypes.

#4 about 4 min

Implementing complex customer requirements and obscure features

Handling collations, evolving time zones, and niche operations like match recognize introduces significant engineering overhead.

#5 about 6 min

Leveraging telemetry to target query performance improvements

Running widespread execution profilers extracts actionable production trends instead of relying on artificial benchmarks.

#6 about 4 min

Implementing extensive automated testing for query correctness

Guaranteeing result consistency requires continuous static analysis, query permutation, and historical query re-execution checks.

#7 about 3 min

Safeguarding code deployments via granular parameter protection

Isolating new code paths with internal feature parameters mitigates release rollbacks and enables progressive rollouts.

#8 about 5 min

Managing query edge cases and hardware failures

Rotating engineers onto support reveals nondeterministic queries, distributed race conditions, and hidden hardware degradation.

#9 about 3 min

Reconciling development speed with huge operational impact

While rigorous safety mechanisms prevent quick iteration, operating at massive scale compounds the value of optimizations.

Matching moments

2:36 min

Modern application stacks and real-time data requirements

Tim Faulkes · LIVE

7:52 min

Audience questions on database performance, deployments, and data migrations

Gregor Bauer Gregor Bauer · LIVE

2:14 min

Solving complex platform architecture challenges at an enterprise scale

Maria Apazoglou · Coffee With Developers

3:16 min

Q&A on database vendor lock-in and alternative architectural choices

George Asafev · WWC 2023

10:58 min

Exploring query boundaries, data storage, and architecture limits

Denis Washington +1 · WWC 2021

1:50 min

Prioritizing technical trade-offs over widespread industry misconceptions

Josip Stuhli Josip Stuhli · Coffee With Developers

More from World Congress 2023

The year 3000, a brief history of Web Development thumbnail
25:30
over 2 years ago

The year 3000, a brief history of Web Development

Lorenzo Pieri

Demografie nutzen um jedem Bewerber das passende Angebot zu machen! thumbnail
8:52
over 2 years ago

Demografie nutzen um jedem Bewerber das passende Angebot zu machen!

Nathalie Gierth

Guided Memory Management: Rust's Ownership Model thumbnail
0:18
over 2 years ago

Guided Memory Management: Rust's Ownership Model

Stefan Baumgartner

Passwordless future: WebAuthn and Passkeys in practice thumbnail
32:32
over 2 years ago

Passwordless future: WebAuthn and Passkeys in practice

Clemens Hübner

External Secrets Operator: the secrets management toolbox for self-sufficient teams thumbnail
30:07
over 2 years ago

External Secrets Operator: the secrets management toolbox for self-sufficient teams

Moritz Johner

Building Reliable Serverless Applications with AWS CDK and Testing thumbnail
25:18
over 2 years ago

Building Reliable Serverless Applications with AWS CDK and Testing

Raphael Manke

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Boring Failover: Predictable Region Recovery Across 5,000 Microservices

Sahil Sabharwal, Garvit Kataria

Sahil Sabharwal
Garvit Kataria
Open session

World Congress 2026 North America

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Senior Vice President of Research and Development

Wei Hu
Open session

World Congress 2026 North America

Databases in the Agent Era

Monica Sarbu

Founder and CEO of xata.io

Monica Sarbu
Open session

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

Testing React Backends Like a Pro: Mocking Databases with SQLite

David Morris

Solution Architect

David Morris
Open session

World Congress 2026 North America

Making Science Larger, not just Faster

Yuval Dvir

Commercial Executive, SandboxAQ

Yuval Dvir

Related videos

Leveraging Real time data in FSIs thumbnail
41:51
LIVE • over 2 years ago

Leveraging Real time data in FSIs

Tim Faulkes

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases thumbnail
32:46
WWC 2024 • almost 2 years ago

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases thumbnail
28:41
WWC 2025 • 12 months ago

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client thumbnail
30:28
WWC Europe 2026 • about 1 month ago

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client

Max Fischer, Michael O'Toole

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone thumbnail
27:27
WWC Europe 2026 • about 1 month ago

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone

Daniël van Eeden, Mattias Jonsson

From Messy Queries to Scalable Systems - How Data Engineering actually works thumbnail
31:15
WWC Europe 2026 • about 1 month ago

From Messy Queries to Scalable Systems - How Data Engineering actually works

Sandhya Menon

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.