WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2023 • Oct 6, 2023

How building an industry DBMS differs from building a research one

Markus Dreseler

Building a planetary-scale DBMS requires radically different engineering than academic research. Learn why defensive programming, petabyte-scale telemetry, and obsessive reliability beat rapid prototyping in the real world.

Pause
Mute Enter Fullscreen
#1 about 3 min

Building a research database prototype from scratch

Developing an end-to-end open-source in-memory database enables unobstructed academic experimentation.

#2 about 4 min

Understanding decoupled compute and central storage architecture

Separating central storage from an independent compute layer scales resources efficiently without hardware constraints.

#3 about 3 min

Comparing textbook query planning to industry reality

Examining how commercial databases parse, optimize, and execute logical query plans mirrors academic prototypes.

#4 about 4 min

Implementing complex customer requirements and obscure features

Handling collations, evolving time zones, and niche operations like match recognize introduces significant engineering overhead.

#5 about 6 min

Leveraging telemetry to target query performance improvements

Running widespread execution profilers extracts actionable production trends instead of relying on artificial benchmarks.

#6 about 4 min

Implementing extensive automated testing for query correctness

Guaranteeing result consistency requires continuous static analysis, query permutation, and historical query re-execution checks.

#7 about 3 min

Safeguarding code deployments via granular parameter protection

Isolating new code paths with internal feature parameters mitigates release rollbacks and enables progressive rollouts.

#8 about 5 min

Managing query edge cases and hardware failures

Rotating engineers onto support reveals nondeterministic queries, distributed race conditions, and hidden hardware degradation.

#9 about 3 min

Reconciling development speed with huge operational impact

While rigorous safety mechanisms prevent quick iteration, operating at massive scale compounds the value of optimizations.

Matching moments

2:36 min

Modern application stacks and real-time data requirements

Tim Faulkes · LIVE

7:52 min

Audience questions on database performance, deployments, and data migrations

Gregor Bauer Gregor Bauer · LIVE

2:14 min

Solving complex platform architecture challenges at an enterprise scale

Maria Apazoglou · Coffee With Developers

3:16 min

Q&A on database vendor lock-in and alternative architectural choices

George Asafev · World Congress 2023

3:29 min

Database infrastructure and tech industry history

10:58 min

Exploring query boundaries, data storage, and architecture limits

Denis Washington +1 · World Congress 2021

More from World Congress 2023

The year 3000, a brief history of Web Development thumbnail
25:30
over 2 years ago

The year 3000, a brief history of Web Development

Lorenzo Pieri

Demografie nutzen um jedem Bewerber das passende Angebot zu machen! thumbnail
8:52
over 2 years ago

Demografie nutzen um jedem Bewerber das passende Angebot zu machen!

Nathalie Gierth

Guided Memory Management: Rust's Ownership Model thumbnail
0:18
almost 3 years ago

Guided Memory Management: Rust's Ownership Model

Stefan Baumgartner

Passwordless future: WebAuthn and Passkeys in practice thumbnail
32:32
almost 3 years ago

Passwordless future: WebAuthn and Passkeys in practice

Clemens Hübner

External Secrets Operator: the secrets management toolbox for self-sufficient teams thumbnail
30:07
almost 3 years ago

External Secrets Operator: the secrets management toolbox for self-sufficient teams

Moritz Johner

Building Reliable Serverless Applications with AWS CDK and Testing thumbnail
25:18
almost 3 years ago

Building Reliable Serverless Applications with AWS CDK and Testing

Raphael Manke

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 3

Real-Time Data Platforms at Trillion-Event Scale

Diptamay Sanyal

Principal Engineer | Data, AI & Cybersecurity Platforms

Diptamay Sanyal
Open session

World Congress 2026 North America

September 24, 2026 · 17:30–18:00

Stage 4

Boring Failover: Predictable Region Recovery Across 5,000 Microservices

Garvit Kataria, Sahil Sabharwal

Garvit Kataria
Sahil Sabharwal
Open session

World Congress 2026 North America

September 24, 2026 · 12:50–13:20

Stage 3

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Senior Vice President of Research and Development

Wei Hu
Open session

World Congress 2026 North America

September 25, 2026 · 16:50–17:20

Stage 4

Engineering Moneyball: How We Benchmarked Google vs Meta

Jirka Bachel

Co-Founder & CEO

Jirka Bachel
Open session

World Congress 2026 North America

September 24, 2026 · 14:50–15:20

Stage 2

Databases in the Agent Era

Monica Sarbu

Founder and CEO of xata.io

Monica Sarbu
Open session

World Congress 2026 North America

September 25, 2026 · 11:40–12:10

Stage 9

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan

Related videos

Leveraging Real time data in FSIs thumbnail
41:51
LIVE • almost 3 years ago

Leveraging Real time data in FSIs

Tim Faulkes

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases thumbnail
32:46
World Congress 2024 • about 2 years ago

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases thumbnail
28:41
World Congress 2025 • about 1 year ago

Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases

Wei Hu

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client thumbnail
30:28
World Congress 2026 Europe • about 2 months ago

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client

Max Fischer, Michael O'Toole

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone thumbnail
27:27
World Congress 2026 Europe • about 2 months ago

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone

Daniël van Eeden, Mattias Jonsson

From Messy Queries to Scalable Systems - How Data Engineering actually works thumbnail
31:15
World Congress 2026 Europe • about 2 months ago

From Messy Queries to Scalable Systems - How Data Engineering actually works

Sandhya Menon

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.