WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2026 Europe • Jul 10, 2026 • Session details

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client

Max Fischer , Michael O'Toole

Imagine migrating a 620TB active data warehouse without a single maintenance window. See how a custom proxy transpiles SQL mid-flight to swap compute engines invisibly beneath millions of queries.

Pause
Mute Enter Fullscreen
#1 about 2 min

Scaling data infrastructure for a rapidly growing fintech platform

Expanding datasets and customer transactions forced a fundamental reevaluation of legacy data architectures to support growing operational demands.

#2 about 2 min

Addressing data ingestion and cost challenges while scaling architectures

Expanding computational costs necessitated a shift from rigid daily batch jobs toward more frequent and flexible analytical ingestion.

#3 about 2 min

Decoupling storage and compute with open lakehouse architectures

Adopting standard table formats ensures independent scaling for data stores while letting teams dynamically select appropriate execution engines.

#4 about 3 min

Migrating moving data targets without disrupting active engineering teams

Abstracting the database connection behind a persistent proxy maintains application uptime during underlying infrastructure transitions.

#5 about 4 min

Routing queries and transpiling syntax dynamically through a proxy

Intercepting network traffic enables systems to safely modify connection hosts and translate execution commands directly within the request path.

#6 about 4 min

Demonstrating dynamic interactive querying across different execution engine backends

Transpilation workflows rewrite backend-specific functions automatically behind the scenes for identical queries submitted across divergent database engines.

#7 about 2 min

Sharing data seamlessly across multiple compute engines using custom syntax

Bespoke proxy commands actively align output records across different processing tools to eliminate complex intermediate persistence layers.

#8 about 3 min

Implementing proxy plugins to intercept schemas and database clones

Automating underlying configurations allows the proxy layer to silently enforce index modifications and rapidly scale basic table operations.

#9 about 2 min

Optimizing sequential data pipeline performance with intelligent query delays

Injecting conditional polling handlers at the routing tier blocks orchestration commands efficiently until dependent upstream facts resolve.

#10 about 3 min

Enhancing caching patterns and access controls via the proxy layer

Evaluating analytical queries by an abstract syntax tree unlocks precise cache hits regardless of textual differences or formatting.

#11 about 3 min

Extracting operational principles from processing four million daily incoming queries

Designing flexible interception protocols empowers vast application ecosystems to scale horizontally without introducing hard client reconfigurations.

#12 about 1 min

Integrating conversational algorithms directly into the query interception proxy

Owning the database communication tier directly supports implementing specialized conversational agents and automated query optimization metrics internally.

#13 about 6 min

Addressing runtime migration specifics and future open source framework ambitions

Audience questions probe analytical tokenization specifics and the logistical pacing behind migrating legacy schemas incrementally using automation agents.

Matching moments

1:21 min

Summary of decoupling analytical compute and storage

Matthias Niehoff Matthias Niehoff · WWC Europe 2026

4:56 min

Boosting developer productivity via consolidated converged database architectures

Wei Hu Wei Hu · WWC 2023

2:14 min

Solving complex platform architecture challenges at an enterprise scale

Maria Apazoglou · Coffee With Developers

2:11 min

Executing massive cloud network migrations while maintaining live systems

Dana Lawson Dana Lawson +1 · WWC Europe 2026

4:30 min

Modernizing queries using microservices and federated schemas

Florian Mair Florian Mair · WWC 2024

3:00 min

Designing data ingestion architecture with system integration

Eldert Grootenboer +1 · WWC 2023

More from World Congress 2026 Europe

Building the Nervous System of AI - Michael Kagan (NVIDIA) thumbnail
29:17
about 1 month ago

Building the Nervous System of AI - Michael Kagan (NVIDIA)

Michael Kagan, Sead Ahmetović

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote thumbnail
19:41
about 1 month ago

Five Things in Tech that Matter Now - WeAreDevelopers World Congress 2026 Closing Keynote

Christian Heilmann

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend thumbnail
32:06
about 1 month ago

The Sound of Your Secrets: Teaching Your Model to Spy, So You Can Learn to Defend

David vonThenen

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail
35:13
about 1 month ago

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬 thumbnail
24:23
about 1 month ago

Running Secure Life Science Research at Scale using Hybrid GPU HPC and Kubernetes 🧬

Jeremy Murray

Angstfreude - AI and Corporate Culture - The Thrill and the Threat thumbnail
30:26
about 1 month ago

Angstfreude - AI and Corporate Culture - The Thrill and the Threat

Axel Ebert, Karin Krobath

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Boring Failover: Predictable Region Recovery Across 5,000 Microservices

Sahil Sabharwal, Garvit Kataria

Sahil Sabharwal
Garvit Kataria
Open session

World Congress 2026 North America

The $0.15/month Algo-Trader: Architecting High-Performance Serverless Trading Engines

Vishwak Thatikonda

Lead Full Stack Engineer at SDH Systems

Vishwak Thatikonda
Open session

World Congress 2026 North America

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

When Agents Became Users: Rearchitecting Identity and Permissions for AI at Scale

Dor Cohen, Yoav Gal

Dor Cohen
Yoav Gal
Open session

World Congress 2026 North America

GitHub’s Team X-Ray: Your Repository Knows More About Your Team Than Your Team Does

Andrea Griffiths

Senior Developer Advocate

Andrea Griffiths
Open session

World Congress 2026 North America

Databases in the Agent Era

Monica Sarbu

Founder and CEO of xata.io

Monica Sarbu

Related videos

How building an industry DBMS differs from building a research one thumbnail
30:34
WWC 2023 • almost 3 years ago

How building an industry DBMS differs from building a research one

Markus Dreseler

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone thumbnail
27:27
WWC Europe 2026 • about 1 month ago

TiDB, One Layer at a Time: How Distributed SQL Became an Agentic AI Backbone

Daniël van Eeden, Mattias Jonsson

From Messy Queries to Scalable Systems - How Data Engineering actually works thumbnail
31:15
WWC Europe 2026 • about 1 month ago

From Messy Queries to Scalable Systems - How Data Engineering actually works

Sandhya Menon

Swapping Low Latency Data Storage Under High Load thumbnail
22:32
WWC 2023 • almost 3 years ago

Swapping Low Latency Data Storage Under High Load

George Asafev

Agentic employees in world's most downloaded FinTech app thumbnail
30:37
WWC Europe 2026 • about 1 month ago

Agentic employees in world's most downloaded FinTech app

Sasa Fajkovic

How to Destroy a Monolith? thumbnail
26:45
WWC 2025 • 12 months ago

How to Destroy a Monolith?

Babette Wagner

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.