WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2023 • Oct 6, 2023

Swapping Low Latency Data Storage Under High Load

George Asafev

How do you migrate 140 million accounts at 100,000 requests per second with zero downtime? Uncover the DynamoDB strategy Miro used to slash core latency by 4x.

Pause
Mute Enter Fullscreen
#1 about 3 min

Migrating millions of user accounts at scale without downtime

An overview of the challenge to orchestrate a seamless migration of over 140 million user profiles out of Redis in a high-load environment.

#2 about 3 min

Evaluating data storage limitations and monolithic architectural challenges

Why adopting microservices resolves the difficult deployment processes and unclear responsibility borders that complicate a monolithic setup.

#3 about 3 min

Comparing PostgreSQL and DynamoDB for read-heavy storage

Evaluating the trade-offs in speed and cost when deciding between a relational database and a fully-managed NoSQL solution for specific query patterns.

#4 about 1 min

Accelerating DynamoDB performance with DAX memory caching

Utilizing an in-memory cache accelerator layer to push read query times down from milliseconds to microseconds.

#5 about 3 min

Load testing production traffic for database selection

Simulating high-load scenarios using synthetic traffic tools to confidently guide the core database infrastructure decision.

#6 about 2 min

Navigating the limitations of DynamoDB and DAX caching

Working around access pattern differences, cache invalidation delays, and rigid vertical scaling constraints when provisioning clusters.

#7 about 3 min

Executing a zero-downtime database migration with dual writes

Utilizing sequential queries and protected background data inserts to migrate systems securely while avoiding race conditions.

#8 about 2 min

Validating data consistency through asynchronous shadow reads

Protecting data integrity during switchovers by conducting offline evaluations matched against live interactions with the original store.

#9 about 1 min

Managing gradual rollouts and internal testing for UI impact

Minimizing external disruption by internally testing features to prevent eventual consistency logic from degrading application interfaces.

#10 about 2 min

Troubleshooting connection throttling and hidden CPU spikes in DAX

Diagnosing undocumented processing surges caused by rapid request influxes that initially bypassed standard system observability thresholds.

#11 about 2 min

Resolving DAX replication lag and measuring final performance improvements

Understanding cache performance degradation caused by AWS synchronization delays when manually expanding node counts.

#12 about 4 min

Q&A on database vendor lock-in and alternative architectural choices

Addressing long-term engineering choices, potential vendor constraints, and evaluating in-memory cluster alternatives like Elasticache.

Matching moments

8:01 min

Solving architectural challenges during rapid public cloud migrations

Liam Hurrel +1 · World Congress 2021

7:52 min

Audience questions on database performance, deployments, and data migrations

Gregor Bauer Gregor Bauer · LIVE

2:11 min

Executing massive cloud network migrations while maintaining live systems

Dana Lawson Dana Lawson +1 · World Congress 2026 Europe

3:07 min

The database outage that inspired the creation of Dynamo

Werner Vogels Werner Vogels · World Congress 2026 Europe

4:47 min

Final performance results and key architectural takeaways

Yedidya Schwartz · LIVE

1:49 min

Debugging serverless timeouts and replacing databases for lookups

Christoph Eicke Christoph Eicke · World Congress 2025

More from World Congress 2023

The year 3000, a brief history of Web Development thumbnail
25:30
over 2 years ago

The year 3000, a brief history of Web Development

Lorenzo Pieri

Demografie nutzen um jedem Bewerber das passende Angebot zu machen! thumbnail
8:52
almost 3 years ago

Demografie nutzen um jedem Bewerber das passende Angebot zu machen!

Nathalie Gierth

Guided Memory Management: Rust's Ownership Model thumbnail
0:18
almost 3 years ago

Guided Memory Management: Rust's Ownership Model

Stefan Baumgartner

Passwordless future: WebAuthn and Passkeys in practice thumbnail
32:32
almost 3 years ago

Passwordless future: WebAuthn and Passkeys in practice

Clemens Hübner

External Secrets Operator: the secrets management toolbox for self-sufficient teams thumbnail
30:07
almost 3 years ago

External Secrets Operator: the secrets management toolbox for self-sufficient teams

Moritz Johner

Building Reliable Serverless Applications with AWS CDK and Testing thumbnail
25:18
almost 3 years ago

Building Reliable Serverless Applications with AWS CDK and Testing

Raphael Manke

Related videos

How to Destroy a Monolith? thumbnail
26:45
World Congress 2025 • about 1 year ago

How to Destroy a Monolith?

Babette Wagner

Scaling: from 0 to 20 million users thumbnail
28:12
World Congress 2023 • about 3 years ago

Scaling: from 0 to 20 million users

Josip Stuhli

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes thumbnail
35:13
World Congress 2026 Europe • 3 months ago

Super scaling for the Super Bowl: How to survive 30 million users hitting your backend in 30 minutes

Irina Branovic

AWS Migration within 3 months thumbnail
37:08
World Congress 2021 • over 5 years ago

AWS Migration within 3 months

Steffen Heilmann

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client thumbnail
30:28
World Congress 2026 Europe • 3 months ago

Swapping a Data Warehouse at Runtime: Zero-Downtime Migration Without Changing a Single Client

Max Fischer, Michael O'Toole

Reliable scalability: How Amazon.com scales on AWS thumbnail
22:24
World Congress 2024 • about 2 years ago

Reliable scalability: How Amazon.com scales on AWS

Florian Mair

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.