Alex Soto

Practical Change Data Streaming Use Cases With Debezium And Quarkus

Stop using risky dual writes. Learn how Debezium uses your database's transaction log to reliably stream every change and guarantee data consistency across your services.

Practical Change Data Streaming Use Cases With Debezium And Quarkus
#1about 3 minutes

Introduction to change data capture with Debezium

An overview of how change data capture (CDC) with Debezium and Quarkus can solve the problem of dual writes in microservices.

#2about 4 minutes

The challenge of data consistency with dual writes

Dual writes to multiple databases or services can lead to data inconsistencies when one of the writes fails.

#3about 6 minutes

Core concepts of Apache Kafka for event streaming

Apache Kafka is a fault-tolerant, scalable, publish-subscribe system designed for real-time event stream processing.

#4about 4 minutes

How change data capture (CDC) works

Change data capture automatically captures database changes like inserts, updates, and deletes and streams them as events.

#5about 5 minutes

Using Debezium for transaction log-based CDC

Debezium is a Kafka connector that taps into database transaction logs to reliably capture and propagate data changes.

#6about 2 minutes

The structure of a Debezium change event message

Debezium change events are JSON messages containing before and after states of the data, plus metadata about the operation.

#7about 5 minutes

Solving dual writes with the transactional outbox pattern

The outbox pattern ensures data consistency by writing business data and an event to an outbox table within a single database transaction.

#8about 5 minutes

Migrating monoliths with the strangler fig pattern

The strangler fig pattern uses CDC to replicate data from a monolith to a new microservice, enabling a gradual and safe migration.

#9about 3 minutes

Implementing the outbox pattern with Quarkus and Kubernetes

Use Quarkus to implement the outbox pattern and deploy the entire system, including Kafka managed by Strimzi, on Kubernetes.

#10about 6 minutes

Live demo of Debezium capturing database changes

A practical demonstration shows how inserting data into a database table automatically triggers Debezium to publish a change event to a Kafka topic.

#11about 10 minutes

Q&A on CDC implementation and operational challenges

Discussion covers the challenges of building a custom CDC solution, Debezium's fault tolerance, and handling lost transaction logs.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

A traditional approach to streaming with Kafka and Debezium
05:20 MIN

A traditional approach to streaming with Kafka and Debezium

Python-Based Data Streaming Pipelines Within Minutes

Using change data capture for real-time alerts
07:01 MIN

Using change data capture for real-time alerts

From event streaming to event sourcing 101

Implementing a CQRS banking demo with Kafka
16:38 MIN

Implementing a CQRS banking demo with Kafka

From event streaming to event sourcing 101

Decoupling microservices with event streams
10:34 MIN

Decoupling microservices with event streams

From event streaming to event sourcing 101

Managing data consistency with change data capture
09:57 MIN

Managing data consistency with change data capture

Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure

Real-world Kafka use cases at scale
05:04 MIN

Real-world Kafka use cases at scale

Tips, Techniques, and Common Pitfalls Debugging Kafka

Understanding Kafka's role in modern architectures
03:40 MIN

Understanding Kafka's role in modern architectures

Tips, Techniques, and Common Pitfalls Debugging Kafka

Decomposing monoliths with change data capture
32:43 MIN

Decomposing monoliths with change data capture

The Rise of Reactive Microservices

Featured Partners

Related Videos

See all videos
Quarkus. A Bliss for developers44:40

Quarkus. A Bliss for developers

Alex Soto

From event streaming to event sourcing 10127:33

From event streaming to event sourcing 101

Gerard Klijs

Let's Get Started With Apache Kafka® for Python Developers38:50

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Kafka Streams Microservices45:48

Kafka Streams Microservices

Denis Washington & Olli Salonen

Development of reactive applications with Quarkus39:10

Development of reactive applications with Quarkus

Niklas Heidloff

Why and when should we consider Stream Processing frameworks in our solutions30:51

Why and when should we consider Stream Processing frameworks in our solutions

Soroosh Khodami

Don't Change the Partition Count for Kafka Topics!24:53

Don't Change the Partition Count for Kafka Topics!

Dainius Jocas

Developer Joy with Quarkus28:21

Developer Joy with Quarkus

Daniel Oh

From learning to earning

Jobs that call for the skills explored in this talk.