Alex Soto

Practical Change Data Streaming Use Cases With Debezium And Quarkus

Stop using risky dual writes. Learn how Debezium uses your database's transaction log to reliably stream every change and guarantee data consistency across your services.

Practical Change Data Streaming Use Cases With Debezium And Quarkus
#1about 3 minutes

Introduction to change data capture with Debezium

An overview of how change data capture (CDC) with Debezium and Quarkus can solve the problem of dual writes in microservices.

#2about 4 minutes

The challenge of data consistency with dual writes

Dual writes to multiple databases or services can lead to data inconsistencies when one of the writes fails.

#3about 6 minutes

Core concepts of Apache Kafka for event streaming

Apache Kafka is a fault-tolerant, scalable, publish-subscribe system designed for real-time event stream processing.

#4about 4 minutes

How change data capture (CDC) works

Change data capture automatically captures database changes like inserts, updates, and deletes and streams them as events.

#5about 5 minutes

Using Debezium for transaction log-based CDC

Debezium is a Kafka connector that taps into database transaction logs to reliably capture and propagate data changes.

#6about 2 minutes

The structure of a Debezium change event message

Debezium change events are JSON messages containing before and after states of the data, plus metadata about the operation.

#7about 5 minutes

Solving dual writes with the transactional outbox pattern

The outbox pattern ensures data consistency by writing business data and an event to an outbox table within a single database transaction.

#8about 5 minutes

Migrating monoliths with the strangler fig pattern

The strangler fig pattern uses CDC to replicate data from a monolith to a new microservice, enabling a gradual and safe migration.

#9about 3 minutes

Implementing the outbox pattern with Quarkus and Kubernetes

Use Quarkus to implement the outbox pattern and deploy the entire system, including Kafka managed by Strimzi, on Kubernetes.

#10about 6 minutes

Live demo of Debezium capturing database changes

A practical demonstration shows how inserting data into a database table automatically triggers Debezium to publish a change event to a Kafka topic.

#11about 10 minutes

Q&A on CDC implementation and operational challenges

Discussion covers the challenges of building a custom CDC solution, Debezium's fault tolerance, and handling lost transaction logs.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

A traditional approach to streaming with Kafka and Debezium
04:23 MIN

A traditional approach to streaming with Kafka and Debezium

Python-Based Data Streaming Pipelines Within Minutes

Managing data consistency with change data capture
01:34 MIN

Managing data consistency with change data capture

Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure

Using change data capture for real-time alerts
03:33 MIN

Using change data capture for real-time alerts

From event streaming to event sourcing 101

Answering questions on Kafka use cases, careers, and learning
22:41 MIN

Answering questions on Kafka use cases, careers, and learning

Let's Get Started With Apache Kafka® for Python Developers

Implementing a CQRS banking demo with Kafka
04:50 MIN

Implementing a CQRS banking demo with Kafka

From event streaming to event sourcing 101

Decoupling microservices with event streams
03:41 MIN

Decoupling microservices with event streams

From event streaming to event sourcing 101

Evolving the architecture with a hybrid database approach
12:25 MIN

Evolving the architecture with a hybrid database approach

Kafka Streams Microservices

Evolving from classic microservices to event-driven design
05:40 MIN

Evolving from classic microservices to event-driven design

Kafka Streams Microservices

Featured Partners

Related Videos

See all videos
Event Messaging and Streaming with Apache Pulsar57:19

Event Messaging and Streaming with Apache Pulsar

Mary Grygleski

Quarkus. A Bliss for developers44:40

Quarkus. A Bliss for developers

Alex Soto

Let's Get Started With Apache Kafka® for Python Developers38:50

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Development of reactive applications with Quarkus39:10

Development of reactive applications with Quarkus

Niklas Heidloff

Application Modernization and Rabbits33:40

Application Modernization and Rabbits

Niklas Heidloff

Cloud Chaos and Microservices Mayhem39:16

Cloud Chaos and Microservices Mayhem

Holly Cummins

2021: Familiar APIs on Kickass Runtimes #slideless37:55

2021: Familiar APIs on Kickass Runtimes #slideless

Adam Bien

How to Benchmark Your Apache Kafka35:27

How to Benchmark Your Apache Kafka

Kirill Kulikov

Related Articles

View all articles
CH
Chris Heilmann
Dev Digest 134 - Where pixels sing?
News and ArticlesWeAreDevelopers LIVE Data and Security Day is on Wednesday, 25/09/2024. Learn about OPC UA Updates, Best Practices for Using GitHub Secrets, Passwordless Web 1.5, Emerging AI Security Risks, Data Privacy in LLMs and get a chance to t...
Dev Digest 134 - Where pixels sing?

From learning to earning

Jobs that call for the skills explored in this talk.