Alex Soto
Practical Change Data Streaming Use Cases With Debezium And Quarkus
#1about 3 minutes
Introduction to change data capture with Debezium
An overview of how change data capture (CDC) with Debezium and Quarkus can solve the problem of dual writes in microservices.
#2about 4 minutes
The challenge of data consistency with dual writes
Dual writes to multiple databases or services can lead to data inconsistencies when one of the writes fails.
#3about 6 minutes
Core concepts of Apache Kafka for event streaming
Apache Kafka is a fault-tolerant, scalable, publish-subscribe system designed for real-time event stream processing.
#4about 4 minutes
How change data capture (CDC) works
Change data capture automatically captures database changes like inserts, updates, and deletes and streams them as events.
#5about 5 minutes
Using Debezium for transaction log-based CDC
Debezium is a Kafka connector that taps into database transaction logs to reliably capture and propagate data changes.
#6about 2 minutes
The structure of a Debezium change event message
Debezium change events are JSON messages containing before and after states of the data, plus metadata about the operation.
#7about 5 minutes
Solving dual writes with the transactional outbox pattern
The outbox pattern ensures data consistency by writing business data and an event to an outbox table within a single database transaction.
#8about 5 minutes
Migrating monoliths with the strangler fig pattern
The strangler fig pattern uses CDC to replicate data from a monolith to a new microservice, enabling a gradual and safe migration.
#9about 3 minutes
Implementing the outbox pattern with Quarkus and Kubernetes
Use Quarkus to implement the outbox pattern and deploy the entire system, including Kafka managed by Strimzi, on Kubernetes.
#10about 6 minutes
Live demo of Debezium capturing database changes
A practical demonstration shows how inserting data into a database table automatically triggers Debezium to publish a change event to a Kafka topic.
#11about 10 minutes
Q&A on CDC implementation and operational challenges
Discussion covers the challenges of building a custom CDC solution, Debezium's fault tolerance, and handling lost transaction logs.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
04:23 MIN
A traditional approach to streaming with Kafka and Debezium
Python-Based Data Streaming Pipelines Within Minutes
01:34 MIN
Managing data consistency with change data capture
Software Engineering Social Connection: Yubo’s lean approach to scaling an 80M-user infrastructure
03:33 MIN
Using change data capture for real-time alerts
From event streaming to event sourcing 101
22:41 MIN
Answering questions on Kafka use cases, careers, and learning
Let's Get Started With Apache Kafka® for Python Developers
04:50 MIN
Implementing a CQRS banking demo with Kafka
From event streaming to event sourcing 101
03:41 MIN
Decoupling microservices with event streams
From event streaming to event sourcing 101
12:25 MIN
Evolving the architecture with a hybrid database approach
Kafka Streams Microservices
05:40 MIN
Evolving from classic microservices to event-driven design
Kafka Streams Microservices
Featured Partners
Related Videos
44:40Quarkus. A Bliss for developers
Alex Soto
27:33From event streaming to event sourcing 101
Gerard Klijs
45:48Kafka Streams Microservices
Denis Washington & Olli Salonen
38:50Let's Get Started With Apache Kafka® for Python Developers
Lucia Cerchie
24:53Don't Change the Partition Count for Kafka Topics!
Dainius Jocas
28:21Developer Joy with Quarkus
Daniel Oh
45:49Databases on Kubernetes: Why you should care
Denis Wilson Souza Rosa & Steffen Schneider
30:51Why and when should we consider Stream Processing frameworks in our solutions
Soroosh Khodami
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
Red Bull Media House GmbH
Elsbethen, Austria
Intermediate
Java
NoSQL
Docker
Angular
Hibernate
+6
AUTO1 Group SE
Berlin, Germany
Intermediate
Senior
ELK
Terraform
Elasticsearch
Callista Group
Basel, Switzerland
Senior
Apache Kafka
Microservices
Infosys Limited
Ansible
Kubernetes
Apache Kafka
Microservices
Akros Ag
Junior
Docker
Jenkins
Openshift
Kubernetes
Microservices
+1