WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
World Congress 2021 • Jun 30, 2021

Kafka Streams Microservices

Denis Washington , Olli Salonen

Replacing tightly coupled microservices with Kafka Streams guarantees continuous data availability. Learn to build resilient, event-sourced architectures by treating Kafka as primary persistent storage.

Pause
Mute Enter Fullscreen
#1 about 2 min

Overcoming integration challenges with Kafka Streams microservices

Designing a robust platform to integrate disconnected cloud systems without building monolithic data silos.

#2 about 4 min

Core concepts of Apache Kafka and topic topologies

Understanding the fundamental mechanics of distributed event logs, consumer offsets, and stream topology abstractions.

#3 about 6 min

Transitioning from synchronous microservices to event sourcing

Replacing direct commands and system querying with primary event streams to eliminate tight coupling.

#4 about 4 min

Designing stream topologies and managing system failure scenarios

Leveraging stored event logs to replay historical data and maintain data read availability during outages.

#5 about 4 min

Building a consistent product catalog stream data pipeline

Aggregating, cleaning, and enriching unreliable inbound event streams into an interactive full-text search view.

#6 about 9 min

Solving race conditions and distributed state with repartitioning

Restructuring Kafka topics and IDs to enforce synchronous property validations across concurrent backend processors.

#7 about 4 min

Automating system behavior across independent event data streams

Writing processor topologies that combine separate state entities to trigger automated actions without API orchestration.

#8 about 5 min

Operational complexities and performance optimization of stream applications

Overcoming deployment pitfalls arising from transactional properties and overriding shared configurations under heavy loads.

#9 about 2 min

Blending traditional relational databases with event driven streams

Hooking classic database engines up to log publisher connectors to simplify complex unique entity logic.

#10 about 11 min

Exploring query boundaries, data storage, and architecture limits

Clarifying the practical boundaries of local processing safety, testing frameworks, and scalable long-term message compaction.

Matching moments

2:00 min

Moving from traditional databases to decoupled event streaming

Gerard Klijs · LIVE

3:45 min

Reviewing core Apache Kafka architecture and distributed fundamentals

Kirill Kulikov · LIVE

4:30 min

Introducing data management and the shift to streaming

Mary Grygleski Mary Grygleski · LIVE

1:56 min

Overview of current stream processing frameworks

Hartmut Armbruster Hartmut Armbruster · World Congress 2026 Europe

1:47 min

Advantages of adding Kafka to streaming architecture

Developersteve · LIVE

5:03 min

Introduction to Apache Kafka as architecture glue

Developersteve · LIVE

More from World Congress 2021

Micro Frontends with Module Federation: Why and How? thumbnail
45:40
about 5 years ago

Micro Frontends with Module Federation: Why and How?

Manfred Steyer

Data Science on Software Data thumbnail
48:10
about 5 years ago

Data Science on Software Data

Markus Harrer

The journey from developer to devops - what i've learnt along the way thumbnail
42:44
about 5 years ago

The journey from developer to devops - what i've learnt along the way

Liam Hurrel, Alireza Chegini

How Data is Shaping our Games thumbnail
43:30
about 5 years ago

How Data is Shaping our Games

Johanna Pirker

Cross platform Augmented Reality development with React Native thumbnail
40:50
about 5 years ago

Cross platform Augmented Reality development with React Native

Vladimir Novick

How to Build a Monetization Strategy Based on User's in-app Behavior thumbnail
38:59
about 5 years ago

How to Build a Monetization Strategy Based on User's in-app Behavior

Natalie Sieradzki

Upcoming sessions on this topic

Open session

World Congress 2026 North America

September 24, 2026 · 16:50–17:20

Stage 9

From Guesswork to Governance: Data Contracts Bring API Discipline to Apache Kafka

Sandon Jacobs

Senior Developer Advocate at IBM

Sandon Jacobs
Open session

World Congress 2026 North America

September 25, 2026 · 12:55–13:25

Stage 3

Replay-Safe Architecture: Building Event-Driven Systems That Can Recover With Confidence

Ishan Shah

PayPal, Software Engineer | Distributed Systems, AI, and Platform Engineering

Ishan Shah
Open session

World Congress 2026 North America

September 24, 2026 · 14:10–14:40

Stage 3

Real-Time Data Platforms at Trillion-Event Scale

Diptamay Sanyal

Principal Engineer | Data, AI & Cybersecurity Platforms

Diptamay Sanyal
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Stage 7

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

September 25, 2026 · 09:00–09:30

Stage 1

Test Before Release, Enforce at Runtime: Governance for Tool-Using AI Agents

Sachin Gupta

Member of Technical Staff 2 at eBay

Sachin Gupta
Open session

World Congress 2026 North America

September 24, 2026 · 17:30–18:00

Stage 4

Boring Failover: Predictable Region Recovery Across 5,000 Microservices

Garvit Kataria, Sahil Sabharwal

Garvit Kataria
Sahil Sabharwal

Related videos

The Rise of Reactive Microservices thumbnail
46:24
World Congress 2022 • about 4 years ago

The Rise of Reactive Microservices

David Leitner

From event streaming to event sourcing 101 thumbnail
27:33
LIVE • almost 6 years ago

From event streaming to event sourcing 101

Gerard Klijs

Let's Get Started With Apache Kafka® for Python Developers thumbnail
38:50
LIVE • over 3 years ago

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

What If We've Been Scaling Stream Processing Wrong All Along? thumbnail
31:06
World Congress 2026 Europe • 2 months ago

What If We've Been Scaling Stream Processing Wrong All Along?

Hartmut Armbruster

Tips, Techniques, and Common Pitfalls Debugging Kafka thumbnail
54:29
LIVE • over 2 years ago

Tips, Techniques, and Common Pitfalls Debugging Kafka

Developersteve

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen thumbnail
21:08
Coffee With Developers • 6 months ago

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen

Lutz Heunkhen

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.