Kafka Streams Microservices

Denis Washington & Olli Salonen - 4 years ago

We describe a microservice-based platform built fully on Kafka Streams that we designed and built for our client - a global German brand -, and on top of which we built a CRUD-centric B2B inventory application. First, we take the audience through the evolution of the design during the concept phase, explaining how we went from the client problem to the solution. We explain our decision to make Kafka our system’s single source of truth and its interesting consequences. Via several examples, the talk walks step-by-step through patterns we applied using Streams, such as Event Sourcing to persist history, Command and Query Responsibility Segregation (CQRS) via command streams and materialized views, and integration with secondary data stores for full-text search and other use cases. There are musings about data normalization vs. denormalization in Kafka Streams, and what it could mean for GDPR considerations. Furthermore, we use examples from our work to propose how traditional APIs could be replaced with published data streams and their opportunistic consumption across an organization. Expect a lot of diagrams.

Jobs with related skills