WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Apr 18, 2023

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Are request-response APIs bottlenecking your code? Dive into the publish-subscribe model. Learn how to build real-time Python applications using Apache Kafka as your central nervous system.

Pause
Mute Enter Fullscreen
#1 about 2 min

Overcoming the initial dread of learning event streaming

Responding to unfamiliar distributed paradigms and vast tool ecosystems with curiosity rather than anxiety.

#2 about 2 min

Defining Apache Kafka and its primary real-time use cases

An open-source distributed platform built to handle high-throughput event-driven applications and data pipelines.

#3 about 2 min

Understanding events, topics, and immutable logs in architecture

How key-value pairs are logically grouped into topics and persisted within non-deletable underlying logs.

#4 about 2 min

Publishing events to partitions using data producers

Application producers assign data keys to cluster partitions via hashing routines to optimize throughput compression.

#5 about 2 min

Reading events and scaling workloads with consumer groups

Subscriber clients organize read workloads by group IDs to concurrently scan multiple topic partitions.

#6 about 2 min

Building a data producer using the Confluent Python library

Creating a client instance to procedurally generate and flush website mock click events into infrastructure.

#7 about 2 min

Consuming data streams continuously with a Python consumer

Tracking unread broker offsets and pulling synchronized event data feeds into functional console views.

#8 about 2 min

Real-time event concepts and navigating the tooling ecosystem

How immediate asynchronous propagation mimics nervous systems while requiring broad command-line diagnostic utilities.

#9 about 4 min

Leveraging developer communities to accelerate learning and debugging

Reaching out to active tech meetups and online spaces prevents individual burnout and dissolves roadblocks.

#10 about 3 min

Transitioning from education into developer advocacy roles

Translating educational communication into open-source documentation sharing paves distinct paths toward specialized community support.

#11 about 3 min

Decoupling microservices and processing real-world data pipelines

Asynchronous fraud detection structures eliminate backend connectivity dependencies by standardizing asynchronous microservice schemas.

#12 about 5 min

Origin stories, market opportunities, and top educational resources

Scoping out enterprise availability and introductory tutorials offers clear next steps for learning distributed deployments.

#13 about 5 min

Evaluating alternatives, open source streams, and enterprise offerings

Assessing stream aggregation frameworks reveals critical deployment distinctions between raw library packages and managed connections.

#14 about 5 min

Overcoming integration pitfalls and configuring reliable consumer groups

Structuring group identification properly across microservice deployments removes common bottlenecks related to unread data splits.

#15 about 4 min

Navigating technology stacks regardless of software engineering seniority

Grasping core data movement fundamentals proves valuable from junior coding milestones through architectural infrastructure planning.

Matching moments

3:06 min

Core concepts of Apache Kafka and topic topologies

Denis Washington +1 · World Congress 2021

3:45 min

Reviewing core Apache Kafka architecture and distributed fundamentals

Kirill Kulikov · LIVE

5:03 min

Introduction to Apache Kafka as architecture glue

Developersteve · LIVE

2:46 min

Building Kafka producers and consumers in Python

Developersteve · LIVE

2:00 min

Moving from traditional databases to decoupled event streaming

Gerard Klijs · LIVE

1:56 min

Overview of current stream processing frameworks

Hartmut Armbruster Hartmut Armbruster · World Congress 2026 Europe

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
12 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
13 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
20 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
27 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
about 1 month ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - The Shape of SVG to Come! thumbnail
59:56
about 2 months ago

WeAreDevelopers LIVE - The Shape of SVG to Come!

Chris Heilmann, Daniel Cranney, Lionel Péramo

Upcoming sessions on this topic

Open session

World Congress 2026 North America

From Guesswork to Governance: Data Contracts Bring API Discipline to Apache Kafka

Sandon Jacobs

Senior Developer Advocate at IBM

Sandon Jacobs
Open session

World Congress 2026 North America

Replay-Safe Architecture: Building Event-Driven Systems That Can Recover With Confidence

Ishan Shah

PayPal, Software Engineer | Distributed Systems, AI, and Platform Engineering

Ishan Shah
Open session

World Congress 2026 North America

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

Bluesky's Open Source Moderation Tools: LLM-based Event Detection in Python

Alex Garnett

Developer Relations, Bluesky

Alex Garnett
Open session

World Congress 2026 North America

It’s Alive! Taming the MLOps Franken-Stack: Write, Run, and Serve with Michelangelo

Eric Wang, Paul Zimmerman

Eric Wang
Paul Zimmerman
Open session

World Congress 2026 North America

The $0.15/month Algo-Trader: Architecting High-Performance Serverless Trading Engines

Vishwak Thatikonda

Lead Full Stack Engineer at SDH Systems

Vishwak Thatikonda

Related videos

Tips, Techniques, and Common Pitfalls Debugging Kafka thumbnail
54:29
LIVE • over 2 years ago

Tips, Techniques, and Common Pitfalls Debugging Kafka

Developersteve

Kafka Streams Microservices thumbnail
45:48
World Congress 2021 • about 5 years ago

Kafka Streams Microservices

Denis Washington, Olli Salonen

How to Benchmark Your Apache Kafka thumbnail
35:27
LIVE • over 5 years ago

How to Benchmark Your Apache Kafka

Kirill Kulikov

Python-Based Data Streaming Pipelines Within Minutes thumbnail
39:04
LIVE • almost 2 years ago

Python-Based Data Streaming Pipelines Within Minutes

Bobur Umurzokov

From event streaming to event sourcing 101 thumbnail
27:33
LIVE • over 5 years ago

From event streaming to event sourcing 101

Gerard Klijs

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen thumbnail
21:08
Coffee With Developers • 6 months ago

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen

Lutz Heunkhen

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.