WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Apr 18, 2023

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Are request-response APIs bottlenecking your code? Dive into the publish-subscribe model. Learn how to build real-time Python applications using Apache Kafka as your central nervous system.

Pause
Mute Enter Fullscreen
#1 about 2 min

Overcoming the initial dread of learning event streaming

Responding to unfamiliar distributed paradigms and vast tool ecosystems with curiosity rather than anxiety.

#2 about 2 min

Defining Apache Kafka and its primary real-time use cases

An open-source distributed platform built to handle high-throughput event-driven applications and data pipelines.

#3 about 2 min

Understanding events, topics, and immutable logs in architecture

How key-value pairs are logically grouped into topics and persisted within non-deletable underlying logs.

#4 about 2 min

Publishing events to partitions using data producers

Application producers assign data keys to cluster partitions via hashing routines to optimize throughput compression.

#5 about 2 min

Reading events and scaling workloads with consumer groups

Subscriber clients organize read workloads by group IDs to concurrently scan multiple topic partitions.

#6 about 2 min

Building a data producer using the Confluent Python library

Creating a client instance to procedurally generate and flush website mock click events into infrastructure.

#7 about 2 min

Consuming data streams continuously with a Python consumer

Tracking unread broker offsets and pulling synchronized event data feeds into functional console views.

#8 about 2 min

Real-time event concepts and navigating the tooling ecosystem

How immediate asynchronous propagation mimics nervous systems while requiring broad command-line diagnostic utilities.

#9 about 4 min

Leveraging developer communities to accelerate learning and debugging

Reaching out to active tech meetups and online spaces prevents individual burnout and dissolves roadblocks.

#10 about 3 min

Transitioning from education into developer advocacy roles

Translating educational communication into open-source documentation sharing paves distinct paths toward specialized community support.

#11 about 3 min

Decoupling microservices and processing real-world data pipelines

Asynchronous fraud detection structures eliminate backend connectivity dependencies by standardizing asynchronous microservice schemas.

#12 about 5 min

Origin stories, market opportunities, and top educational resources

Scoping out enterprise availability and introductory tutorials offers clear next steps for learning distributed deployments.

#13 about 5 min

Evaluating alternatives, open source streams, and enterprise offerings

Assessing stream aggregation frameworks reveals critical deployment distinctions between raw library packages and managed connections.

#14 about 5 min

Overcoming integration pitfalls and configuring reliable consumer groups

Structuring group identification properly across microservice deployments removes common bottlenecks related to unread data splits.

#15 about 4 min

Navigating technology stacks regardless of software engineering seniority

Grasping core data movement fundamentals proves valuable from junior coding milestones through architectural infrastructure planning.

Matching moments

3:06 min

Core concepts of Apache Kafka and topic topologies

Denis Washington +1 · WWC 2021

3:45 min

Reviewing core Apache Kafka architecture and distributed fundamentals

Kirill Kulikov · LIVE

5:03 min

Introduction to Apache Kafka as architecture glue

Developersteve · LIVE

2:46 min

Building Kafka producers and consumers in Python

Developersteve · LIVE

2:00 min

Moving from traditional databases to decoupled event streaming

Gerard Klijs · LIVE

1:56 min

Overview of current stream processing frameworks

Hartmut Armbruster Hartmut Armbruster · WWC Europe 2026

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

From Guesswork to Governance: Data Contracts Bring API Discipline to Apache Kafka

Sandon Jacobs

Senior Developer Advocate at IBM

Sandon Jacobs
Open session

World Congress 2026 North America

AI Agents are Only as Smart as their Context: Building a Real-Time Context Engine at Intuit

Bharat Patel

Lead Software Engineer at Intuit

Bharat Patel
Open session

World Congress 2026 North America

Bluesky's Open Source Moderation Tools: LLM-based Event Detection in Python

Alex Garnett

Developer Relations, Bluesky

Alex Garnett
Open session

World Congress 2026 North America

It’s Alive! Taming the MLOps Franken-Stack: Write, Run, and Serve with Michelangelo

Paul Zimmerman, Eric Wang

Paul Zimmerman
Eric Wang
Open session

World Congress 2026 North America

The $0.15/month Algo-Trader: Architecting High-Performance Serverless Trading Engines

Vishwak Thatikonda

Lead Full Stack Engineer at SDH Systems

Vishwak Thatikonda
Open session

World Congress 2026 North America

Building the AI Era Through Developer Communities: A Competitive Advantage for Engineers & Teams

John Komarnicki

National Executive Advisor at Code & Coffee | Engineering & AI Community Leader

John Komarnicki

Related videos

Tips, Techniques, and Common Pitfalls Debugging Kafka thumbnail
54:29
LIVE • over 2 years ago

Tips, Techniques, and Common Pitfalls Debugging Kafka

Developersteve

Kafka Streams Microservices thumbnail
45:48
WWC 2021 • about 5 years ago

Kafka Streams Microservices

Denis Washington, Olli Salonen

How to Benchmark Your Apache Kafka thumbnail
35:27
LIVE • over 5 years ago

How to Benchmark Your Apache Kafka

Kirill Kulikov

Python-Based Data Streaming Pipelines Within Minutes thumbnail
39:04
LIVE • almost 2 years ago

Python-Based Data Streaming Pipelines Within Minutes

Bobur Umurzokov

From event streaming to event sourcing 101 thumbnail
27:33
LIVE • over 5 years ago

From event streaming to event sourcing 101

Gerard Klijs

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen thumbnail
21:08
Coffee With Developers • 5 months ago

Event-Driven Microservices: Patterns and Practices - Lutz Heunkhen

Lutz Heunkhen

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.