WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Apr 18, 2023

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Are request-response APIs bottlenecking your code? Dive into the publish-subscribe model. Learn how to build real-time Python applications using Apache Kafka as your central nervous system.

Pause
Mute Enter Fullscreen
#1 about 2 min

Overcoming the initial dread of learning event streaming

Responding to unfamiliar distributed paradigms and vast tool ecosystems with curiosity rather than anxiety.

#2 about 2 min

Defining Apache Kafka and its primary real-time use cases

An open-source distributed platform built to handle high-throughput event-driven applications and data pipelines.

#3 about 2 min

Understanding events, topics, and immutable logs in architecture

How key-value pairs are logically grouped into topics and persisted within non-deletable underlying logs.

#4 about 2 min

Publishing events to partitions using data producers

Application producers assign data keys to cluster partitions via hashing routines to optimize throughput compression.

#5 about 2 min

Reading events and scaling workloads with consumer groups

Subscriber clients organize read workloads by group IDs to concurrently scan multiple topic partitions.

#6 about 2 min

Building a data producer using the Confluent Python library

Creating a client instance to procedurally generate and flush website mock click events into infrastructure.

#7 about 2 min

Consuming data streams continuously with a Python consumer

Tracking unread broker offsets and pulling synchronized event data feeds into functional console views.

#8 about 2 min

Real-time event concepts and navigating the tooling ecosystem

How immediate asynchronous propagation mimics nervous systems while requiring broad command-line diagnostic utilities.

#9 about 4 min

Leveraging developer communities to accelerate learning and debugging

Reaching out to active tech meetups and online spaces prevents individual burnout and dissolves roadblocks.

#10 about 3 min

Transitioning from education into developer advocacy roles

Translating educational communication into open-source documentation sharing paves distinct paths toward specialized community support.

#11 about 3 min

Decoupling microservices and processing real-world data pipelines

Asynchronous fraud detection structures eliminate backend connectivity dependencies by standardizing asynchronous microservice schemas.

#12 about 5 min

Origin stories, market opportunities, and top educational resources

Scoping out enterprise availability and introductory tutorials offers clear next steps for learning distributed deployments.

#13 about 5 min

Evaluating alternatives, open source streams, and enterprise offerings

Assessing stream aggregation frameworks reveals critical deployment distinctions between raw library packages and managed connections.

#14 about 5 min

Overcoming integration pitfalls and configuring reliable consumer groups

Structuring group identification properly across microservice deployments removes common bottlenecks related to unread data splits.

#15 about 4 min

Navigating technology stacks regardless of software engineering seniority

Grasping core data movement fundamentals proves valuable from junior coding milestones through architectural infrastructure planning.

Matching moments

3:06 min

Core concepts of Apache Kafka and topic topologies

Denis Washington +1 · World Congress 2021

2:20 min

Event streaming and distributed systems with Apache Kafka

Sandon Jacobs Sandon Jacobs · World Congress 2026 North America

3:45 min

Reviewing core Apache Kafka architecture and distributed fundamentals

Kirill Kulikov · LIVE

5:43 min

Processing real-time event streams with Apache Kafka

Alex Soto Alex Soto · LIVE

5:03 min

Introduction to Apache Kafka as architecture glue

Developersteve · LIVE

2:46 min

Building Kafka producers and consumers in Python

Developersteve · LIVE

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Read more, Prompt Less thumbnail
1:01:30
3 days ago

WeAreDevelopers LIVE - Read more, Prompt Less

Chris Heilmann, Daniel Cranney, Luise Freese

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail
1:00:18
10 days ago

WeAreDevelopers LIVE - Tech Education in Two Languages

Daniel Cranney, Roxy Rodriguez-Becker

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40
17 days ago

WeAreDevelopers LIVE - Playing in Production

Andrew MacLean, Daniel Cranney, Christian Heilmann

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
about 1 month ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
about 1 month ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof Cieślak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
about 1 month ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

How We Cut Our API's p99 Latency from Minutes to Under a Second

  • Deepak Agrawal

    Atlassian

    Principal Software Engineer

Open session

Modern forecasting with sktime - foundation models, agents, and benchmarks

  • Aryan Saini

    sktime

    AI engineer

  • Franz Kiraly

    sktime

    Director

  • Jigyasu ㅤ

    sktime

    Core developer

Open session

Java is Coming for Python’s Lunch: The Rise of JVM AI Agents

  • Frederik Pietzko

    Jetbrains

    Developer Advocate

Open session

Agentic Forecasting with sktime: Towards Autonomous Time Series Analysis

  • Franz Kiraly

    sktime

    Director

Open session

Beyond File Dumps: Context Engineering for Coding Agents

  • Animesh Dutta

    Arm

    Senior Software Engineer

Open session

React Under Fire: Rendering Performance for Streaming Agent UIs

  • Abhirupa Mitra

    Slack

    Senior Member of Technical Staff

Related videos

Tips, Techniques, and Common Pitfalls Debugging Kafka thumbnail
54:29
LIVE • almost 3 years ago

Tips, Techniques, and Common Pitfalls Debugging Kafka

Developersteve

Kafka Streams Microservices thumbnail
45:48
World Congress 2021 • over 5 years ago

Kafka Streams Microservices

Denis Washington, Olli Salonen

From Guesswork to Governance: Data Contracts Bring API Discipline to Apache Kafka thumbnail
34:30
World Congress 2026 North America • 22 days ago

From Guesswork to Governance: Data Contracts Bring API Discipline to Apache Kafka

Sandon Jacobs

How to Benchmark Your Apache Kafka thumbnail
35:27
LIVE • almost 6 years ago

How to Benchmark Your Apache Kafka

Kirill Kulikov

Python-Based Data Streaming Pipelines Within Minutes thumbnail
39:04
LIVE • almost 2 years ago

Python-Based Data Streaming Pipelines Within Minutes

Bobur Umurzokov

Convert batch code into streaming with Python thumbnail
46:43
LIVE • over 2 years ago

Convert batch code into streaming with Python

Bobur Umurzokov

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.