Bobur Umurzokov

Python-Based Data Streaming Pipelines Within Minutes

Build production-ready streaming pipelines in minutes, not months. This talk introduces a Python-native solution that eliminates complex infrastructure.

Python-Based Data Streaming Pipelines Within Minutes
#1about 2 minutes

The growing role of Python in real-time data processing

Python is becoming a primary language for real-time data science and machine learning, challenging traditional Java-based tools like Kafka.

#2about 3 minutes

Understanding the challenges of adopting real-time data streaming

Companies hesitate to adopt real-time streaming due to high initial infrastructure costs, the mental shift from batch processing, and inefficient resource usage.

#3about 4 minutes

A traditional approach to streaming with Kafka and Debezium

A common but complex streaming architecture involves using Debezium for change data capture and Kafka as a message broker, which presents DevOps challenges.

#4about 7 minutes

Exploring the operational complexity of Kafka and Flink

Combining Kafka for messaging and Apache Flink for computation creates significant operational overhead, requiring specialized roles and complex infrastructure management.

#5about 4 minutes

Simplifying streaming with modern Python-native frameworks

Modern Python frameworks unify the message broker and stream processor, abstracting away infrastructure complexity and enabling developers to focus on business logic.

#6about 3 minutes

Practical applications for real-time Python data pipelines

Real-time Python pipelines can power various applications, including clickstream analytics, ad enrichment, vector database updates, and anomaly detection alerts.

#7about 8 minutes

How to build a serverless pipeline with GlassFlow

A step-by-step guide shows how to create a real-time data pipeline using a visual editor, a Python transformation function, and webhooks for integration.

#8about 4 minutes

A live demo of a real-time price prediction pipeline

Watch a live demonstration where new data inserted into a Supabase database is instantly processed by a GlassFlow pipeline to predict a price using AI.

#9about 3 minutes

Key benefits of using Python-native streaming frameworks

Python-native frameworks provide self-sufficiency for data teams, reduce infrastructure management with serverless execution, and accelerate the development of real-time applications.

Related jobs
Jobs that call for the skills explored in this talk.

Featured Partners

Related Videos

See all videos
Convert batch code into streaming with Python46:43

Convert batch code into streaming with Python

Bobur Umurzokov

Let's Get Started With Apache Kafka® for Python Developers38:50

Let's Get Started With Apache Kafka® for Python Developers

Lucia Cerchie

Tips, Techniques, and Common Pitfalls Debugging Kafka54:29

Tips, Techniques, and Common Pitfalls Debugging Kafka

DeveloperSteve

Fully Orchestrating Databricks from Airflow39:14

Fully Orchestrating Databricks from Airflow

Alan Mazankiewicz

Accelerating Python on GPUs59:43

Accelerating Python on GPUs

Paul Graham

PySpark - Combining Machine Learning & Big Data43:26

PySpark - Combining Machine Learning & Big Data

Ayon Roy

Python Data Visualization @ Deepnote (w/ PyViz overview)50:38

Python Data Visualization @ Deepnote (w/ PyViz overview)

Radovan Kavický

Anvil: Full Stack Web Apps With Nothing But Python58:28

Anvil: Full Stack Web Apps With Nothing But Python

Meredydd Luff

From learning to earning

Jobs that call for the skills explored in this talk.

AWS DevOps / Python

RED Global
Charing Cross, United Kingdom

€51K
DevOps
Python
Kubernetes
Amazon Web Services (AWS)

AWS DevOps/Python

Red - The Global SAP Solutions Provider
Belfast, United Kingdom

DevOps
Python
Kubernetes
Amazon Web Services (AWS)