Bobur Umurzokov
Python-Based Data Streaming Pipelines Within Minutes
#1about 2 minutes
The growing role of Python in real-time data processing
Python is becoming a primary language for real-time data science and machine learning, challenging traditional Java-based tools like Kafka.
#2about 3 minutes
Understanding the challenges of adopting real-time data streaming
Companies hesitate to adopt real-time streaming due to high initial infrastructure costs, the mental shift from batch processing, and inefficient resource usage.
#3about 4 minutes
A traditional approach to streaming with Kafka and Debezium
A common but complex streaming architecture involves using Debezium for change data capture and Kafka as a message broker, which presents DevOps challenges.
#4about 7 minutes
Exploring the operational complexity of Kafka and Flink
Combining Kafka for messaging and Apache Flink for computation creates significant operational overhead, requiring specialized roles and complex infrastructure management.
#5about 4 minutes
Simplifying streaming with modern Python-native frameworks
Modern Python frameworks unify the message broker and stream processor, abstracting away infrastructure complexity and enabling developers to focus on business logic.
#6about 3 minutes
Practical applications for real-time Python data pipelines
Real-time Python pipelines can power various applications, including clickstream analytics, ad enrichment, vector database updates, and anomaly detection alerts.
#7about 8 minutes
How to build a serverless pipeline with GlassFlow
A step-by-step guide shows how to create a real-time data pipeline using a visual editor, a Python transformation function, and webhooks for integration.
#8about 4 minutes
A live demo of a real-time price prediction pipeline
Watch a live demonstration where new data inserted into a Supabase database is instantly processed by a GlassFlow pipeline to predict a price using AI.
#9about 3 minutes
Key benefits of using Python-native streaming frameworks
Python-native frameworks provide self-sufficiency for data teams, reduce infrastructure management with serverless execution, and accelerate the development of real-time applications.
Related jobs
Jobs that call for the skills explored in this talk.
ROSEN Technology and Research Center GmbH
Osnabrück, Germany
Intermediate
Python
Docker
+2
Matching moments
04:36 MIN
Why Python is ideal for data streaming frameworks
Convert batch code into streaming with Python
01:52 MIN
Key use cases for Python streaming frameworks
Convert batch code into streaming with Python
01:31 MIN
Key takeaways for modern data processing
Convert batch code into streaming with Python
00:38 MIN
The challenge of managing traditional SQL data pipelines
Enjoying SQL data pipelines with dbt
04:18 MIN
Using streaming data to power real-time agent applications
Unlocking Value from Data: The Key to Smarter Business Decisions-
01:09 MIN
Overview of popular stream processing frameworks
Why and when should we consider Stream Processing frameworks in our solutions
04:18 MIN
Why modern applications adopt event streaming
Event Messaging and Streaming with Apache Pulsar
02:49 MIN
Q&A: MLOps tools for building CI/CD pipelines
Data Science in Retail
Featured Partners
Related Videos
Convert batch code into streaming with Python
Bobur Umurzokov
Let's Get Started With Apache Kafka® for Python Developers
Lucia Cerchie
Tips, Techniques, and Common Pitfalls Debugging Kafka
DeveloperSteve
Fully Orchestrating Databricks from Airflow
Alan Mazankiewicz
Accelerating Python on GPUs
Paul Graham
PySpark - Combining Machine Learning & Big Data
Ayon Roy
Python Data Visualization @ Deepnote (w/ PyViz overview)
Radovan Kavický
Anvil: Full Stack Web Apps With Nothing But Python
Meredydd Luff
Related Articles
View all articles



From learning to earning
Jobs that call for the skills explored in this talk.

bdevit
Canton of Issy-les-Moulineaux, France
API
DevOps
Python
Splunk
Docker
+5

Squarepoint Capital
Charing Cross, United Kingdom
Intermediate
API
C++
Python
PostgreSQL
Amazon Web Services (AWS)

Client Server
Newcastle upon Tyne, United Kingdom
£95-120K
Azure
Spark
Python
Pandas
+2

Digital Talent Agency
Municipality of Madrid, Spain
Java
Azure
NoSQL
Spark
Kafka
+1


Client Server
Charing Cross, United Kingdom
Remote
£100-150K
Spark
Python
Data analysis

Askills
Paris, France
Intermediate
Scrum
Spark
Python
Gitlab
Terraform
+4

ADMIRAL Technologies
Gumpoldskirchen, Austria
Remote
€55K
ETL
Java
Linux
+5
