Python-Based Data Streaming Pipelines Within Minutes

Build production-ready streaming pipelines in minutes, not months. This talk introduces a Python-native solution that eliminates complex infrastructure.

#1about 2 minutes

The growing role of Python in real-time data processing

Python is becoming a primary language for real-time data science and machine learning, challenging traditional Java-based tools like Kafka.

#2about 3 minutes

Understanding the challenges of adopting real-time data streaming

Companies hesitate to adopt real-time streaming due to high initial infrastructure costs, the mental shift from batch processing, and inefficient resource usage.

#3about 4 minutes

A traditional approach to streaming with Kafka and Debezium

A common but complex streaming architecture involves using Debezium for change data capture and Kafka as a message broker, which presents DevOps challenges.

#4about 7 minutes

Exploring the operational complexity of Kafka and Flink

Combining Kafka for messaging and Apache Flink for computation creates significant operational overhead, requiring specialized roles and complex infrastructure management.

#5about 4 minutes

Simplifying streaming with modern Python-native frameworks

Modern Python frameworks unify the message broker and stream processor, abstracting away infrastructure complexity and enabling developers to focus on business logic.

#6about 3 minutes

Practical applications for real-time Python data pipelines

Real-time Python pipelines can power various applications, including clickstream analytics, ad enrichment, vector database updates, and anomaly detection alerts.

#7about 8 minutes

How to build a serverless pipeline with GlassFlow

A step-by-step guide shows how to create a real-time data pipeline using a visual editor, a Python transformation function, and webhooks for integration.

#8about 4 minutes

A live demo of a real-time price prediction pipeline

Watch a live demonstration where new data inserted into a Supabase database is instantly processed by a GlassFlow pipeline to predict a price using AI.

#9about 3 minutes

Key benefits of using Python-native streaming frameworks

Python-native frameworks provide self-sufficiency for data teams, reduce infrastructure management with serverless execution, and accelerate the development of real-time applications.