WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Oct 12, 2020

PySpark - Combining Machine Learning & Big Data

Ayon Roy

Bypass complex Scala programming and deploy native Python operations across distributed clusters. Learn how PySpark and MLlib unite to accelerate robust, scalable machine learning pipelines.

Starting at 8:42 · Watch from the beginning

PySpark - Combining Machine Learning & Big Data thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

2:38 min

Overview of Databricks and interactive data processing

Alan Mazankiewicz · LIVE

2:56 min

Options for database machine learning integration architectures

Akmal Chaudhri Akmal Chaudhri · LIVE

4:18 min

Exploring the big data and machine learning portfolio

Qiyang Duan · LIVE

2:04 min

Comparing offline data analytics with online stream processing

Artem Volk Artem Volk +1 · World Congress 2024

1:41 min

Visualizing the complex developer journey for JVM ecosystems

Bobur Umurzokov · LIVE

15:08 min

Audience questions on practical machine learning operational strategies

Lina Weichbrodt · LIVE

More from WeAreDevelopers LIVE

Humans in the Loop: Storytelling in Tech thumbnail
1:01:28

Humans in the Loop: Storytelling in Tech

Christian Heilmann +2

Last week

WeAreDevelopers LIVE: Your Brain Is Lying to You: Cognitive Biases in Development thumbnail
1:19:46

Your Brain Is Lying to You: Cognitive Biases in Development

Kate Astrid +2

2 weeks ago

Like Peas in a Pod - What Happens When People Clone Your App? thumbnail
1:16:14

Like Peas in a Pod - What Happens When People Clone Your App?

Ivan Kutskir +2

3 weeks ago

WeAreDevelopers LIVE - Read more, Prompt Less thumbnail
1:01:30

Read more, Prompt Less

Christian Heilmann +2

4 weeks ago

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail
1:00:18

Tech Education in Two Languages

Daniel Cranney +1

Oct 2026

WeAreDevelopers LIVE - Playing in Production thumbnail
1:06:40

Playing in Production

Andrew MacLean +2

Sep 2026

Upcoming sessions on this topic

Open session

Modern forecasting with sktime - foundation models, agents, and benchmarks

  • Aryan Saini

    sktime

    AI engineer

  • Franz Kiraly

    sktime

    Director

  • Jigyasu ㅤ

    sktime

    Core developer

  • Shashank Shekhar Singh

    Sktime

    Open Source Developer

Open session

Talk to Your Data Warehouse: Building Real-Time Analyst Agents with ADK and BigQuery MCP

  • Priyanka Malhotra

    Google Cloud

    Platform Customer Engineer

Open session

Practical Python AI on the JVM with GraalPy and Pyronaut

  • Sachin Pikle

    Oracle

    Product Manager

Open session

Engineering AI Strategy: Beyond the Hype

  • Omkar Srivastava

    ADSK

    Engineering Leader

Open session

Architecting Trusted AI: Coupling Multi-Agent Systems with dbt Semantic Store for Zero-Hallucination

  • Pushkar Mishra

    J.P. Morgan

    Senior Vice President

Open session

Agentic Forecasting with sktime: Towards Autonomous Time Series Analysis

  • Franz Kiraly

    sktime

    Director

Related videos

Alibaba Big Data and Machine Learning Technology thumbnail
43:57

Alibaba Big Data and Machine Learning Technology

Qiyang Duan

LIVE · Oct 2020

Overview of Machine Learning in Python thumbnail
57:46

Overview of Machine Learning in Python

Adrian Schmitt

LIVE · Dec 2023

Introduction to Azure Machine Learning thumbnail
54:27

Introduction to Azure Machine Learning

Jose Luis Latorre Millas

LIVE · Mar 2022

Data Science in Retail thumbnail
58:21

Data Science in Retail

Julian Joseph

LIVE · Jun 2023

Let's Get Aggregated: Custom UDAFs in Spark thumbnail
19:58

Let's Get Aggregated: Custom UDAFs in Spark

Nikolai Nikolaev · Eurowings Holidays

World Congress · 2025

Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization thumbnail
45:58

Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization

Andreas Christian

LIVE · Sep 2021

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.