WeAreDevelopers LIVE • Oct 12, 2020

PySpark - Combining Machine Learning & Big Data

Ayon Roy

Bypass complex Scala programming and deploy native Python operations across distributed clusters. Learn how PySpark and MLlib unite to accelerate robust, scalable machine learning pipelines.

PySpark - Combining Machine Learning & Big Data thumbnail

Checking access…

Playback and chapters load privately for Free videos.

Matching moments

2:38 min

Overview of Databricks and interactive data processing

Alan Mazankiewicz · LIVE

2:56 min

Options for database machine learning integration architectures

Akmal Chaudhri

Akmal Chaudhri · LIVE

4:18 min

Exploring the big data and machine learning portfolio

Qiyang Duan · LIVE

2:04 min

Comparing offline data analytics with online stream processing

Artem Volk

Artem Volk +1 · World Congress 2024

1:41 min

Visualizing the complex developer journey for JVM ecosystems

Bobur Umurzokov · LIVE

15:08 min

Audience questions on practical machine learning operational strategies

Lina Weichbrodt · LIVE

More from WeAreDevelopers LIVE

Why Your Side Projects Are Worth Sharing thumbnail

Why Your Side Projects Are Worth Sharing

Haimantika Mitra +2

Yesterday

Humans in the Loop: Storytelling in Tech thumbnail

Humans in the Loop: Storytelling in Tech

Pritesh Kiri +2

Last week

WeAreDevelopers LIVE: Your Brain Is Lying to You: Cognitive Biases in Development thumbnail

Your Brain Is Lying to You: Cognitive Biases in Development

2 weeks ago

Like Peas in a Pod - What Happens When People Clone Your App? thumbnail

Like Peas in a Pod - What Happens When People Clone Your App?

Ivan Kutskir +2

3 weeks ago

WeAreDevelopers LIVE - Read more, Prompt Less thumbnail

Read more, Prompt Less

Luise Freese +2

4 weeks ago

WeAreDevelopers LIVE - Tech Education in Two Languages thumbnail

Tech Education in Two Languages

Roxy Rodriguez-Becker +1

Oct 2026

Upcoming sessions on this topic

Starts in 12 days

Conference India 2026

November 25–26, 2026

Bengaluru, India

Get tickets All 107 sessions

Modern forecasting with sktime - foundation models, agents, and benchmarks
Aryan Saini & 3 more

Wed 25 Nov · Room 9
Talk to Your Data Warehouse: Building Real-Time Analyst Agents with ADK and BigQuery MCP
Priyanka Malhotra · Google Cloud

Thu 26 Nov · Room 9
Practical Python AI on the JVM with GraalPy and Pyronaut
Sachin Pikle · Oracle

Wed 25 Nov · Stage 4

Related videos

Alibaba Big Data and Machine Learning Technology thumbnail

Alibaba Big Data and Machine Learning Technology

Qiyang Duan

LIVE · Oct 2020

Overview of Machine Learning in Python thumbnail

Overview of Machine Learning in Python

Adrian Schmitt

LIVE · Dec 2023

Introduction to Azure Machine Learning thumbnail

Introduction to Azure Machine Learning

Jose Luis Latorre Millas

LIVE · Mar 2022

Data Science in Retail thumbnail

Data Science in Retail

Julian Joseph

LIVE · Jun 2023

Let's Get Aggregated: Custom UDAFs in Spark thumbnail

Let's Get Aggregated: Custom UDAFs in Spark

Nikolai Nikolaev · Eurowings Holidays

World Congress · 2025

Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization thumbnail

Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization

Andreas Christian

LIVE · Sep 2021

Related articles

DS

Dhannush Subramani

Top Big Data Technologies That You Need to Know

Quite possibly the most advancing technology in today’s world is Big Data. It is a term that is utilised to portray an assortment of information that is immense in size and is dramatically expandin...

over 4 years ago

Top Big Data Technologies That You Need to Know

BB

Benedikt Bischof

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Bas Geerdink who gave advice on MLOps.‍About the speaker:‍Bas is a programmer, scientist, a...

about 4 years ago

MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production

BB

Benedikt Bischof

MLOps – What’s the deal behind it?

Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOpsAbout the speaker:Nico Axtmann is a seasoned machine...

about 4 years ago

MLOps – What’s the deal behind it?

DC

Daniel Cranney

Stephan Gillich - Bringing AI Everywhere

In the ever-evolving world of technology, AI continues to be the frontier for innovation and transformation. Stephan Gillich, from the AI Center of Excellence at Intel, dove into the subject in a r...

about 2 years ago

Stephan Gillich - Bringing AI Everywhere

From learning to earning

Jobs that call for the skills explored in this talk.

Deal Data Technology & Analytics, Senior Associate

Los Angeles, CA, United States

Expert $77k–202k Spacy Claude OpenAI

Deal Data Technology & Analytics, Senior Associate

San Jose, CA, United States

Expert $77k–202k Spacy Claude OpenAI

Deal Data Technology & Analytics, Senior Associate

New York, NY, United States

Expert $77k–202k Spacy Claude OpenAI

Analytics Engineer

Port Angeles, WA, United States

Expert $149k–187k Remote Presto Twilio DataOps

Senior Analytics Engineer

San Francisco, CA, United States

Expert $149k–187k Remote Presto Twilio DataOps

Machine Learning Engineer

Leganés, Spain

Twilio Pytorch Tensorflow