Dr. Qiyang Duan

Alibaba Big Data and Machine Learning Technology

The world's largest shopping festival generates petabytes of data. Discover the integrated big data and AI platform Alibaba built to power its massive e-commerce empire.

Alibaba Big Data and Machine Learning Technology
#1about 4 minutes

How e-commerce scale drives Alibaba's big data technology

Alibaba's big data platform evolved to handle massive transaction volumes from events like the Double 11 festival, processing petabytes of data daily.

#2about 4 minutes

Exploring Alibaba's big data and AI product portfolio

The platform is structured around a typical data flow, including tools for data ingestion, storage, processing, visualization, and a dedicated AI platform called PAI.

#3about 4 minutes

Comparing core data processing engines on Alibaba Cloud

A look at key processing tools including MaxCompute for massive-scale SQL, E-MapReduce for Hadoop migrations, and Flink for real-time stream processing.

#4about 5 minutes

Using DataWorks as a unified IDE for big data

DataWorks provides a single environment for data development and operations, integrating various engines like MaxCompute and Hadoop and including data governance tools.

#5about 7 minutes

Developing models with the Platform for AI (PAI)

The PAI platform supports both visual drag-and-drop model building with PAI Studio and code-based development using the Data Science Workshop (DSW) with JupyterLab.

#6about 2 minutes

Announcing the Alibaba Cloud AI Champion 2020 competition

An overview of the AI Champion 2020 competition, which offers cash prizes and cloud credits for innovative projects built on the PAI platform.

#7about 5 minutes

Live demo of data analysis with DataWorks

A walkthrough of the DataWorks interface, demonstrating how to write and execute SQL queries, visualize results, and manage data integration jobs.

#8about 8 minutes

Live demo of building models on the PAI platform

A demonstration of building a recommendation model using the drag-and-drop PAI Studio and running a deep learning example in the PAI-DSW JupyterLab environment.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Increasing the value of talk recordings post-event
04:57 MIN

Increasing the value of talk recordings post-event

Cat Herding with Lions and Tigers - Christian Heilmann

Organizing a developer conference for 15,000 attendees
01:32 MIN

Organizing a developer conference for 15,000 attendees

Cat Herding with Lions and Tigers - Christian Heilmann

Selecting strategic partners and essential event tools
03:17 MIN

Selecting strategic partners and essential event tools

Cat Herding with Lions and Tigers - Christian Heilmann

Establishing a single source of truth for all data
02:39 MIN

Establishing a single source of truth for all data

Cat Herding with Lions and Tigers - Christian Heilmann

Why corporate AI adoption lags behind the hype
03:28 MIN

Why corporate AI adoption lags behind the hype

What 2025 Taught Us: A Year-End Special with Hung Lee

Moving beyond headcount to solve business problems
04:27 MIN

Moving beyond headcount to solve business problems

What 2025 Taught Us: A Year-End Special with Hung Lee

The future of recruiting beyond talent acquisition
03:15 MIN

The future of recruiting beyond talent acquisition

What 2025 Taught Us: A Year-End Special with Hung Lee

Rapid-fire thoughts on the future of work
02:44 MIN

Rapid-fire thoughts on the future of work

What 2025 Taught Us: A Year-End Special with Hung Lee

Featured Partners

Related Videos

See all videos
Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization45:58

Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization

Andreas Christian

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More1:09:49

Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More

Simi Olabisi

Serverless on Cloud41:13

Serverless on Cloud

Cheng Zhang

Data Science in Retail58:21

Data Science in Retail

Julian Joseph

Empowering Retail Through Applied Machine Learning22:41

Empowering Retail Through Applied Machine Learning

Christoph Fassbach & Daniel Rohr

PySpark - Combining Machine Learning & Big Data43:26

PySpark - Combining Machine Learning & Big Data

Ayon Roy

AI in Africa: How can we bounce back?26:12

AI in Africa: How can we bounce back?

Ayoub Alouane

From Syntax to Singularity: AI’s Impact on Developer Roles58:06

From Syntax to Singularity: AI’s Impact on Developer Roles

Anna Fritsch-Weninger

Related Articles

View all articles
DC
Daniel Cranney
The State of WebDev AI 2025 Results: What Can We Learn?
Introduction The 2025 edition of The State of WebDev AI offers a detailed snapshot of how developers are using AI today, which tools have gained the most traction over the past year, and what these trends suggest about the future of the industry. In...
The State of WebDev AI 2025 Results: What Can We Learn?
CH
Chris Heilmann
With AIs wide open - WeAreDevelopers at All Things Open 2025
Last week our VP of Developer Relations, Chris Heilmann, flew to Raleigh, North Carolina to present at All Things Open . An excellent event he had spoken at a few times in the past and this being the “Lucky 13” edition, he didn’t hesitate to come and...
With AIs wide open - WeAreDevelopers at All Things Open 2025

From learning to earning

Jobs that call for the skills explored in this talk.

Data Scientist

Data Scientist

Ai-powered Stackadapt Marketing Platform

Remote
Machine Learning