WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Data Engineer (PySpark + Python)

Byteware Inc.
New York, NY, United States
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours
Job source
www.dice.com

Tech stack

Amazon Web Services Microsoft Azure Big Data Cloud Computing Information Engineering Extract Transform Load (ETL) Data Warehousing Python (Programming Language) Performance Tuning Standard Sql SQL Databases Google Cloud
+3 more
Apache Spark Pyspark Data Pipelines

Requirements

We are looking for a Senior Data Engineer with strong experience in PySpark and Python to build and optimize scalable data pipelines. The ideal candidate should have hands-on experience with Spark, SQL, ETL processes, and cloud platforms (AWS/Azure/Google Cloud Platform). Experience with data warehousing, performance tuning, and big data technologies is preferred., * 10+ years of Data Engineering experience

  • Strong expertise in PySpark and Python
  • Hands-on experience with Apache Spark and SQL
  • Experience with ETL/Data Pipeline development
  • Cloud experience (AWS/Azure/Google Cloud Platform)
  • Excellent problem-solving and communication skills

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

Moments

Explore playlists

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

1:19 min

The true role and evolution of data engineering

Sandhya Menon Sandhya Menon · WWC Europe 2026

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

4:06 min

Using ClickHouse as a foundation for fast analytics

Hellmar Becker Hellmar Becker · WWC Europe 2026

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

Videos

See all
PySpark - Combining Machine Learning & Big Data thumbnail
43:26
LIVE • almost 6 years ago

PySpark - Combining Machine Learning & Big Data

Ayon Roy

Enjoying SQL data pipelines with dbt thumbnail
26:53
WWC 2023 • almost 3 years ago

Enjoying SQL data pipelines with dbt

Matthias Niehoff

Alibaba Big Data and Machine Learning Technology thumbnail
43:57
LIVE • almost 6 years ago

Alibaba Big Data and Machine Learning Technology

Qiyang Duan

How building an industry DBMS differs from building a research one thumbnail
30:34
WWC 2023 • almost 3 years ago

How building an industry DBMS differs from building a research one

Markus Dreseler

Why and when should we consider Stream Processing frameworks in our solutions thumbnail
30:51
WWC 2024 • almost 2 years ago

Why and when should we consider Stream Processing frameworks in our solutions

Soroosh Khodami

Data Science in Retail thumbnail
58:21
LIVE • about 3 years ago

Data Science in Retail

Julian Joseph

Related articles

See all