Pyspark Developer

Tata Consultancy Services Limited
Irving, TX, United States
12 days ago
Apply on diversityjobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
$90,000 - $110,000
Working hours
Regular working hours

Tech stack

Amazon Web Services Microsoft Azure Big Data Extract Transform Load (ETL) Data Transformation Apache Hadoop Python (Programming Language) Performance Tuning Standard Sql Data Processing Data Lakes Pyspark
+2 more
Data Pipelines Databricks

Job description

We are seeking a PySpark Developer with strong experience in Python, ETL, and Databricks to design, build, and optimize scalable data pipelines. The role involves processing large datasets, developing data transformation workflows, and implementing data engineering solutions on the Databricks Lakehouse Platform. Required Technical/Functional Skills

  • PySpark
  • Python
  • Databricks
  • ETL
  • Hadoop
  • AWS

Roles & Responsibilities

  • Design, build, and optimize scalable data pipelines.
  • Process large datasets using PySpark.
  • Develop efficient data transformation and ETL workflows.
  • Implement data engineering solutions on the Databricks Lakehouse Platform.
  • Support reliable and scalable data processing requirements.

Requirements

Experience with the following is preferred:

  • AWS, Azure, or GCP
  • SQL
  • Delta Lake
  • Data pipeline and PySpark performance tuning

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on diversityjobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…