Pyspark Developer
Tata Consultancy Services Limited
Irving, TX, United States
12 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on diversityjobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
$90,000 - $110,000
Working hours
Regular working hours
Job source
Tech stack
Amazon Web Services
Microsoft Azure
Big Data
Extract Transform Load (ETL)
Data Transformation
Apache Hadoop
Python (Programming Language)
Performance Tuning
Standard Sql
Data Processing
Data Lakes
Pyspark
+2 more
Data Pipelines
Databricks
Job description
We are seeking a PySpark Developer with strong experience in Python, ETL, and Databricks to design, build, and optimize scalable data pipelines. The role involves processing large datasets, developing data transformation workflows, and implementing data engineering solutions on the Databricks Lakehouse Platform. Required Technical/Functional Skills
- PySpark
- Python
- Databricks
- ETL
- Hadoop
- AWS
Roles & Responsibilities
- Design, build, and optimize scalable data pipelines.
- Process large datasets using PySpark.
- Develop efficient data transformation and ETL workflows.
- Implement data engineering solutions on the Databricks Lakehouse Platform.
- Support reliable and scalable data processing requirements.
Requirements
Experience with the following is preferred:
- AWS, Azure, or GCP
- SQL
- Delta Lake
- Data pipeline and PySpark performance tuning
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on diversityjobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦