Spark Engineer

Tata Consultancy Services Limited
Jacksonville, United States of America
19 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Compensation
$ 100K

Job location

Jacksonville, United States of America

Tech stack

Amazon Web Services (AWS)
Amazon Web Services (AWS)
Data analysis
Azure
Big Data
Cloud Storage
ETL
Data Systems
Data Warehousing
Database Queries
Distributed Systems
Hive
Performance Tuning
Query Optimization
Azure
Data Streaming
Data Processing
Cloud Platform System
Snowflake
Spark
Information Technology
Stream Processing

Job description

  • Design, develop, and maintain large-scale batch and real-time data processing solutions using Apache Spark (v3.5.x or later)

  • Work extensively with Spark DataFrames, Datasets, and RDDs for high-volume data processing

  • Develop and optimize queries using Spark SQL

  • Build and manage ETL pipelines to support analytics and downstream data consumption

  • Perform Spark job tuning and performance optimization to improve efficiency and scalability

  • Collaborate with data engineers, architects, and analytics teams to deliver end-to-end data solutions

  • Ensure data quality, reliability, and adherence to best practices across pipelines

Requirements

We are looking for an experienced Apache Spark Developer with strong expertise in large-scale data processing and performance optimization. The role involves designing, building, and optimizing scalable ETL pipelines using Spark in cloud environments. Key Skills

  • Strong hands-on expertise in Apache Spark 3.5.x or later

  • Solid experience with Spark DataFrames, Datasets, and RDDs

  • Strong proficiency in Spark SQL

  • Experience working in cloud environments (AWS and/or Azure)

  • Hands-on experience with cloud storage solutions such as Amazon S3 and Azure Data Lake Storage (ADLS)

  • Strong understanding of ETL pipeline design and data modeling

  • Proven experience in Spark job performance tuning and query optimization

Preferred / Good to Have

  • Experience with Snowflake for data warehousing and analytics

  • Exposure to real-time/streaming data processing

  • Familiarity with distributed systems and large-scale data architectures

  • Scala Programming Language, * Strong hands on experience with Spark

  • In depth knowledge of Apache Spark architecture, internals, and execution model

  • Experience working with Spark DataFrames, datasets, RDDs (large scale data processing - batch & real-time)

  • Strong SQL skills for data analysis and transformation

  • Familiarity with ETL pipelines and data modeling(Snowflake-preferrred)

  • Experience with Linux/Unix environments

  • Good understanding of distributed computing pri nciples

Good to Have

  • Experience with Snowflake for data warehousing and analytics

  • Exposure to real-time/streaming data processing

  • Familiarity with distributed systems and large-scale data architectures, Qualifications : BACHELOR OF COMPUTER SCIENCE You must create an Indeed account before continuing to the company website to apply

Benefits & conditions

Salary Range $ 80,000-$100,000 Per year TCS Employee Benefits Summary: Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & amp; Training Reimbursement. Time Off: Vacation, Time Off, Sick Leave & Holidays.

Apply for this position