Data Engineer (Databricks / Spark)

Savantis Solutions, LLC

Columbus, OH, United States

3 months ago

Apply on indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

7 years minimum

Compensation

$124,800.0

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Airflow Amazon Web Services Microsoft Azure Big Data Cloud Computing Code Review Continuous Integration Information Engineering Extract Transform Load (ETL) Data Systems Data Warehousing DevOps

+16 more

Python (Programming Language) Operational Databases Performance Tuning SQL Databases Data Ingestion Snowflake Apache Spark Git Data Lakes Pyspark Apache Kafka Data Management Stream Processing Software Version Control Data Pipelines Databricks

Job description

We are seeking an experienced Data Engineer with strong expertise in Databricks and Apache Spark to join a high-performing data engineering team supporting enterprise-scale data initiatives at JPMC. The ideal candidate will have hands-on experience building scalable data pipelines, optimizing Spark workloads, and working with large datasets in cloud-based environments., * Design, develop, and maintain scalable data pipelines using Databricks and Spark.

Build and optimize batch and real-time data processing solutions.
Collaborate with business stakeholders, architects, and development teams to understand data requirements.
Perform data ingestion, transformation, cleansing, and validation activities.
Monitor and troubleshoot production data pipelines.
Implement data quality, governance, and security best practices.
Optimize Spark jobs and Databricks clusters for performance and cost efficiency.
Participate in code reviews and ensure adherence to development standards.

Requirements

Do you have experience in Version control systems?, * 7+ years of experience in Data Engineering and Big Data technologies.

Strong hands-on experience with Databricks and Apache Spark (PySpark/Scala Spark).
Expertise in designing, developing, and optimizing large-scale ETL/ELT pipelines.
Strong experience with Python and SQL.
Experience working with cloud platforms such as AWS, Azure, or GCP.
Hands-on experience with Delta Lake, Data Lake architecture, and data modeling concepts.
Experience with workflow orchestration tools such as Airflow or similar.
Strong understanding of data warehousing concepts and performance tuning.
Experience with version control systems such as Git.
Ability to troubleshoot and optimize Spark jobs for performance and scalability., * Experience working in financial services or banking environments.
Familiarity with JPMC data platforms and enterprise data ecosystems.
Experience with Kafka, Snowflake, or other modern data technologies.
Knowledge of CI/CD processes and DevOps practices.
Databricks or Cloud certifications are a plus.