Lead Data Engineer

Cystems Logic Inc.

United States

3 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Airflow Amazon Web Services Microsoft Azure Cloud Database Continuous Integration Data as a Services Data Architecture Information Engineering Data Infrastructure Data Warehousing DevOps Python (Programming Language)

+13 more

Performance Tuning Software Engineering SQL Databases Workflow Management Systems Enterprise Data Management Performance Testing Apache Spark Infrastructure as Code (IaC) Data Lakes Information Technology Data Management Data Pipelines Databricks

Job description

We are seeking an experienced Data Lead with strong expertise in AWS and Databricks to lead the design, development, and modernization of enterprise data platforms. The ideal candidate will have hands-on experience building scalable data solutions in AWS and leading technical teams through cloud-based data transformation initiatives.

Requirements

Experience with Microsoft Azure is highly desirable, as the organization is currently migrating its data platform from Azure to AWS., * 8+ years of experience in Data Engineering, Data Architecture, or related roles.

Bachelor’s degree in Computer Science, Information Technology, Software Engineering, Data Science, or a related technical field.
Strong hands-on experience with AWS data services.
Extensive experience with Databricks, Apache Spark, and Delta Lake.
Experience on performance testing and performance optimization of data pipelines, Databricks workloads, Spark jobs and data platforms.
Experience establishing performance benchmarks, SLAs, KPIs, and capacity planning for enterprise data platforms.
Proficiency in SQL and Python.
Experience designing and implementing enterprise-scale data pipelines.
Strong understanding of data warehousing, data lakes, and modern data architectures.
Excellent leadership, communication, and stakeholder management skills.

Preferred Skills

Experience with Microsoft Azure data services (ADF, Synapse, ADLS, etc.).
Prior experience leading Azure-to-AWS migration projects.
Knowledge of CI/CD, Infrastructure as Code (IaC), and DevOps practices.
Experience with orchestration tools such as Apache Airflow or AWS Step Functions.
AWS and/or Databricks certifications are a plus.