> Markdown version of [/jobs/ext/3877029-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3877029-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** SWIFT
- **Location:** Chicago, IL, United States
- **Experience:** Expert
- **Contract:** Temporary to permanent
- **Skills:** Unity 3d, Artificial Intelligence, Amazon Web Services, Amazon S3, Microsoft Azure, Continuous Integration, Data Architecture, Information Engineering, Extract Transform Load (ETL), DevOps, Distributed Computing Environment, Identity and Access Management, Python (Programming Language), Standard Sql, Data Streaming, Workflow Management Systems, Apache Spark, Data Lakes, Information Technology, Data Pipelines, Databricks
- **Published:** October 22, 2026
- **Apply:** https://www.careerjet.com/jobad/us86c22f27f090dad64b53d627c6687ea1

## About the Role

10+ years of overall IT experience, with strong Data Engineering and ETL expertise.
4+ years of hands-on Databricks experience, specifically on AWS. Azure or other-platform experience alone will not qualify.
Strong expertise in AWS services, including S3, IAM, and networking.
Hands-on experience with Apache Spark, Delta Lake, Unity Catalog, and Databricks Lakehouse architecture.
Strong SQL, Python, data modeling, and distributed data processing skills.
Experience with ETL/ELT pipelines, batch and streaming data processing, CI/CD, and DevOps.
Knowledge of Delta Live Tables (DLT), Databricks SQL, workflow orchestration, and medallion architecture.
Experience leading technical design, establishing engineering standards, mentoring team members, and collaborating with business stakeholders., Candidates must have recent, hands-on Databricks-on-AWS experience. Databricks on Azure or another platform is not a substitute.

## Description

Lead the design and implementation of client's modern AWS and Databricks Lakehouse platform. Own the end-to-end ETL/ELT lifecycle, build scalable and reliable data pipelines, optimize performance and costs, and support enterprise analytics, reporting, and AI initiatives.
Interview Process

 1. 60 90-minute technical interview with the Director, preferably onsite.
 2. 30 45-minute behavioral and team-fit interview with the Manager.
Important Notes
Candidates must be local to the Chicagoland area.
The first month is onsite in downtown Chicago; the role will then be mostly remote, with onsite visits as needed.
This is a contract-to-hire opportunity, with conversion expected within 3 6 months.