Lead Data Engineer

Publicis Groupe

Chicago, IL, United States

27 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

3 years minimum

Compensation

$135,500.0

Working hours

Regular working hours

Job source

www.indeed.com

Tech stack

Sql Data Warehouse Amazon Web Services Big Data Cloud Database Code Review Continuous Integration Information Engineering Data Governance Extract Transform Load (ETL) Data Migration Relational Databases Performance Tuning

+13 more

Query Optimization Cloud Services SQL Databases Sql Optimization Apache Spark Git Containerization Information Technology AWS Glue Software Version Control Data Pipelines Amazon Redshift Databricks

Job description

Design, build, and maintain scalable ETL and ELT pipelines based on business requirements, user stories, and architectural standards.
Assemble, transform, and manage large, complex datasets using cloud-native technologies (AWS, Databricks, Amazon Redshift) and SQL-based processing.
Design and maintain relational data models to support analytics, reporting, and downstream consumption.
Lead performance, reliability, and scalability efforts across the data platform, including monitoring, tuning, and optimization of pipelines and workloads.
Develop and maintain Databricks solutions leveraging Spark, Delta Tables, and managed workflows (jobs, orchestration).
Implement data governance and access controls using Unity Catalog and platform best practices.
Use Git-based version control to manage code, support peer collaboration, and enable CI/CD workflows.
Lead and support data platform migrations into Databricks, including legacy warehouse modernization and pipeline refactoring.
Mentor and train junior data engineers through code reviews, technical coaching, and knowledge sharing.
Create clear, accurate, and maintainable technical documentation.
Participate in on-call rotation and provide escalation support for production issues as needed.

Requirements

Bachelor’s degree in Information Technology, Computer Science, Engineering, or a related field.
6+ years of experience in data engineering or a related role.
Strong hands-on experience with ETL/ELT frameworks and cloud data platforms (e.g., Databricks, AWS Glue, Amazon Redshift).
Strong experience with relational databases and advanced SQL, including schema design, joins, window functions, performance tuning, and query optimization-particularly in Amazon Redshift or comparable cloud data warehouses.
Proficiency 3+ years in Python for data processing, automation, and pipeline development.
Deep hands-on experience with Databricks, including Spark optimization, Delta Tables, and Unity Catalog.
Experience migrating data pipelines and workloads into Databricks from legacy or cloud data warehouse platforms (plus).
Databricks certifications (e.g., Data Engineer Associate or Professional) a plus.
Proficiency with Git and version control best practices in a collaborative development environment.
Proven ability to lead to small engineering team.
Excellent communication skills, with the ability to clearly explain technical concepts to technical and non-technical stakeholders.
Strong customer service mindset with the ability to translate business needs into reliable, scalable data solutions.
Highly organized, self-motivated, and able to manage multiple priorities in a fast-paced environment.
Digital media or ad tech experience, especially in data-centric roles, is a plus