Lead Data Engineer- AWS

As part of the Big Data Engineering and Analytics organization, this role will focus on enabling a modern data lake and data mesh architecture that supports high volume, high velocity, and high variety data across enterprise and platform domains.

The Lead Data Engineer will bring deep hands-on expertise in Kubernetes, Kafka streaming platforms. This role partners closely with data architects, analytics teams, and platform engineering to deliver reliable, governed, and high-quality data solutions.

Responsibilities

• Lead the design and implementation of scalable big data pipelines for batch and real time processing

• Build and operate streaming data platforms using Kafka

• Design and deploy cloud native data solutions across AWS

• Develop and manage containerized workloads using Kubernetes

• Enable data mesh architecture with domain oriented data products

• Design and implement data lake and lakehouse architectures

• Ensure data quality, reliability, and observability

• Implement governance capabilities including metadata and lineage

• Collaborate with analytics, AI, and business teams

• Optimize performance and cost efficiency

10+ years of experience in big data engineering

• Strong experience with Kafka

• Strong experience with Kubernetes

• Experience with AWS

• Experience building data lakes or lakehouse platforms

• Experience with data mesh concepts

• Strong programming skills in Python, Scala, or Java

• Experience with Spark, Flink, or Beam

• Experience with Airflow or orchestration tools

• Understanding of data modeling and governance

• Experience with CICD and infrastructure automation

• Experience supporting AI and machine learning workloads including large language models such as Claude or similar platforms

• Strong communication and leadership skills