Data Eng

We are looking for a senior-level Data Engineer to design, build, and optimize scalable data pipelines on AWS using the Databricks Lakehouse Platform. This is an individual contributor role for someone who can work independently, own complex data workflows, and mentor junior engineers. Key Responsibilities Design, develop, and maintain high-performance ETL/ELT pipelines using Databricks, PySpark, and AWS (S3, Glue, Lambda, IAM). Implement and optimize data models following the Medallion Architecture (Bronze/Silver/Gold) with Delta Lake. Implement data governance and access controls using Databricks Unity Catalog. Optimize Spark jobs for performance and cost efficiency through partitioning, clustering, and query tuning. Build and manage CI/CD pipelines for automated deployment of data assets. Perform code reviews, enforce best practices, and mentor junior team members. Partner with data scientists, analysts, and business stakeholders to deliver high-quality data products. Required Qualifications

5 8 years of data engineering experience with production-grade pipelines. AWS: Strong hands-on experience with S3, IAM, Glue, Lambda, and Redshift. Databricks: Expertise in the Lakehouse Platform, including Unity Catalog, Delta Lake, Workflows, and SQL. Programming: Expert-level proficiency in PySpark, Python, and SQL. Data Architecture: Deep understanding of data modeling, Lakehouse architecture, and medallion patterns. CI/CD: Experience with automated deployment pipelines (Azure DevOps, Jenkins, or GitHub Actions). Preferred Qualifications Databricks Certification (Professional or Associate). Experience with streaming data (Kafka/Kinesis) or CDC patterns.