Lead Data Engineer

• Lead the design, architecture, and implementation of enterprise data engineering solutions across the AWS ecosystem
• Collaborate with Lead Developers, Data Scientists, Architects, Product Owners, and business stakeholders to define technical strategy and scalable solutions
• Provide technical leadership and mentorship to Data Engineers and development teams, promoting engineering excellence and best practices
• Drive key architectural decisions in partnership with Data Architects and Solution Architects to ensure scalability, security, reliability, and maintainability
• Design and oversee data warehouse and data lake solutions that balance business usability, performance, and long-term sustainability
• Establish engineering standards for data modeling, ETL frameworks, pipeline reliability, monitoring, and operational excellence
• Lead end-to-end solution delivery, ensuring alignment with business requirements, enterprise architecture standards, and regulatory requirements
• Oversee production support and operational management of AWS-based data platforms, driving root-cause analysis and performance optimization initiatives
• Champion data quality, governance, observability, and data stewardship practices across platforms and teams
• Identify opportunities to modernize data architecture and improve operational efficiency through automation and cloud-native technologies

• 8+ years of experience in Data Engineering with at least 5+ years working extensively within AWS ecosystems
• Expert-level experience with AWS services including S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, and Step Functions
• Experience creating AI applications with AWS Bedrock
• Strong experience designing and implementing enterprise-scale data lake and data warehouse solutions using Lake Formation, Amazon Redshift, and Amazon Athena
• Extensive experience with Kafka-based streaming architectures, preferably Confluent Kafka
• Advanced SQL and data modeling expertise, including dimensional modeling, data vault, and large-scale data warehousing solutions
• Deep experience designing, developing, and optimizing scalable, resilient data pipelines within AWS environments
• Strong understanding of distributed data processing frameworks, particularly PySpark and EMR
• Expert knowledge of database management principles, performance tuning, and data architecture best practices
• Advanced Python development skills with extensive hands-on experience using PySpark
• Expertise in Infrastructure as Code using Terraform
• Experience designing and implementing CI/CD frameworks using GitHub and GitHub Actions
• Deep knowledge of AWS IAM roles, policies, governance, and security best practices
• Strong experience with workflow orchestration tools such as AWS Step Functions, Apache Airflow, or equivalent platforms
• Experience leading cloud migration, modernization, and enterprise data platform initiatives
• Strong understanding of data governance, metadata management, data quality frameworks, and observability principles, * Ability to lead hands-on development efforts while providing technical direction, code reviews, and engineering oversight across multiple initiatives
• Experience establishing development environments, infrastructure standards, security controls, and migration strategies across multiple AWS accounts and environments
• Ability to architect, develop, and govern enterprise-scale data pipelines, ETL processes, data ingestion frameworks, and orchestration workflows
• Proven ability to identify data gaps, define strategic remediation plans, and implement scalable automation solutions that improve analytical capabilities across the organization
• Ability to design and implement highly reliable data pipelines with strong focus on data quality, observability, resiliency, and operational supportability
• Experience building and optimizing large-scale data warehousing solutions that prioritize both business user experience and system performance
• Ability to drive technical decision-making, influence architectural direction, and effectively communicate complex technical concepts to both technical and non-technical stakeholders
• Proven track record mentoring engineers, fostering technical growth, and building high-performing data engineering teams
• Experience leading cross-functional initiatives involving data engineering, analytics, architecture, platform engineering, and business stakeholders
• Strong problem-solving and leadership skills with the ability to manage competing priorities in fast-paced enterprise environments
• Experience building AI-ready data pipelines and ML workflows, including feature engineering and MLOps
• Proficiency in Python, SQL, Spark, and Generative AI technologies for scalable data and analytics solutions
• Hands-on experience with AWS, Azure, or Databricks and cloud-based AI/data platforms
• Knowledge of LLMs, RAG, and vector databases to support AI-powered applications and intelligent search
• Strong understanding of data governance, quality, security, and responsible AI practices

Preferred Qualifications

• AWS Certified Data Engineer, AWS Solutions Architect, or equivalent cloud certifications
• Experience implementing enterprise data governance and metadata management platforms
• Experience with real-time analytics, event-driven architectures, and streaming data platforms
• Knowledge of modern data architecture patterns including Data Mesh, Lakehouse, and domain-oriented design
• Experience leading large-scale cloud transformation or enterprise data modernization programs