AWS Data Lead

We are seeking a highly experienced AWS Data Pipeline Engineer/Technical Lead to design, build, optimize, and support scalable, enterprise-grade data pipelines within a cloud-native AWS environment., * Design, develop, deploy, and maintain batch and real-time data pipelines using AWS services.

• Build scalable and resilient solutions for processing multi-terabyte datasets and high-throughput event streams.
• Implement end-to-end data ingestion, transformation, validation, enrichment, and delivery processes.
• Design and implement serverless data processing solutions using AWS Lambda and Step Functions.
• Develop distributed data processing solutions leveraging AWS Glue, EMR, and PySpark.
• Build and optimize data warehouse solutions using Amazon Redshift.
• Develop high-performance analytical solutions using Amazon Athena and S3-based Data Lakes.
• Optimize data pipelines through partitioning strategies, file layout design, indexing, caching, and workload tuning.

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field.
• 13+ years of experience in Data Engineering, ETL/ELT, Data Integration, or Data Pipeline Development.
• Strong expertise in AWS Cloud Data Engineering Services.

• Hands-on implementation experience with:

• AWS Lambda
• AWS Step Functions
• Amazon S3
• AWS Glue
• Amazon Redshift
• Amazon Athena

Strong proficiency in Python, PySpark, and SQL.

Experience building and supporting enterprise-scale Data Lakes and Data Warehouses.

Expertise in data modeling, partitioning strategies, file formats, and performance optimization.

Strong understanding of query tuning, workload management, and bottleneck analysis.

Experience implementing logging, monitoring, alerting, retry mechanisms, fault tolerance, and disaster recovery.

Proven ability to explain architecture decisions, technical implementations, and performance considerations in detail.

Strong analytical, troubleshooting, and technical leadership skills.

Excellent verbal and written communication skills.

Preferred Qualifications

• Experience working in Investment Banking, Capital Markets, or Financial Services environments.
• Knowledge of Data Governance, Metadata Management, and Master Data Management (MDM).
• Experience with Docker and Kubernetes.
• Experience architecting cloud-native, event-driven data platforms.
• Prior experience leading or mentoring data engineering teams.