Tech Lead - AWS (Data Engineering & Architecture)

We are seeking an experienced Tech Lead with 15+ years of IT experience and deep expertise in AWS, Java, Python, PySpark, and event-driven architectures. The ideal candidate will be responsible for architecting and building scalable batch and streaming data platforms, optimizing cloud-native data solutions, and delivering reliable, high-quality datasets that support analytics, reporting, and machine learning initiatives., Architecture & Solution Design

• Architect, design, and implement scalable, cloud-native data platforms and event-driven data pipelines.
• Design robust batch and real-time streaming solutions that support enterprise analytics and machine learning workloads.
• Lead technical design reviews and establish engineering standards and best practices.
• Design reusable frameworks, accelerators, and reference architectures to improve delivery efficiency and maintainability.

AWS Data Engineering & Cloud Solutions

• Build and maintain event-driven data pipelines utilizing AWS services, including:

• Amazon Kinesis
• Amazon MSK (Managed Kafka)
• AWS Lambda
• AWS Step Functions
• Amazon SQS
• Amazon SNS
• AWS Glue
• Amazon EMR

• Design and manage cloud-native data lakes and data warehouses using:

• Amazon S3
• AWS Glue Catalog
• Amazon Athena
• Amazon Redshift

• Implement cloud security, governance, networking, and cost optimization strategies across AWS environments.

Data Processing & ETL Development

• Develop and optimize ETL/ELT workflows using Python and PySpark.
• Implement Spark-based transformations and optimize:

• Partitioning Strategies
• Distributed Processing Frameworks
• Performance Tuning
• Resource Utilization

• Build highly scalable and resilient data processing pipelines capable of handling large data volumes efficiently.

Streaming & Event-Driven Architecture

• Design and implement streaming solutions with:

• Checkpointing
• Stateful Transformations
• Idempotency
• Schema Evolution

• Develop real-time data pipelines leveraging event-driven architectural patterns and distributed messaging systems.
• Ensure high availability, resiliency, and scalability of streaming applications.

Data Quality, Monitoring & Operations

• Establish data quality frameworks, observability standards, and monitoring solutions.
• Implement monitoring, alerting, and operational dashboards using:

• Amazon CloudWatch
• Datadog
• Other enterprise monitoring platforms

• Proactively identify and resolve performance bottlenecks and operational issues.

DevOps & Automation

• Build and maintain CI/CD pipelines for automated deployments and continuous delivery.
• Utilize Infrastructure-as-Code (IaC) tools, including:

• Terraform
• AWS CloudFormation

• Implement version control and release management processes using Git and enterprise DevOps practices., * Collaborate with data scientists, analysts, architects, and business stakeholders to deliver reliable and well-modeled datasets.
• Mentor junior engineers and provide technical guidance across teams.
• Lead engineering discussions and contribute to strategic technology decisions.
• Drive adoption of best practices in architecture, coding standards, testing, and operational excellence., + Java

• Python
• PySpark
• AWS Cloud Services
• Data Engineering
• Distributed Systems Architecture

• Advanced hands-on experience with:

• Amazon S3
• AWS Glue
• Amazon EMR
• AWS Lambda
• AWS Step Functions
• Amazon Kinesis
• Amazon MSK (Kafka)
• Amazon DynamoDB
• Amazon Athena
• Amazon Redshift

• Deep experience designing and implementing event-driven and streaming data pipelines.
• Strong SQL expertise for analytical and ETL workloads.
• Hands-on experience with workflow orchestration platforms such as:

• Apache Airflow
• AWS Step Functions

This is a hands-on technical leadership role requiring strong architecture, cloud engineering, and data platform expertise, along with the ability to mentor engineering teams and drive best practices across enterprise data initiatives., * 15+ years of professional IT experience., * Experience with:

• CI/CD Pipelines
• Git
• Terraform
• AWS CloudFormation

• Strong understanding of:

• Distributed Systems
• Spark Performance Tuning
• Data Modeling
• Cloud Cost Optimization
• Data Security and Encryption
• Networking and Cloud Governance
• Compliance and Security Best Practices, * Java

• AWS Cloud Platform
• Python
• PySpark
• Event-Driven Architecture
• Apache Kafka / Amazon MSK
• Data Engineering
• Distributed Systems
• Spark Optimization
• Cloud Architecture
• Data Lake and Data Warehouse Design
• Infrastructure as Code (IaC)

Soft Skills

• Strong architectural and solution design capabilities.
• Excellent communication and stakeholder management skills.
• Ability to work effectively within globally distributed teams.
• Strong analytical, problem-solving, and leadership abilities.
• Proven mentoring and team leadership experience.