AWS Data Lead / Lead Consultant - Data on Cloud Platform

Argyll Infotech Inc
Fort Mill, United States of America
2 days ago

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Fort Mill, United States of America

Tech stack

Airflow
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Cloud Database
Cloud Engineering
Databases
Information Engineering
Data Infrastructure
ETL
Data Transformation
Data Systems
Data Warehousing
DevOps
PostgreSQL
Microsoft SQL Server
Cloud Services
SQL Databases
Data Processing
Cloud Platform System
Sql Optimization
Snowflake
Data Build Tool (dbt)
AWS Lambda
Infrastructure as Code (IaC)
Event Driven Architecture
Amazon Web Services (AWS)
PySpark
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Terraform
Data Pipelines

Job description

We are seeking an experienced AWS Data Lead to design, develop, and lead enterprise-scale cloud data integration and data warehousing initiatives. The ideal candidate will have deep expertise in AWS Glue, PySpark, DBT, and Apache Airflow, along with a proven track record of building scalable ETL/ELT pipelines and cloud-native data platforms. This role combines technical leadership with hands-on engineering and requires close collaboration with cross-functional teams to deliver reliable, high-performance data solutions., * Lead the architecture, design, and implementation of scalable ETL/ELT solutions using AWS Glue and PySpark.

  • Develop and automate end-to-end data pipelines that support enterprise data integration and analytics initiatives.
  • Build and manage workflow orchestration using Apache Airflow to ensure reliable and efficient data processing.
  • Design cloud-native, event-driven architectures leveraging AWS services such as:
  • AWS Glue
  • AWS Lambda
  • Amazon EventBridge
  • Amazon S3
  • Amazon RDS (Aurora/PostgreSQL)
  • Amazon MSK and/or Amazon Kinesis
  • Implement data transformation and modeling best practices using DBT and advanced SQL techniques.
  • Work with relational and cloud data platforms including Snowflake, Aurora PostgreSQL, PostgreSQL, and Microsoft SQL Server.
  • Build and maintain CI/CD pipelines and Infrastructure as Code (IaC) solutions using tools such as Terraform.
  • Ensure high standards for data quality, reliability, scalability, and performance across all data pipelines.
  • Provide technical leadership, mentor team members, and collaborate with business stakeholders to deliver strategic data solutions.
  • Drive best practices in cloud engineering, DevOps, and data platform modernization., 1. Design and deliver scalable ETL/ELT pipelines using AWS Glue, PySpark, and Apache Airflow.
  1. Lead cloud-based data integration and data warehouse initiatives while ensuring performance, reliability, and data quality.
  2. Collaborate with technical and business stakeholders to implement modern, automated, and scalable AWS data platform solutions.

Requirements

  • 10+ years of experience in data engineering, ETL development, or cloud data platform engineering.
  • Extensive hands-on experience with:
  • AWS Glue
  • PySpark
  • DBT (Data Build Tool)
  • Apache Airflow
  • Strong expertise in designing and implementing scalable ETL/ELT pipelines.
  • Deep knowledge of AWS services, including Glue, Lambda, EventBridge, S3, RDS (Aurora/PostgreSQL), and MSK/Kinesis.
  • Proficiency in SQL, data modeling, and cloud-native database technologies.
  • Experience with CI/CD pipelines, Infrastructure as Code, and Terraform.
  • Strong communication, leadership, and stakeholder management skills.

Preferred Qualifications

  • Experience with Scala.
  • Hands-on experience with Snowflake.
  • Experience implementing modern DevOps practices in cloud environments.
  • Exposure to enterprise data warehouse modernization initiatives.

Preferred Certifications

  • AWS Certified Data Engineer
  • AWS Certified Solutions Architect

Apply for this position