AWS Data Engineer

Select Minds LLC

Dallas, TX, United States

5 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

2 years minimum

Working hours

Shift work

Job source

www.wayup.com

Tech stack

Agile Methodology Airflow Amazon Web Services Amazon S3 Unit Testing Code Review Continuous Integration Information Engineering Data Integrity Extract Transform Load (ETL) Data Security Data Systems

+16 more

Software Design Patterns Desktop Publishing DevOps Identity and Access Management Python (Programming Language) Scrum Methodology SQL Databases Data Processing Cloud Platform System Test-Driven Development (TDD) Apache Spark Pyspark AWS Data Analytics Cloud Migration Terraform Data Pipelines

Job description

Core Skills: Strong AWS Services, Python/PySpark, Advanced SQL Submission Details: Resume, LinkedIn URL, Work Authorization, Location Details Job Description Our esteemed client is seeking an experienced AWS Data Engineer with strong expertise in ETL testing, cloud migration, and Python programming in a production-grade AWS environment. You will design and maintain scalable data pipelines, ensure data quality through rigorous ETL testing, and play a key role in large-scale cloud migration initiatives. This role is ideal for someone passionate about building efficient, secure, and reliable data solutions, applying functional design principles, and leveraging automation to deliver business impact. Key Responsibilities

Develop and maintain scalable ETL pipelines within the AWS ecosystem.
Conduct ETL testing to ensure data integrity, accuracy, and performance.
Lead and support large-scale cloud migration projects.
Use Python (primary language) along with SQL and PySpark to develop data processing and automation solutions.
Orchestrate data workflows using Apache Airflow (including MWAA).
Collaborate with cross-functional teams to design data models and implement industry-standard data security and classification methodologies.
Monitor, troubleshoot, and optimize pipelines for reliability and performance.
Document design, conduct code reviews, and apply test-driven development (TDD) practices.
Work in Agile environments, participating in sprint planning and daily stand-ups.
Ensure a smooth transition of data during cloud migrations, applying DevOps and CI/CD practices where needed

Requirements

8-12 years of experience in Data Engineering, focusing on ETL, Cloud Migration, and Python development.
5+ years of hands-on experience with Python (primary), SQL, and PySpark, applying functional design principles and software design patterns.
5+ years building and deploying ETL pipelines across on-prem, hybrid, and cloud environments, with orchestration experience using Airflow.
5+ years of production-level AWS experience (MWAA, Glue/EMR (Spark), S3, ECS/EKS, IAM, Lambda).
Solid understanding of core statistical principles, data modeling, and data security best practices.
5+ years working in Agile development with unit testing, TDD, and design documentation.
2+ years of DevOps/CI-CD experience, including Terraform or other Infrastructure-as-Code (IaC) platforms.
Excellent problem-solving skills, with the ability to troubleshoot complex data and performance issues.
Strong communication skills and ability to work effectively in a hybrid model (3 days onsite in Dallas). Flexible work from home options available.