AWS Data Engineer

Brilliant

United States

1 day ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Amazon Web Services Amazon S3 Automation of Tests Computer Programming Continuous Integration Information Engineering Data Integration Extract Transform Load (ETL) Python (Programming Language) Shell Script SQL Databases Performance Testing

+9 more

Delivery Pipeline Information Technology AWS Glue Data Analytics Functional Programming Cloudwatch Terraform Data Pipelines Jenkins

Job description

Data Pipelines: Develop configuration driven ingestion and transformation pipelines using AWS Glue, Lambda, and Redshift.
Automation: Implement CI/CD pipelines for automated build, test, and deployment of data products.
Testing: Establish automated test frameworks for schema validation, data quality, and performance testing.
Monitoring: Set up monitoring and observability dashboards for product level metrics (CloudWatch, custom metrics).
Collaboration: Work with data modelers and architects to ensure alignment with business requirements.

Requirements

Programming: Proficiency in Python, SQL, and shell scripting.
AWS Expertise: Hands on experience with AWS services (Glue, Redshift, S3, Lambda, CloudWatch).
Automation: Experience with CI/CD tools (Terraform, Jenkins, AWS CodePipeline).
Data Integration: Strong knowledge of ETL/ELT processes and frameworks.
Soft Skills: Strong problem solving, collaboration, and communication skills., * Bachelorâ€¿s degree in Computer Science, Data Engineering, or related field.
5+ years of experience in data engineering and pipeline development.
AWS Certified Data Analytics or AWS Certified Developer is a plus.