AWS Data Engineer

LUMINIS STUDIO, INC.
United States
2 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$100,812.0 - $151,408.0
Working hours
Shift work
Job source

Tech stack

Agile Methodology Amazon Web Services Amazon S3 Apache HTTP Server Big Data Continuous Integration Information Engineering Extract Transform Load (ETL) Data Warehousing Apache Hive Identity and Access Management Python (Programming Language)
+14 more
SQL Databases Cloud Platform System Apache Spark Git Cloudformation Data Lakes Pyspark Information Technology AWS Data Analytics Video Streaming Terraform Stream Processing Data Pipelines Amazon Elastic Mapreduce (EMR)

Job description

We are seeking an experienced AWS Data Engineer to design, build, and optimize scalable cloud-based data platforms. The ideal candidate will have strong expertise in AWS data services, data lake architectures, ETL/ELT pipelines, and modern big data technologies including Apache Iceberg, EMR, and Aurora., * Design and develop scalable data pipelines on AWS.

  • Build and maintain data lakes using Apache Iceberg.
  • Develop ETL/ELT workflows using AWS services and big data frameworks.
  • Manage and optimize AWS EMR clusters for large-scale data processing.
  • Design and maintain data solutions using Amazon Aurora.
  • Ensure data quality, governance, security, and performance.
  • Collaborate with data analysts, architects, and business stakeholders.
  • Troubleshoot production issues and optimize data workflows., (ā€œData Engineerā€ OR ā€œAWS Data Engineerā€)AND (Iceberg OR ā€œApache Icebergā€)AND EMRAND AuroraAND (AWS OR Amazon)AND (Python OR PySpark)

Requirements

Do you have experience in Spark implementation?, Do you have a Bachelor’s degree?, * Strong experience with AWS Data Engineering services.

  • Hands-on experience with Apache Iceberg, Amazon EMR, and Amazon Aurora.
  • Proficiency in Python and SQL.
  • Experience with Spark, PySpark, Hive, or similar big data technologies.
  • Knowledge of data warehousing and data lake architectures.
  • Experience with CI/CD, Git, and Agile methodologies.
  • Strong problem-solving and communication skills.

Preferred Skills

  • Experience with AWS Glue, Lambda, S3, Athena, Redshift, and IAM.
  • Knowledge of Terraform or CloudFormation.
  • Exposure to real-time data processing and streaming technologies.

Education

  • Bachelor’s degree in Computer Science, Information Technology, or related field.

Benefits & conditions

$100,812.05 - $151,408.06 a year - Full-time, Pulled from the full job description

  • 401(k)
  • Flexible schedule, * 401(k)
  • Flexible schedule

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy Ā· LIVE

1:43 min

AWS infrastructure stack and data flow pipeline overview

Artem Volk Artem Volk +1 Ā· WWC 2024

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy Ā· LIVE

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer Ā· Coffee With Developers

2:10 min

Why organizations combine big data and machine learning

Ayon Roy Ā· LIVE

2:34 min

Capabilities of the Apache Spark processing engine

Ayon Roy Ā· LIVE

Videos

See all

Related articles

See all