Data Engineer - AWS

Ecloud Labs

Dallas, TX, United States

8 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Java (Programming Language) Artificial Intelligence Airflow Amazon Web Services Microsoft Azure Big Data Computer Programming Continuous Integration Data Systems Python (Programming Language) Data Streaming Google Cloud

+8 more

Large Language Models Apache Spark Containerization Data Lakes Kubernetes Infrastructure Automation Frameworks Apache Flink Apache Kafka

Job description

Design and implementation of scalable big data pipelines for batch and real time processing

Build and operate streaming data platforms using Kafka
Design and deploy cloud native data solutions across AWS
Develop and manage containerized workloads using Kubernetes
Enable data mesh architecture with domain oriented data products
Design and implement data lake and lakehouse architectures
Ensure data quality, reliability, and observability
Implement governance capabilities including metadata and lineage
Collaborate with analytics, AI, and business teams
Optimize performance and cost efficiency
Mentor engineering teams

Requirements

8+ years of experience in big data engineering

Strong experience with Kafka
Strong experience with Kubernetes
Experience with AWS, Azure, and Google Cloud Platform
Experience building data lakes or lakehouse platforms
Experience with data mesh concepts
Strong programming skills in Python, Scala, or Java
Experience with Spark, Flink, or Beam
Experience with Airflow or orchestration tools
Understanding of data modeling and governance
Experience with CICD and infrastructure automation
Experience supporting AI and machine learning workloads including large language models such as Claude or similar platforms
Strong communication and leadership skills