Data Engineer

Insight Global

Charlotte, NC, United States

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

7 years minimum

Working hours

Regular working hours

Job source

www.juju.com

Tech stack

Artificial Intelligence Airflow Amazon Web Services Amazon S3 Microsoft Azure Continuous Integration Information Engineering Data Governance Data Warehousing DevOps Python (Programming Language) Machine Learning

+15 more

SQL Databases Cloud Platform System State Machines AWS Lambda Cloudformation Containerization Data Lakes Pyspark AWS Glue Data Analytics Apache Kafka Data Management Terraform Data Pipelines Docker

Job description

Design, build, and optimize scalable data pipelines using AWS Glue, Apache Kafka, AWS Lambda, and Step Functions.
Develop and maintain robust data lakes and data warehouses on Amazon S3.
Write infrastructure as code using CloudFormation Templates (CFT).
Collaborate with Data Scientists and ML Engineers to integrate AI/ML models into production-grade data workflows.
Write efficient, reusable, and testable code in Python, SQL, and PySpark.
Ensure data quality, governance, and security across all data platforms.
Monitor and troubleshoot data pipeline performance and reliability.
Participate in architectural discussions and contribute to cloud strategy and best practices.

Requirements

7-10+ years of experience in Data Engineering with a strong focus on AWS Cloud
Proficiency in Python, SQL, and PySpark.
Hands-on experience with the following
AWS Glue
Apache Kafka
Amazon S3
AWS Lambda
AWS Step Functions
CloudFormation Templates (CFT)/Terraform
Basic understanding of AI concepts
Experience with CI/CD tools and DevOps practices in cloud environments.
Strong problem-solving skills and ability to work in a fast-paced, collaborative environment. * AWS certifications (e.g., AWS Certified Data Analytics, Solutions Architect)
Experience with other cloud platforms (Azure, GCP)
Familiarity with containerization (Docker, ECS) and orchestration tools (Airflow, Step functions)
Knowledge of data governance frameworks and compliance standards