AWS Data Engineer Insurance Domain
Yotta Systems Inc
Philadelphia, United States of America
yesterday
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
SeniorJob location
Remote
Philadelphia, United States of America
Tech stack
Agile Methodologies
Airflow
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Big Data
Cloud Computing
Code Review
Databases
ETL
Data Systems
Data Warehousing
Database Queries
Dimensional Modeling
Distributed Computing Environment
Revision Control Systems
Identity and Access Management
Python
Scrum
Data Processing
Snowflake
Spark
State Machines
Infrastructure as Code (IaC)
GIT
PySpark
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Data Management
Cloudwatch
Terraform
Data Pipelines
Databricks
Job description
- Design, develop, and optimize scalable data pipelines on AWS.
- Build and maintain ETL/ELT workflows for large-scale data processing.
- Integrate data from multiple enterprise systems into AWS data platforms.
- Optimize data pipelines for performance, scalability, and reliability.
- Collaborate with business analysts, data scientists, and application teams to deliver data solutions.
- Ensure data quality, governance, security, and compliance across data platforms.
- Troubleshoot production issues and implement performance improvements.
- Participate in code reviews, sprint planning, and Agile ceremonies
Requirements
- 8+ years of experience as a Data Engineer.
- Strong hands-on experience with AWS services such as S3, Glue, Lambda, EMR, Redshift, Athena, IAM, CloudWatch, and Step Functions.
- Expertise in Python and PySpark for data processing and transformation.
- Strong SQL skills with experience in relational and cloud-based databases.
- Experience building and maintaining ETL/ELT pipelines.
- Hands-on experience with Apache Spark and distributed data processing.
- Experience with Airflow, AWS Glue Workflows, or other orchestration tools.
- Knowledge of data warehousing concepts and dimensional modeling.
- Experience with version control tools such as Git and CI/CD pipelines.
- Familiarity with Agile/Scrum development methodologies
- Experience working with policy, claims, underwriting, billing, customer, or actuarial data.
- Understanding of insurance business processes and regulatory requirements, * AWS Certified Data Engineer or AWS Certified Solutions Architect.
- Experience with Databricks is a plus.
- Experience with Snowflake is a plus.
- Knowledge of Terraform or Infrastructure as Code (IaC) is preferred.
- Strong communication and stakeholder management skills