Cyber SDC - Data Engineer - Senior - Location Open
Role details
Job location
Tech stack
Job description
We are seeking a Data Engineer with strong AWS cloud expertise to design, build, and maintain scalable data pipelines and data lake solutions. The ideal candidate will have hands-on experience with Python, SQL, and AWS data services, with a proven ability to develop robust data ingestion frameworks, optimize data quality, and support analytics and reporting initiatives. This role requires close collaboration with customers, data engineers, and cross-functional teams to onboard and transform data from multiple sources into a centralized AWS-based data lake., + Develop, deploy, and support AWS Lambda functions using Python to ingest and process data into an Amazon S3-based data lake.
-
Maintain and troubleshoot existing data ingestion solutions, including Lambda functions, Python code, and Terraform infrastructure.
-
Design, develop, and optimize end-to-end data pipelines leveraging AWS services such as Lambda, Step Functions, Glue ETL Jobs, Glue Crawlers, Athena, and S3.
-
Collaborate with customers and stakeholders to onboard new data sources while ensuring compliance with data governance and security requirements.
-
Implement data transformation, normalization, and cleansing processes to improve data quality and usability.
-
Monitor, troubleshoot, and resolve data pipeline issues to ensure reliability and performance.
-
Partner with Data Engineering and Analytics teams to improve existing workflows, enhance code quality, and optimize data processing logic.
-
Support data lake architecture and contribute to best practices for scalable and maintainable data engineering solutions.
-
Document technical designs, workflows, and operational procedures.
Requirements
-
Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field, or equivalent work experience.
-
3+ years of experience in Data Engineering or a related role.
-
Strong proficiency in Python and SQL.
-
Hands-on experience with AWS services, including:
-
AWS Lambda
-
AWS Glue
-
Amazon Athena
-
Amazon S3
-
AWS Step Functions
-
Experience developing and supporting ETL/ELT pipelines in cloud environments.
-
Experience with Terraform