AWS Data Engineer

Diverse Lynx LLC

Charlotte, NC, United States

10 days ago

Apply on www.careerjet.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Compensation

$80,000.0 - $165,000.0

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Amazon Web Services Amazon S3 Cloud Computing Continuous Integration Data Integrity Extract Transform Load (ETL) Fraud Prevention and Detection Monitoring of Systems Python (Programming Language) Lynx Enterprise Messaging Systems Data Streaming

+16 more

Systems Architecture Systems Integration Amazon Connect Parquet Data Storage Technologies Delivery Pipeline Servicebus Event Driven Architecture Data Lakes Pyspark AWS Glue AWS Data Analytics Apache Kafka Terraform Stream Processing Data Pipelines

Job description

Develop a comprehensive plan for migrating near real-time fraud detection campaigns from on-premises systems to AWS.
Design and implement event-driven architectures to process inbound dialer data (fraud events) using services such as Amazon EventBridge, Kafka, Kinesis Data Streams, and Kinesis Firehose.
Build and manage scalable data pipelines using AWS Glue (ETL jobs, Crawlers), PySpark, and Python for data ingestion, transformation, and processing.
Configure and manage Glue Crawlers to automatically Client schemas and update the Data Catalog.
Store and optimize data using Parquet format and enable analytics through Amazon Athena for efficient querying.
Develop integrations between Customer Profiles and messaging platforms to automatically trigger profile updates and downstream processes.
Implement automation to trigger fraud-related outbound calls based on updates in customer profiles.
Design and orchestrate workflows using AWS Step Functions to manage complex processing pipelines.
Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
Optimize system architecture for scalability, reliability, cost-efficiency, and ensure data integrity and security.
Conduct end-to-end testing o f the entire framework to validate functionality, performance, and reliability.
Deploy, automate, and manage resources using CI/CD pipelines.
Continuously monitor system performance and implement optimizations post-deployment.
Maintain detailed documentation of architecture, workflows, and operational processes.

Must Have Technical/Functional Skills

Strong expertise in AWS services including:
Lambda
S3
EventBridge
Kinesis (Data Streams & Firehose)
Glue (ETL + Crawlers)
Step Functions
Amazon Connect
Athena
Macie

Proficient in:

Python
PySpark
Experience in:
Glue Crawlers for schema discovery and cataloging
Parquet-based data storage
Building scalable data pipelines
Strong understanding of event-driven architectures (Pub/Sub model)
Hands-on experience with Terraform (Infrastructure as Code)
Familiarity with CI/CD tools and automation pipelines
Messaging platforms like Kafka and Amazon EventBridge
Designing real-time data processing systems
Using Glue Crawlers + Athena for data lake architectures

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

Requirements

AWS Data Engineer Charlotte, NC Fulltime $135k Job Description Must Have Technical/Functional Skills Technical Skills Strong expertise in AWS services including: L…