AWS Data Engineer

Purple Drive Technologies LLC
Charlotte, United States of America
22 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Charlotte, United States of America

Tech stack

Amazon Web Services (AWS)
Amazon Web Services (AWS)
Data analysis
Cloud Database
Cloud Engineering
Data Governance
Data Integrity
ETL
Data Transformation
Data Security
DevOps
Fraud Prevention and Detection
Python
Enterprise Messaging Systems
Meta-Data Management
Performance Tuning
E2e Testing
Cloud Services
Data Streaming
Amazon Connect
Parquet
AWS Lambda
Servicebus
Event Driven Architecture
PySpark
Infrastructure Automation Frameworks
Deployment Automation
Amazon Web Services (AWS)
Data Analytics
Amazon Web Services (AWS)
Kafka
Terraform
Stream Processing
Data Pipelines

Job description

  • AWS Lambda
  • Amazon S3
  • Amazon EventBridge
  • Amazon Kinesis (Data Streams & Firehose)
  • AWS Glue (ETL & Crawlers)
  • AWS Step Functions
  • Amazon Athena
  • Amazon Connect
  • Amazon Macie
  • Experience designing and implementing AWS Data Lake architectures, * Design and implement scalable AWS-based data engineering solutions
  • Develop real-time event-driven architectures using EventBridge, Kafka, and Kinesis
  • Build and optimize ETL pipelines using AWS Glue, Python, and PySpark
  • Configure and manage AWS Glue Crawlers for automated schema discovery and metadata management
  • Design and maintain data lakes using Amazon S3, Parquet, Glue Catalog, and Athena
  • Develop integrations between Customer Profiles, messaging platforms, and downstream systems
  • Build automated workflows using AWS Step Functions
  • Provision and manage AWS infrastructure using Terraform
  • Implement CI/CD pipelines for automated deployment and infrastructure management
  • Perform end-to-end testing, monitoring, troubleshooting, and performance tuning
  • Optimize cloud architecture for scalability, reliability, cost efficiency, and security
  • Maintain technical documentation, architecture diagrams, and operational runbooks

Requirements

  • Strong proficiency in:
  • Python
  • PySpark
  • Experience developing:
  • ETL/ELT Pipelines
  • Data Transformation Workflows
  • Real-Time Data Processing
  • Hands-on experience with:
  • Parquet File Format
  • Glue Crawlers
  • AWS Glue Data Catalog

Streaming & Event-Driven Architecture

  • Experience with:
  • Event-Driven Architecture
  • Publish/Subscribe (Pub/Sub) Models
  • Apache Kafka
  • Amazon EventBridge
  • Kinesis Data Streams
  • Kinesis Firehose

DevOps & Infrastructure

  • Hands-on experience with:
  • Terraform (Infrastructure as Code)
  • CI/CD Pipelines
  • Cloud Automation
  • Experience deploying and managing AWS infrastructure

Data Analytics & Security

  • Experience using:
  • Amazon Athena
  • AWS Glue Catalog
  • Amazon Macie
  • Strong understanding of:
  • Data Governance
  • Data Security
  • Data Integrity
  • Performance Optimization, * Experience in fraud detection or financial services platforms
  • AWS Certifications (Solutions Architect, Data Engineer, or Developer Associate)
  • Experience with:
  • Lakehouse Architectures
  • Streaming Analytics
  • Cloud-Native Data Platforms
  • Knowledge of DevOps best practices and Infrastructure Automation

Apply for this position