> Markdown version of [/jobs/ext/2044039-senior-aws-data-engineer-real-time-streaming](https://www.wearedevelopers.com/jobs/ext/2044039-senior-aws-data-engineer-real-time-streaming). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior AWS Data Engineer (Real-Time / Streaming)

- **Company:** Infinite Computer Solutions Inc
- **Location:** Atlanta, GA, United States
- **Experience:** Expert
- **Contract:** Temporary to permanent
- **Skills:** Amazon Web Services, Amazon S3, Cloud Computing, Continuous Integration, Data Integrity, Extract Transform Load (ETL), Fraud Prevention and Detection, Monitoring of Systems, Python (Programming Language), Enterprise Messaging Systems, Data Streaming, Systems Architecture, Systems Integration, Amazon Connect, Parquet, Data Storage Technologies, Delivery Pipeline, Servicebus, Event Driven Architecture, Data Lakes, Pyspark, AWS Glue, AWS Data Analytics, Apache Kafka, Terraform, Stream Processing, Data Pipelines
- **Published:** August 13, 2026
- **Apply:** https://www.careerjet.com/jobad/usc6bd952a78b9a9ba863ff7ca39330bde

## About the Role

* Experience in:

 * Glue Crawlers for schema discovery and cataloging
 * Parquet-based data storage
 * Building scalable data pipelines
 * Strong understanding of event-driven architectures (Pub/Sub model)
 * Hands-on experience with Terraform (Infrastructure as Code)
 * Familiarity with CI/CD tools and automation pipelines

Preferred Qualifications

 * AWS Certifications:

 * AWS Certified Developer
 * AWS Solutions Architect
 * Experience with:

 * Messaging platforms like Kafka and Amazon EventBridge
 * Designing real-time data processing systems
 * Using Glue Crawlers + Athena for data lake architectures

## Description

* Develop a comprehensive plan for migrating near real-time fraud detection campaigns from on-premises systems to AWS.
 * Design and implement event-driven architectures to process inbound dialer data (fraud events) using services such as Amazon EventBridge, Kafka, Kinesis Data Streams, and Kinesis Firehose.
 * Build and manage scalable data pipelines using AWS Glue (ETL jobs, Crawlers), PySpark, and Python for data ingestion, transformation, and processing.
 * Configure and manage Glue Crawlers to automatically discover schemas and update the Data Catalog.
 * Store and optimize data using Parquet format and enable analytics through Amazon Athena for efficient querying.
 * Develop integrations between Customer Profiles and messaging platforms to automatically trigger profile updates and downstream processes.
 * Implement automation to trigger fraud-related outbound calls based on updates in customer profiles.
 * Design and orchestrate workflows using AWS Step Functions to manage complex processing pipelines.
 * Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
 * Optimize system architecture for scalability, reliability, cost-efficiency, and ensure data integrity and security.
 * Conduct end-to-end testing of the entire framework to validate functionality, performance, and reliability.
 * Deploy, automate, and manage resources using CI/CD pipelines.
 * Continuously monitor system performance and implement optimizations post-deployment.
 * Maintain detailed documentation of architecture, workflows, and operational processes.

Technical Skills

 * Strong expertise in AWS services including:

 * Lambda
 * S3
 * EventBridge
 * Kinesis (Data Streams & Firehose)
 * Glue (ETL + Crawlers)
 * Step Functions
 * Amazon Connect
 * Athena
 * Macie
 * Proficient in:

 * Python
 * PySpark


## Related Videos

- [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions)
- [How we built an AI-powered code reviewer in 80 hours](https://www.wearedevelopers.com/videos/1511-how-we-built-an-ai-powered-code-reviewer-in-80-hours)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Dev Digest 162: AI careers, MCP, AWS best practices & floppy sweaters](https://www.wearedevelopers.com/magazine/571-dev-digest-162-ai-careers-mcp-aws-best-practices-floppy-sweaters)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)