> Markdown version of [/jobs/ext/559363-data-engineer](https://www.wearedevelopers.com/jobs/ext/559363-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** IBA InfoTech Inc.
- **Location:** Charlotte, NC, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Amazon Elastic Compute Cloud, Amazon S3, Batch Processing, Cloud Computing, Continuous Integration, Data Architecture, Extract Transform Load (ETL), Data Mining, IBM DB2, Database Theory, Software Debugging, Distributed Systems, Fault Tolerance, Apache Hadoop, Apache Hive, Identity and Access Management, Machine Learning, Oracle (Applications), Performance Tuning, Software Architecture, Data Processing, Load Balancing, Data Ingestion, Autoscaling, Apache Spark, Amazon Virtual Private Cloud (VPC), AWS ECS, Amazon Relational Database Service, Information Technology, Cloudwatch, Amazon Simple Queue Service (SQS), Data Pipelines
- **Published:** June 14, 2026
- **Apply:** https://ibainfotech.com/data-engineer-3

## About the Role

* 10 plus years of Experience as Data Engineer
 * Expertise in Amazon EC2, Amazon S3, Amazon RDS, VPC, IAM, Amazon Elastic Load Balancing, Auto Scaling, Cloud Front, CloudWatch, SNS, SES, SQS and other services of the AWS family.
 * Experience in working with Oracle and DB2. And making the data to be batch processing using distributed computing.
 * Good knowledge of High-Availability, Fault Tolerance, Scalability, Database Concepts, System and Software Architecture, Security and IT Infrastructure.
 * Expertise in Spark SQL, Tuning and Debugging the Spark Cluster
 * Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modeling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows.
 * Knowledge of tools like Datameer

## Description

* Create a Low-Level implementation plan for the ingestion pipeline.
 * Develop the ingestion pipeline based on HLD/LLD specifications.
 * Define the storage directories in S3 and script the compaction and portioning based on consumption patterns.
 * Work on the issues identified in the data pipeline.
 * Integrate the pipeline jobs to enterprise scheduler and CI/CD Process.
 * Work with support teams to promote it to prod


## Related Videos

- [Data Fabric in Action - How to enhance a Stock Trading App with ML and Data Virtualization](https://www.wearedevelopers.com/videos/253-data-fabric-in-action-how-to-enhance-a-stock-trading-app-with-ml-and-data-virtualization)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Celery on AWS ECS - the art of background tasks & continuous deployment](https://www.wearedevelopers.com/videos/561-celery-on-aws-ecs-the-art-of-background-tasks-continuous-deployment)
- [WeAreDevelopers LIVE - CSS is DOOMed](https://www.wearedevelopers.com/videos/1838-wearedevelopers-live-css-is-doomed)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Tips, Techniques, and Common Pitfalls Debugging Kafka](https://www.wearedevelopers.com/videos/838-tips-techniques-and-common-pitfalls-debugging-kafka)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)