> Markdown version of [/jobs/ext/2581930-data-engineer](https://www.wearedevelopers.com/jobs/ext/2581930-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** Bartech Staffing
- **Location:** Charlotte, NC, United States
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Airflow, Amazon Web Services, Amazon S3, Business Logic, Continuous Integration, Data Deduplication, Data Governance, Data Integrity, Extract Transform Load (ETL), Data Warehousing, Github, Identity and Access Management, Python (Programming Language), Operational Databases, Regression Testing, Standard Sql, Amazon Simple Notification Service (SNS), User Environment Management, Delivery Pipeline, State Machines, Pytest, Data Lakes, Pyspark, Apache Kafka, Cloudwatch, Amazon Simple Queue Service (SQS), Terraform, Data Pipelines, Confluent, Amazon Redshift
- **Published:** August 11, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=ae80f8c301151b6f

## About the Role

* 2+ years of AWS experience
 * AWS services: S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, Redshift
 * Experience with Kafka, preferably Confluent Kafka
 * Experience with Lake Formation, Amazon Redshift and Amazon Athena
 * Strong SQL and data modeling skills, executing ETL processes tailored for data warehousing
 * Competence in developing and refining data pipelines within AWS
 * Extensive understanding of database management fundamentals
 * Tools and Languages: Python (good experience in PySpark), SQL
 * Infrastructure as Code technology: Terraform
 * DevOps pipeline (CI/CD): GitHub
 * Deep knowledge of IAM roles and policies
 * Experience with AWS workflow orchestration tools like Airflow or Step Functions

## Description

* Collaborate with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) to understand requirements and use cases, outline technical scope, and deliver technical solutions
 * Collaborate with Data and Solution architects on key technical decisions
 * Develop data pipelines with focus on long-term reliability and maintaining high data quality
 * Design data lake and warehousing solutions with the end-user in mind, ensuring ease of use without compromising on performance
 * Manage and resolve issues in production data warehouse environments on AWS

Core Experience and Abilities:

 * Perform hands-on development and peer review for certain components and tech stack
 * Set up development instances and migration paths with required security, access, and roles
 * Develop components and related processes (e.g., data pipelines, ETL processes, workflows)
 * Build new data pipelines, identify existing data gaps, and provide automated solutions to deliver analytical capabilities and enriched data to applications
 * Implement data pipelines with attentiveness to durability and data quality
 * Implement data warehousing products with focus on end-user experience (ease of use with appropriate performance)
 * Implement data quality frameworks and validation rules (e.g., schema validation, null checks, referential integrity, deduplication)
 * Design and implement automated data tests for ETL pipelines using Python, PySpark, and SQL
 * Write unit, integration, and regression tests for data pipelines (e.g., pytest-based testing for transformations and business rules)
 * Demonstrate familiarity with data observability and monitoring concepts, including freshness, volume, and anomaly detection
 * Understand data reconciliation and source-to-target validation techniques to ensure business logic accuracy Embed data quality checks into CI/CD pipelines to prevent defective data from reaching downstream consumers
*


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)
- [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Dev Digest 162: AI careers, MCP, AWS best practices & floppy sweaters](https://www.wearedevelopers.com/magazine/571-dev-digest-162-ai-careers-mcp-aws-best-practices-floppy-sweaters)