> Markdown version of [/jobs/ext/2918843-data-engineer](https://www.wearedevelopers.com/jobs/ext/2918843-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** SGS Consulting
- **Location:** United States (Remote available)
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon S3, Information Engineering, Data Systems, Software Debugging, Apache Hive, Python (Programming Language), SQL Databases, Data Processing, Information Technology, Presto, Data Pipelines
- **Published:** September 15, 2026
- **Apply:** https://www.dice.com/job-detail/fb80ef61-36c9-4421-a2b5-1983cca3e43a

## About the Role

* Strong Python, SQL/Presto for large-scale batch data processing.
 * hands-on data pipeline and workflow engineering orchestration, restart ability, idempotency, debugging at scale.
 * practical data-quality and validation rigor (schema checks, dedupe, threshold gating) with clear documentation habits.

Nice-to-Have Skills

 * Media/video data processing decode, clipping, frame extraction, embeddings.
 * Experience with AI coding agents / prompt-and-skill authoring to automate repetitive data workflows.

Years of Experience: 04 - 05 Years

Education: BS in Computer Science, Data Engineering, or a related technical field; equivalent hands-on data pipeline experience accepted in lieu of a degree. Advanced degree not required.

## Description

* The main function of the Data Engineer is to develop, evaluate, test, and maintain architectures and data solutions within our organization. The typical Data Engineer executes plans, policies, and practices that control, protect, deliver, and enhance the value of the organization s data assets.

Day-to-day responsibilities

 * Build and maintain data mitigation pipelines that scan, classify, and remediate research datasets before research use.
 * Create and manage Hive tables and namespaces for mitigated outputs.
 * Move data across Manifold, S3, and NFS
 * Debug and triage pipeline failures (workflow conflicts, rate limits, cross-namespace query errors) and document runbooks.
 * Request, track, and validate dataset read/write access grants for mitigation workflows.
 * Author and iterate on agentic skills/automation (e.g., fdp-mitigate) to make mitigation repeatable and self-service.
 * Partner with AI researchers and XFN reviewers on dataset readiness, and report status to the data engineering lead

Key Projects: AI dataset mitigations (e.g. CSAM image/video mitigation. copyright) using agentic tooling. MAANG Experience Highly Wanting.


## Related Videos

- [WeAreDevelopers LIVE - CSS is DOOMed](https://www.wearedevelopers.com/videos/1838-wearedevelopers-live-css-is-doomed)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Hate organising your photos? Try it with 5 Terabytes](https://www.wearedevelopers.com/videos/79-hate-organising-your-photos-try-it-with-5-terabytes)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Building Systems that Last](https://www.wearedevelopers.com/videos/1389-building-systems-that-last)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)