> Markdown version of [/jobs/ext/1978402-big-data-engineer](https://www.wearedevelopers.com/jobs/ext/1978402-big-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Big Data Engineer

- **Company:** EXL SERVICE
- **Location:** Pittsburgh, PA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Big Data, Cloudera Impala, Information Engineering, Extract Transform Load (ETL), Data Structures, Data Systems, Distributed Computing Environment, Distributed Systems, Apache Hadoop, Hadoop Distributed File System, Apache Hive, Python (Programming Language), Machine Learning, Standard Sql, Apache Spark, Data Management, Data Pipelines, Control M
- **Published:** August 7, 2026
- **Apply:** https://us.experteer.com/career/view-jobs/big-data-engineer-pittsburgh-pa-usa-58832500

## About the Role

proven using Python, Spark, and the Hadoop ecosystem * Work with Hive, HDFS, Impala to manage and query large-scale data * Manage batch scheduling and orchestration with CA7 or Control-M * Ensure data quality, integrity, and performance across platforms * Collaborate with analysts, scientists, and stakeholders to translate requirements into designs * Troubleshoot complex issues and act as escalation point * Champion coding standards, testing, and documentation * Stay current with emerging technologies, especially AI/ML capabilities, and apply where appropriate Tasks * 7-10 years of data engineering experience with proven technical leadership * Strong Python development for production-grade data solutions * Deep on-premises Big Data / Hadoop ecosystem experience (Hadoop, Hive, Impala, HDFS) * Apache Spark experience for large-scale distributed processing * Hands-on experience with CA7 or Control-M (or comparable schedulers) * Solid understanding of data structures, ETL, and SQL * aaaaa
data experience with large-scale data processing and distributed systems * Ability to make sound architecture/design decisions and mentor others * Exposure to AI/ML concepts or tools with willingness to learn Key requirements *

## Description

Experteer Overview As a Big Data Engineer at EXL, you will own the design, development, and maintenance of scalable on-prem data pipelines that feed analytics and business insights. You'll set technical direction, guide a team of engineers, and collaborate with data analysts, scientists, and stakeholders to translate requirements into robust data solutions. You'll leverage Python, Spark, and the Hadoop ecosystem to build reliable workflows and ensure data quality at scale. This role combines hands-on engineering with architectural ownership to drive high-impact data platforms and stay ahead with emerging AI/ML opportunities. Compensation / Benefits * Lead design, development, and maintenance of scalable data pipelines for ingestion, transformation, and processing of large datasets on-premises * Own architectural decisions and define technical standards for the team * Mentor and support data engineers through reviews, coaching, and hands-on problem-solving * Build and optimize data workflows using Python, Spark, and the Hadoop ecosystem * Work with Hive, HDFS, Impala to manage and query large-scale data * Manage batch scheduling and orchestration with CA7 or Control-M * Ensure data quality, integrity, and performance across platforms * Collaborate with analysts, scientists, and stakeholders to translate requirements into designs * Troubleshoot complex issues and act as escalation point * Champion coding standards, testing, and documentation * Stay current with emerging technologies, especially AI/ML capabilities, and apply where appropriate Tasks * 7-10 years of data engineering experience with proven technical leadership * Strong Python development for production-grade data solutions * Deep on-premises Big Data / Hadoop ecosystem experience (Hadoop, Hive, Impala, HDFS) * Apache Spark experience for large-scale distributed processing * Hands-on experience with CA7 or Control-M (or comparable schedulers) * Solid understanding of data structures, ETL, and SQL * Extensive experience with large-scale data processing and distributed systems * Ability to make sound architecture/design decisions and mentor others * Exposure to AI/ML concepts or tools with willingness to learn Key requirements *


## Related Videos

- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Brewing Tea over the Internet](https://www.wearedevelopers.com/videos/697-brewing-tea-over-the-internet)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Business Strategies for Scaling European Tech](https://www.wearedevelopers.com/videos/1694-business-strategies-for-scaling-european-tech)
- [A Technical Introduction to Bitcoin's 2nd Layer- The Lightning Network](https://www.wearedevelopers.com/videos/15-a-technical-introduction-to-bitcoin-s-2nd-layer-the-lightning-network)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)