> Markdown version of [/jobs/ext/2694581-data-engineer](https://www.wearedevelopers.com/jobs/ext/2694581-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** The Aes Group, Inc - **Location:** Blue Bell, PA, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Agile Methodology, Data Analysis, Big Data, Databases, Data Validation, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Transformation, Data Mining, Data Normalization, Relational Databases, Software Debugging, Linux, File Systems, Apache Hadoop, Apache Hive, Linux System Administration, Microsoft SQL Server, Oracle (Applications), Standard Sql, DataOps, Scala (Programming Language), Scaled Agile Framework, Shell Script, SQL Databases, Transact-SQL, Data Ingestion, Sql Optimization, Apache Spark, Data Lakes, Information Technology, Data Management, Data Pipelines - **Published:** September 3, 2026 - **Apply:** https://www.dice.com/job-detail/a275d61c-a5d5-4d3c-9e92-a52446394c00 ## About the Role * MUST: 5+ years experience with data aggregation, standardization, linking, quality check mechanisms, and reporting. * MUST: 5+ years experience with big data technologies like Hadoop and Spark. * MUST: 5+ years experience with RDBMS (Oracle, MS SQL Server) and using SQL or other data integration/ETL tools. * MUST: Solid understanding of Linux environments; strong knowledge of shell scripting and file systems. * Bachelor's degree in relevant field such as Computer Science, Engineering, a related field, with 8-10 years of industry experience., SQL Queries ETL Process - End to End Linux experience Healthcare experience preferred Hadoop - nice to have Spark Advanced SQL experience EST preferred time zone Desired Skills & Experience - * Experience with healthcare data preferably in a data operations role. * Data expert with the ability to debug data issues, identify root causes and fix the data issues in a fast-paced environment. * Experience operating in an Agile environment. ## Description As a Senior Data Engineer, you will work with the team to build complex data pipelines using a new cutting-edge Data Ingestion Platform as well as several new and exciting products in the Big Data/Advanced Analytics space. You will operate in a fast-paced environment where multiple project deliverables are coordinated within specified deadlines. The primary responsibility of this role is to deliver accurate and timely data across various products to enable revenue for the company. The ideal candidate would have data analysis and data pipeline development experience using big data tools (like Spark, Scala, Hive), relational databases, intermediate SQL skills. Healthcare knowledge is a plus., Principal Responsibilities and Essential Duties: * Build data pipelines as per data transformation specifications to convert source data to be loaded into data lake using proprietary big data processing platform * Supports and improves current data ingestion processes for our proprietary healthcare data applications and systems * Develop and maintain data engineering processes using a variety of tools including T-SQL, Spark and Scala, and shell scripting. Generally focused on data ingestion for healthcare data management, data validation, statistical report generation, and program validation. * Develop tools and techniques for improving process efficiencies and data performance. * Review & test the data to ensure accuracy & validity of the data prior to uploading the data to the data lake. * Data Troubleshooting and Analysis * Perform data analysis, data mining and investigations and identify root cause of issues using several cutting-edge data analysis tools. * Work with Technical Operations to troubleshoot complex database issues related to the entire environment including OS, storage, and servers. Provide off hours support to resolve production issues when necessary * Mentor junior team members in data engineering and quality best practices * Oversees the delivery of business priorities in a Scaled Agile Framework (SAFe) environment ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Docker exec without Docker](https://www.wearedevelopers.com/videos/1094-docker-exec-without-docker) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)