> Markdown version of [/jobs/ext/2721250-data-engineer-big-data-spark-scala-hadoop-sql-python](https://www.wearedevelopers.com/jobs/ext/2721250-data-engineer-big-data-spark-scala-hadoop-sql-python). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - Big Data (Spark/Scala, Hadoop, SQL, Python) - **Company:** Computer Task Group, Inc - **Location:** Piscataway, NJ, United States - **Salary:** $104,000.0 - $124,800.0 - **Contract:** Temporary contract - **Skills:** Agile Methodology, Big Data, Cloudera Impala, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Mining, Database Queries, Software Debugging, Distributed Computing Environment, Distributed Data Store, Apache Hadoop, Apache Hive, Python (Programming Language), Open Source Technology, Operational Databases, Performance Tuning, Scala (Programming Language), Software Engineering, SQL Databases, Computational Statistics, Enterprise Data Management, Data Processing, Scripting, Freeform SQL, Sql Optimization, Apache Spark, Information Technology, Api Design, Data Pipelines - **Published:** September 4, 2026 - **Apply:** https://dejobs.org/x/x/2EF1DAB82EB24576BC75495B89769DE3/job/ ## About the Role * Strong hands-on experience with Apache Spark and Scala for distributed data processing. * Strong experience with Hadoop, Hive, and Impala and enterprise Big Data platforms. * Advanced SQL skills, including complex queries, joins, transformations, reconciliation, and performance tuning. * Strong Python scripting and data-processing experience. * Strong application development, coding, debugging, and problem-solving skills. * Knowledge of ETL/data pipelines, data quality, and data reconciliation. * Understanding of analytics libraries, statistical computing, and open-source data-processing technologies. * Experience with Agile development methodologies and CI/CD practices is preferred., * Demonstrated experience developing and supporting applications in enterprise Big Data environments. * Hands-on experience working with large datasets and distributed data-processing technologies. * Experience designing, developing, testing, and debugging complex code. * Experience with large-volume transactional or financial datasets is preferred. * Banking or financial services experience is preferred. * Experience supporting production data applications and resolving technical issues is a plus. Education: * Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent combination of education and experience. Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required. ## Description * Design, develop, test, and maintain scalable data-processing applications using Apache Spark and Scala. * Develop and support Big Data applications and data pipelines utilizing Hadoop, Hive, and Impala. * Write complex SQL queries for data extraction, transformation, reconciliation, analysis, and performance optimization. * Develop Python scripts and utilities to support data processing, automation, and data engineering activities. * Design, build, and maintain scalable ETL/data pipelines for large-volume datasets. * Analyze, troubleshoot, and debug complex application and data-processing code. * Support data quality, validation, reconciliation, and integrity across enterprise data platforms. * Collaborate with application developers, data engineers, analysts, and business stakeholders in an Agile environment. * Contribute to API development and application solutions supporting Big Data platforms and analytics. * Apply knowledge of analytics libraries, open-source technologies, statistical computing, and Big Data processing frameworks as appropriate. ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [JavaScript? No. Java Scripts! - Scripting with Java](https://www.wearedevelopers.com/videos/2094-javascript-no-java-scripts-scripting-with-java) - [API Design - Getting Started](https://www.wearedevelopers.com/videos/33-api-design-getting-started) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)