> Markdown version of [/jobs/ext/3006645-big-data-spark-engineer](https://www.wearedevelopers.com/jobs/ext/3006645-big-data-spark-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Big Data/Spark Engineer

- **Company:** Software Guidance & Assistance Inc
- **Location:** United States
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), Agile Methodology, Artificial Intelligence, Build Automation, Automation of Tests, Big Data, Configuration Management, Databases, Data Integration, Distributed Systems, Apache Hadoop, Apache Hive, Python (Programming Language), Object-Oriented Software Development, Scrum Methodology, Software Engineering, Data Ingestion, GitHub Copilot, Apache Spark, Data Pipelines
- **Published:** September 19, 2026
- **Apply:** https://www.thejobnetwork.com/job/fddd7dce-6fe1-4e29-bafc-e4e8e38385b3/big-dataspark-engineer

## About the Role

* Bachelor's degree in Computer Science, Information Systems or related discipline with at least five (5) years of related experience, or equivalent training and/or work experience; Master's degree and past Financial Services industry experience preferred.
\n

 * Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions.
\n

 * Past experience with developing enterprise quality solutions in an iterative or Agile environment.
\n

 * Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
\n

 * Experience with object oriented programming languages such as Java, Scala or Python.
\n, * AI Tool Proficiency: Hands-on experience with AI development tools (GitHub Copilot, Q Developer, ChatGPT, Claude, etc.).
\n

 * Technical Background: Strong software development background with ability to contribute to technical discussions.
\n

 * Agile Methodology: Extensive experience with Scrum, Kanban, and continuous improvement practices.
\n

## Description

Seeking a Big Data Engineer to design, develop, and optimize large-scale data processing systems. Will work closely with cross-functional teams to architect data pipelines, implement data integration solutions, and ensure the performance, scalability, and reliability of big data platforms. Ideally will have deep expertise in distributed systems, cloud platforms, and modern big data technologies such as Hadoop, Spark etc.\n
\n, * Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).


## Related Videos

- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Kubernetes and Microservices with Multi-Model Databases](https://www.wearedevelopers.com/videos/382-kubernetes-and-microservices-with-multi-model-databases)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases](https://www.wearedevelopers.com/videos/1146-fault-tolerance-and-consistency-at-scale-harnessing-the-power-of-distributed-sql-databases)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Branch your database like your code:  How schema changes and pull requests go hand in hand](https://www.wearedevelopers.com/videos/350-branch-your-database-like-your-code-how-schema-changes-and-pull-requests-go-hand-in-hand)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)