> Markdown version of [/jobs/ext/971634-apache-spark-developer](https://www.wearedevelopers.com/jobs/ext/971634-apache-spark-developer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Apache Spark Developer - **Company:** Absolute Business Solutions Corp - **Location:** Aurora, CO, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Amazon S3, Business Analytics Applications, Microsoft Azure, Bash Shell, Big Data, Cloud Computing, Continuous Integration, Information Engineering, DevOps, Distributed Computing Environment, Distributed Data Store, Python (Programming Language), Performance Tuning, SQL Databases, Data Streaming, Parquet, Data Processing, Scripting, Feature Engineering, Data Ingestion, Apache Spark, Git, Containerization, Data Lakes, Pyspark, Kubernetes, Machine Learning Operations, Terraform, Stream Processing, Data Pipelines, Docker, Service Stack, Jenkins - **Published:** June 16, 2026 - **Apply:** https://www.clearancejobs.com/jobs/8980239/apache-spark-developer ## About the Role * TS/SCI (eligibility) with ability/willingness to obtain/maintain counterintelligence polygraph * Bachelor's degree plus 5 years' experience in data engineering or Spark development (will entertain additional years' experience in lieu of degree) * Strong hands-on experience with: o Apache Spark (mandatory) o Python (PySpark) o Data processing at scale * Experience working with: o Parquet and/or Delta Lake o Distributed data systems * Familiarity with: o Docker / containerization o Kubernetes (basic to intermediate experience) * Experience with object storage systems (e.g., S3 or equivalent) * Strong troubleshooting and performance tuning skills * Proficiency in Bash or scripting Preferred Qualifications: * Experience with Scala for Spark development * Experience with Structured Streaming in production environments * Familiarity with Iceberg or lakehouse architectures * Experience with CI/CD pipelines (Jenkins, Git) * Exposure to Terraform or Infrastructure as Code * Experience supporting AI/ML data pipelines * Prior experience supporting NGA, IC, or DoD programs ## Description We are actively hiring a Secret or Top Secret/ TS/SCI-cleared Apache Spark Developer to support NGA's Data Modernization Services (DMS) mission by building and optimizing large-scale data processing pipelines. This role focuses on developing high-performance Spark applications within a containerized, Kubernetes-based environment, supporting mission analytics, data exploitation, and AI/ML integration. The ideal candidate thrives in distributed data environments, understands performance tuning deeply, and can operate effectively in secure, air-gapped systems. This role is on-site/flexible hours in Herndon, VA; Springfield, VA; St. Louis, MO; or Aurora, CO. Clearance Required for this role: Min. Secret but TS/SCI eligibility with willingness/ability to obtain CI polygraph preferred. Core Technology Stack Data / Processing * Apache Spark (PySpark, Scala) * Delta Lake, Parquet * Structured Streaming Infrastructure * Kubernetes (execution environment) * Docker Storage / Cloud (Abstracted) * S3 / object storage * AWS / GCP / Azure (environment-dependent) DevOps (Exposure Level) * Git, Jenkins (CI/CD) Languages * Python (PySpark) * Scala (preferred) * Bash / scripting, * Design, develop, and maintain Apache Spark pipelines (batch and streaming) using PySpark and/or Scala * Process and transform large-scale datasets using modern data lake architectures (Delta Lake, Parquet) * Optimize Spark jobs for performance, including: o Partitioning strategies o Shuffle optimization o Memory tuning o File sizing and storage efficiency * Implement Structured Streaming pipelines for near real-time data processing * Develop and deploy Spark applications within containerized environments (Docker) * Execute workloads in Kubernetes clusters, supporting scalable and distributed processing * Integrate Spark pipelines with downstream systems, including: o Analytics platforms (SQL, notebooks) o AI/ML workflows and feature engineering pipelines * Support data ingestion and storage in object-based systems (e.g., S3-compatible storage) * Troubleshoot data pipeline failures and ensure reliability in mission-critical environments * Operate within secure, air-gapped environments, including ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)