PySpark Data Engineer

We are seeking a Data Engineer to support a high-impact defense program focused on delivering advanced data solutions for mission-critical operations. This position offers the opportunity to work with modern cloud and big data technologies, building scalable data architectures and pipelines that support enterprise analytics, data integration, and decision-making across a secure environment.

The ideal candidate will bring strong expertise in Databricks, PySpark, SQL, and DevOps practices, along with experience designing and optimizing large-scale data processing solutions., Architect, develop, and maintain scalable data pipelines supporting enterprise data ingestion, processing, and transformation. Design and optimize ETL workflows for large, complex datasets using PySpark and SQL. Build and manage data engineering solutions within the Databricks platform. Implement performance tuning strategies to improve reliability, scalability, and efficiency of data processing workloads. Develop and maintain CI/CD pipelines to support automated testing, deployment, and monitoring. Containerize and deploy applications using modern technologies such as Docker and Kubernetes. Collaborate with software engineers, analysts, architects, and stakeholders to deliver scalable data solutions. Apply DevOps principles and best practices to streamline development and deployment processes. Support data governance, data quality, and system performance initiatives. Troubleshoot and resolve data pipeline, platform, and performance-related issues.

Active TS/SCI clearance preferred; candidates with an active Secret clearance will also be considered. Strong experience working within Databricks environments.

Advanced proficiency with: PySpark SQL ETL development and optimization Data pipeline design and engineering Experience implementing CI/CD pipelines and DevOps best practices. Hands-on experience with Git version control. Experience deploying and managing containerized applications using Docker and Kubernetes. Strong understanding of distributed data processing and enterprise-scale data architectures. Ability to optimize and tune high-volume data processing workloads. Excellent problem-solving and analytical skills., Experience supporting federal or defense programs. Databricks Certified Data Engineer Associate or Professional certification. Experience working with financial management data. Qlik dashboard and reporting experience. Experience with secure federal data platforms such as Advana or Warfighter Data Platform (WDP). Familiarity with cloud-native data engineering architectures and modern data integration frameworks.

Technical Environment Databricks PySpark SQL Docker Kubernetes Git CI/CD Pipelines DevOps Enterprise Data Platforms Qlik Analytics