Databricks Data Engineer

Location: Arlington, VA Salary: $55.00 USD Hourly - $65.00 USD Hourly Description: The Judge Group is currently seeking a Databricks Data Engineer to support a newly awarded contact with a large federal agency. This is a hybrid position. It is a w2 only opportunity, no third parties.

What You Will Do

• Build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines.
• Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake.
• Implement source-to-target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data-quality requirements.
• Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products.
• Create reconciliation routines and support source-to-target validation, exception analysis, and defect resolution.
• Build unit, integration, regression, performance, and data-quality tests and document technical results.
• Support Git-based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization.
• Analyze and resolve job failures, cluster-performance issues, pipeline bottlenecks, schema changes, and data-quality problems.
• Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source-system teams.
• Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves.
• Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews.

• Bachelor’s degree in computer science, data engineering, information systems, engineering, analytics, or a related field.
• Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline.
• Hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration.
• Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers.
• Experience with enterprise data migration, source-to-target mapping, reconciliation, validation, and data-quality controls.
• Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting.
• Strong analytical, problem-solving, documentation, collaboration, and communication skills.
• Ability to obtain and maintain a Federal Public Trust.

What Would Be Nice to Have

• Databricks Certified Data Engineer Associate or Professional certification.
• Experience with Unity Catalog, Delta Live Tables, Databricks Workflows, or Auto Loader.
• Experience developing APIs, integration pipelines, or data-consumption services.
• Experience with Azure DevOps, GitHub Actions, Jenkins, Terraform, or related engineering tools.
• Experience with streaming technologies such as Kafka or Azure Event Hubs.
• Experience migrating HR, ERP, workforce, financial, or sensitive enterprise data.
• Experience supporting federal agencies, regulated organizations, or public-sector modernization.
• Experience with Microsoft Azure or Amazon Web Services.