Databricks Data Engineer - Clearance Required

TSGi is seeking a Senior Databricks Data Engineer to support a large-scale federal technology modernization program. This is a remote position supporting the development and migration of cloud-based data solutions.

The Data Engineer will build and operate ingestion pipelines and Bronze, Silver, and Gold data layers within Databricks, helping migrate data and business logic from legacy environments into a modern cloud architecture., * Build, operate, and maintain ingestion pipelines within Databricks.

• Develop and support Bronze, Silver, and Gold data pipelines using Medallion Architecture.
• Migrate data and business logic from legacy systems into modern cloud-based environments.
• Gather and analyze technical and business requirements.
• Design and develop batch, incremental, and streaming ingestion pipelines.
• Monitor data jobs and troubleshoot pipeline and production issues.
• Provide post-go-live production support and break/fix assistance.
• Document technical processes, data workflows, and procedures.
• Collaborate with technical teams and stakeholders in an Agile environment.

Cloud Computing, Data Migration, Cloud Architecture, United States Citizen, DoD Secret Clearance, Data Management, Technical Analysis, Business Analysis, Identify Issues, Production Support, * Per our Federal Government Contract, Active DoD Secret clearance and U.S. citizenship required.

• Strong experience in data engineering, ETL/ELT development, or data warehousing.
• Strong PySpark and/or Spark SQL skills with solid SQL fundamentals.
• Experience with Databricks and Medallion Architecture, including Bronze, Silver, and Gold layering.
• Experience with Delta Lake, including ACID transactions, schema evolution, time travel, and MERGE/upsert patterns.
• Experience designing batch and incremental/streaming ingestion pipelines.
• Familiarity with Agile processes.
• Excellent written and verbal communication skills.
• Strong interpersonal, organizational, and prioritization skills.
• Databricks certification or additional hands-on Databricks experience preferred.
• DoD, Navy, or other federal experience preferred.
• AWS GovCloud experience preferred.
• Experience migrating code from non-accredited to accredited AWS environments is a plus.