Databricks Engineer

• Build and maintain production data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake.

• Implement and manage medallion architecture (bronze/silver/gold).

• Administer Databricks workspaces: cluster provisioning, job scheduling, access controls, Unity Catalog, and notebook management.

• Manage AWS cloud infrastructure (S3, Lambda, IAM, Aurora, Glue).

• Provision environments using Terraform IaC.

• Implement data quality controls: validation rules, completeness checks, anomaly detection, and lineage tracking.

• Support DevSecOps CI/CD pipelines with embedded security gates.

• Support FISMA/FedRAMP compliance, ATO reassessment, and POA&M resolution.

• 8+ years data engineering / platform administration experience.

• 4+ years hands-on Databricks experience.

• Production experience with PySpark, Spark SQL, and Delta Lake.

• AWS cloud experience (S3, IAM, Lambda, Glue, Aurora).

• Terraform or equivalent IaC experience.

• CI/CD pipeline experience (Jenkins, GitLab CI, or equivalent).

• Python and SQL proficiency.

• Agile/Scrum experience.

• U.S. Citizenship.

• Bachelor’’s degree in CS, IT, Data Science, or related (or equivalent experience).

Preferred

• Databricks Certified Data Engineer (Associate or Professional).

• AWS Solutions Architect or Data Analytics certification.

• Unity Catalog and Delta Live Tables experience.

• Federal government contract experience.

• FISMA/FedRAMP environment experience.

• AI/ML pipeline support (MLflow, model registry).

• Active security clearance.