> Markdown version of [/jobs/ext/3743948-lead-databricks-data-engineer](https://www.wearedevelopers.com/jobs/ext/3743948-lead-databricks-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Databricks Data Engineer

- **Company:** Booz Allen Hamilton Inc.
- **Location:** Bethesda, MD, United States
- **Experience:** Expert
- **Salary:** $77,600.0 - $176,000.0
- **Contract:** Permanent contract
- **Skills:** Query Performance, Amazon Web Services, Amazon S3, Information Systems, Computer Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Data Structures, Software Debugging, Python (Programming Language), Metadata, Performance Tuning, Raw Data, Data Streaming, Data Logging, Data Processing, Cloud Platform System, Caching, AWS Lambda, Amazon Relational Database Service, Data Lakes, Pyspark, Information Technology, Data Lineage, Collibra, AWS Glue, Cloudwatch, Data Pipelines, Serverless Computing, Databricks
- **Published:** October 15, 2026
- **Apply:** https://www.dice.com/job-detail/59b3bc3c-b8a0-477f-a2b5-9e38b2f26f37

## About the Role

* 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines utilizing comprehensive logging and robust error handling mechanisms to troubleshoot and resolve issues
 * 3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools using Databricks tools such as Delta Lake, Databricks Delta, and Unity Catalog
 * 3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations
 * Experience in data modeling, including when to use dimensional vs relation data models within a medallion architecture, including Bronze, Silver, or Gold
 * Experience using various AWS services, including AWS Glue, Amazon S3, Amazon RDS
 * Knowledge of performance optimization techniques for ETL jobs such as partitioning, bucketing, caching, and troubleshooting to improve query performance
 * Ability to design and implement data models from initially defining schemas to optimizing data structures that support efficient queries and analytics
 * Public Trust
 * Bachelor's degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology

Nice If You Have:

 * Experience with AWS cloud-native data services such as AWS Lambda or CloudWatch that support serverless architectures for data processing
 * Experience with tools such as AWS Glue Data Catalog or Collibra for managing metadata, data lineage, and cataloging data assets
 * Knowledge of data governance practices, data lineage, and compliance requirements, including GDPR or CCPA, in support of data quality, privacy, and security
 * Ability to explain complex technical concepts to a non-technical audience, including describing tradeoffs in architectural design

Vetting:

Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client; Public Trust determination is required.

## Description

As a lead Databricks Data Engineer, you will be hands-on designing, building, and maintaining robust data pipelines within Databricks and contributing independently while also providing technical leadership and mentorship within the team. Your key responsibilities will include creating and optimizing end-to-end data flows, handling extract, transform, load (ETL) processes, designing data models, ensuring data security and compliance, collaborating with cross-functional teams, and continuously improving performance and scalability. Your experience in both Databricks and AWS environments and services will position you as a technical team leader. Your expertise will bridge the gap between raw data and actionable insights, enabling efficient data processing and analytics across the business.

Here, you'll mentor a multi-disciplinary team of analysts, data engineers, developers, and data consumers in a fast-paced, agile environment. You'll use your expertise in analytical exploration and data examination while you oversee the assessment, design, building, and maintenance of scalable platforms for your clients.