> Markdown version of [/jobs/ext/3455590-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/3455590-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - Databricks - **Company:** Metaphase Consulting - **Location:** United States - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Data Analysis, Automation of Tests, Microsoft Azure, Cloud Computing, Code Review, Continuous Integration, Data as a Services, Data Validation, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Mapping, Software Debugging, Python (Programming Language), Machine Learning, Metadata, Software Engineering, SQL Databases, Data Streaming, Data Processing, Google Cloud, Apache Spark, Generative AI, Git, Data Lakes, Software Version Control, Data Pipelines, Databricks - **Published:** September 28, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=cf35f6ee9339d86d ## About the Role * Bachelor's degree in a technical discipline and three or more years of relevant experience in data engineering, software engineering, analytics engineering, or a related field. * Demonstrated proficiency in Python and SQL, with experience developing, debugging, and maintaining ETL/ELT processes and data-processing code. * One or more years of hands-on experience with Databricks, Apache Spark, or a comparable cloud data-engineering platform. * Experience working with structured and/or unstructured data sources, data validation, source-to-target mapping, and production-support activities. * Databricks Certified Data Engineer Associate certification preferred; candidates without the certification must be willing to obtain it within three months of start date. * Ability to obtain a U.S. Public Trust suitability determination. * U.S. Citizenship Required(Clearance / Citizenship Requirements). Bonus Points (Desired) * Experience with Databricks capabilities such as Delta Lake, Auto Loader, Databricks SQL, Lakeflow Jobs, Unity Catalog, or streaming data pipelines. * Familiarity with Git-based version control, code reviews, automated testing, CI/CD, and Agile delivery practices. * Experience supporting data governance activities, including metadata documentation, data-quality checks, lineage, and access-control implementation. * Experience with AWS, Azure, or Google Cloud data services and cloud-based integrations. * Experience supporting regulated, public-sector, or security-sensitive data environments. * Additional Databricks certifications (e.g., Databricks Machine Learning Engineer Associate or Professional, Databricks Generative AI Engineer Associate), Remote (travel 5%)Education BA/BS in a Technical Discipline Clearance Requirements Must be able to obtain a U.S. Public Trust suitability determination. U.S. Citizenship Required. ## Description The Data Engineer - Databricks will support the design, development, testing, deployment, operation, and continuous improvement of data pipelines and data products within a Databricks environment. Working under the direction of the Principal Databricks Architect / Engineer and delivery leadership, this role will convert approved data requirements into reliable, well-documented, and supportable code. The engineer will contribute to data governance, platform operations, troubleshooting, and sustainment while building practical expertise in Databricks engineering patterns and enterprise data delivery. What You Will Be Doing * Develop, test, deploy, and maintain batch and streaming data pipelines using Databricks, Python, SQL, Apache Spark, and Delta Lake. * Build and enhance ingestion, transformation, validation, and publishing processes that move data from source systems into governed data products and analytics-ready datasets. * Implement approved data models, data-quality rules, metadata, and documentation in accordance with established architecture and governance standards. * Configure and maintain Databricks notebooks, workflows, jobs, compute resources, and related deployment artifacts. * Participate in code reviews, peer testing, release preparation, defect remediation, and CI/CD activities. * Monitor pipeline performance, job execution, data-quality results, and platform alerts; troubleshoot issues and support resolution of production incidents. * Collaborate with architects, analysts, data owners, and other engineers to clarify requirements, identify dependencies, and deliver iterative improvements. * Maintain technical documentation for pipelines, data sources, transformations, interfaces, test results, and operating procedures. ## Related Videos - [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow) - [A Data Mesh needs Open Metadata](https://www.wearedevelopers.com/videos/505-a-data-mesh-needs-open-metadata) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers) - [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source) - [Crafting Custom Frameworks with Rust: A Deep Dive into Procedural Macros](https://www.wearedevelopers.com/videos/849-crafting-custom-frameworks-with-rust-a-deep-dive-into-procedural-macros) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)