> Markdown version of [/jobs/ext/4692251-senior-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/4692251-senior-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer - Databricks

- **Company:** Capgemini
- **Location:** Troy, MI, United States
- **Experience:** Expert
- **Salary:** $89,400 - $166,398
- **Contract:** Permanent contract
- **Skills:** Unity 3d, Query Performance, Artificial Intelligence, Amazon Web Services, Automation of Tests, Microsoft Azure, Cluster Analysis, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, Software Design Patterns, DevOps, Github, Python (Programming Language), Machine Learning, Performance Tuning, Search Technologies, SQL Databases, Data Streaming, Enterprise Data Management, Google Cloud, Cloud Platform System, Feature Engineering, Large Language Models, Gitlab, Git, Pytest, Data Lakes, Pyspark, Data Lineage, Machine Learning Operations, Terraform, Software Version Control, Data Pipelines, Databricks
- **Published:** November 12, 2026
- **Apply:** https://www.capgemini.com/jobs/592537-en_US_SAPBTP/x/

## About the Role

Candidates must be able to work onsite at the customer location in Troy, MI a minimum of four (4) days per week., * 5-8 years of experience in Data Engineering.Strong proficiency in Python (PySpark) and SQL.Hands-on experience with

## Description

We are seeking a highly skilled Senior Databricks Data Engineer with 5-8 years of experience in data engineering and cloud-based data platforms. The ideal candidate will have strong hands-on expertise in the Databricks ecosystem, including PySpark, Delta Lake, Unity Catalog, Structured Streaming, and Databricks Workflows. This role will be responsible for designing, developing, and optimizing scalable data solutions that support analytics, machine learning, and generative AI initiatives.

The successful candidate will work closely with data architects, data scientists, business stakeholders, and application teams to build modern lakehouse architectures and deliver high-performance data pipelines.

Your role

 * Design, develop, and maintain scalable batch and real-time data pipelines using PySpark, SQL, Delta Lake, and Auto Loader.
 * Build robust ETL/ELT frameworks to ingest, transform, and process large-scale datasets from multiple sources.
 * Implement data quality checks, monitoring, and pipeline optimization to ensure reliability and performance.
 * Design and manage modern Databricks Lakehouse solutions using Medallion Architecture (Bronze, Silver, Gold layers).
 * Develop and maintain Delta Lake tables while ensuring optimal query performance.
 * Perform performance tuning using Databricks best practices, (e.g. Photon Engine,Liquid Clustering, Partitioning)
 * Configure and administer Unity Catalog to manage data governance across the enterprise.
 * Implement catalog, schema, table, row-level, and column-level security controls.
 * Enable data lineage, auditing, and compliance requirements across the data platform.
 * Develop and maintain workflows using Databricks Workflows.
 * Manage deployment processes using Databricks Asset Bundles (DABs).
 * Implement CI/CD pipelines leveraging GitHub, GitLab, and DevOps best practices.
 * Provision and manage infrastructure using Terraform and Infrastructure-as-Code methodologies.
 * Prepare and curate datasets for machine learning and generative AI applications.
 * Support feature engineering and feature store implementations.
 * Enable vector search and retrieval capabilities to support large language model (LLM) solutions and AI-driven use cases.
 * Collaborate with Data Science and AI teams to operationalize ML workloads., + Databricks Lakehouse Platform
 + Delta Lake
 + Unity Catalog
 + Databricks Workflows
 + Structured Streaming
 + Auto Loader
 * Experience designing and supporting large-scale enterprise data platforms.
 * Strong understanding of data modeling, data warehousing, and ETL/ELT design patterns.
 * Experience implementing CI/CD pipelines, version control, and automated testing practices.
 * Hands-on experience with Git, GitHub/GitLab, Terraform, and pytest.
 * Experience working with at least one major cloud platform: (Azure, GCP, AWS)