> Markdown version of [/jobs/ext/1974693-senior-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/1974693-senior-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer - Databricks

- **Company:** SugarCRM
- **Location:** Denver, CO, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Microsoft Azure, Continuous Integration, Customer Data Management, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Python (Programming Language), PostgreSQL, Performance Tuning, Role-Based Access Control, SQL Databases, SQL Server Integration Services, Data Streaming, Apache Spark, Build Management, Data Lakes, Pyspark, Data Delivery, Software Version Control, Data Pipelines, Databricks
- **Published:** August 7, 2026
- **Apply:** https://us.experteer.com/career/view-jobs/senior-data-engineer-databricks-denver-co-usa-58827512

## About the Role

for operational readiness * Apply CI/CD practices to data pipelines with version control, testing, and deployment tooling Tasks * 4+ years of data engineering experience * 2+ years on Databricks or Apache Spark in Azure and/or AWS * Proficiency in PySpark, SQL, and Python with production-grade pipelines under SLA * Hands-on experience with Delta Lake (schema evolution, ACID, optimize/vacuum, incremental/streaming) * Pipeline performance tuning and compute optimization in Databricks * PostgreSQL knowledge for production pipelines * Experience maintaining legacy ETL tooling (SSIS, Informatica, custom pipelines) * Experience with large-scale multi-tenant architectures emphasizing isolation and data privacy * Ability to collaborate across data science, product, and infrastructure teams * Strong understanding of data governance, security, and compliance across multi-tenant environments Key requirements * Excellent healthcare package for you and your family * 401(k) match * Unlimited Paid a
and Off * Paid Parental Leave * Online Legal Services (Rocket Lawyer) * Financial Planning Services (Origin)

## Description

Experteer Overview In this Senior Data Engineer role, you will own and optimize the Databricks-based data pipelines that power Sugar Predict, ensuring reliable, scalable data delivery across Azure and AWS. You'll collaborate with ML, product, and Enterprise Architecture teams to maintain a fast, clean data backbone that supports global scale. You'll reduce compute costs, modernize legacy pipelines, and enforce security and data quality across multi-tenant environments. This position offers hands-on impact on how revenue intelligence is delivered to mid-market customers and involves on-call incident response and cross-functional work to drive platform growth. Compensation / Benefits * Own Databricks production support across all data flows, including monitoring, alerting, and incident response * Maintain and report SLA metrics for data pipeline delivery and platform health * Optimize pipelines to reduce Databricks compute costs and improve throughput * Migrate legacy ETL/ELT pipelines to Databricks with automation to minimize manual intervention * Onboard and harden tenant data pipelines for new customers to ensure isolated, reliable data from day one * Design and build high-performance Databricks pipelines ingesting and transforming ERP and CRM data at scale across Azure and AWS * Own Delta Lake architecture including schema design, partitioning, quality enforcement, and incremental processing * Enforce data security best practices across Databricks (RBAC, secrets, Unity Catalog, Attestation Maps, compliance) * Implement data quality monitoring and observability to support model inputs and prediction accuracy * Apply multi-tenant isolation patterns for reliable, secure data delivery across enterprise customers * Partner with Enterprise Architecture to integrate pipelines with the SugarAI product ecosystem * Support globally distributed operations with on-call rotation and after-hours incident response * Maintain documentation, runbooks, and architectural decision records for operational readiness * Apply CI/CD practices to data pipelines with version control, testing, and deployment tooling Tasks * 4+ years of data engineering experience * 2+ years on Databricks or Apache Spark in Azure and/or AWS * Proficiency in PySpark, SQL, and Python with production-grade pipelines under SLA * Hands-on experience with Delta Lake (schema evolution, ACID, optimize/vacuum, incremental/streaming) * Pipeline performance tuning and compute optimization in Databricks * PostgreSQL knowledge for production pipelines * Experience maintaining legacy ETL tooling (SSIS, Informatica, custom pipelines) * Experience with large-scale multi-tenant architectures emphasizing isolation and data privacy * Ability to collaborate across data science, product, and infrastructure teams * Strong understanding of data governance, security, and compliance across multi-tenant environments Key requirements * Excellent healthcare package for you and your family * 401(k) match * Unlimited Paid Time Off * Paid Parental Leave * Online Legal Services (Rocket Lawyer) * Financial Planning Services (Origin)


## Related Videos

- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)