> Markdown version of [/jobs/ext/2851187-senior-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/2851187-senior-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer - Databricks

- **Company:** Data Canopy Colocation LLC
- **Location:** Raleigh, NC, United States (Remote available)
- **Experience:** Expert
- **Salary:** $73,450.0 - $132,775.0
- **Contract:** Permanent contract
- **Skills:** Unity 3d, Airflow, Amazon Web Services, Microsoft Azure, Cloud Computing, Computer Programming, Continuous Integration, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Distributed Systems, Apache Hive, Python (Programming Language), Meta-Data Management, Performance Tuning, SQL Databases, Data Streaming, Cloud Platform System, Snowflake, Apache Spark, Git, Data Lakes, Pyspark, Data Pipelines, Databricks
- **Published:** September 11, 2026
- **Apply:** https://www.careerjet.com/jobad/us938f33e3ceff43f4e97c004b204b42f1

## About the Role

* Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).

 * Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.

 * Develop optimized transformations, data quality rules, and reusable framework components.

 * Implement best practices for job orchestration, monitoring, alerting, and automation.

 * Hands-on experience: Spark, Delta Lake, Workflows, Unity Catalog.

 * Strong SQL programming and performance tuning skills.

 * Experience with cloud environments (AWS/Azure/GCP).

 * Experience with modern data lakehouse concepts and distributed systems.

 * Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.

 * Should be able to explain these concepts using real project examples and architecture decisions.

Requirements

 * Strong Python (PySpark) and SQL programming

 * Databricks - Spark, Delta Lake, Workflows, Unity Catalog

 * ETL/ELT pipeline development - Medallion Architecture (Bronze/Silver/Gold)

 * Delta Live Tables, Auto-Loader, Structured Streaming

 * Data modeling - dimensional (star/snowflake), normalization/denormalization

 * CI/CD, Git, job orchestration

 * Cloud experience - AWS, Azure, or GCP

 * 7-10+ years in data engineering

Nice-to-Have Skills

 * Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases

 * Data governance, metadata management, Unity Catalog advanced features

 * Airflow, dbt, or similar orchestration tools

 * Data security, compliance, and access models

 * Cost optimization and performance tuning in cloud environments

 * Corporate/enterprise data warehousing background



## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)