> Markdown version of [/jobs/ext/1994560-sr-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/1994560-sr-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Sr Data Engineer - Databricks - **Company:** ADASTRA DESIGNS, INCORPORATED - **Location:** Irvine, CA, United States - **Experience:** Expert - **Contract:** Contract - **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Microsoft Azure, Cloud Storage, Code Review, Databases, Continuous Integration, Customer Data Management, Information Engineering, Data Governance, Data Integration, Extract Transform Load (ETL), DevOps, Apache Hive, Python (Programming Language), Operational Databases, Performance Tuning, Power BI, Data Streaming, Apache Spark, Git, Data Lakes, Pyspark, Information Technology, Apache Kafka, Databricks - **Published:** August 8, 2026 - **Apply:** https://www.techcareers.com/job.asp?id=3347493101&tx=FP2118FFG&pt=1&aff=0B19D771-A501-4A5E-8338-2A822B784D54&utm_source=Job%20Feed&utm_medium=textkernel&utm_campaign=DE&utm_term=0B19D771-A501-4A5E-8338-2A822B784D54 ## About the Role * Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field * 5+ years of data engineering experience building and operating production data pipelines * Hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, and Unity Catalog * Advanced experience with PySpark and Spark SQL * Strong SQL development and performance tuning skills * Solid Python development skills and experience with modern engineering practices * Experience using Git, CI/CD pipelines, testing frameworks, and code review processes * Experience working with AWS, Azure, or GCP and their core compute and storage services * Strong understanding of data modeling concepts, including dimensional and medallion architectures * Experience designing ELT/ETL solutions and integrating data from databases, APIs, files, and event streams * Strong communication skills and the ability to work independently within a client-facing environment NICE TO HAVE * Databricks Certified Data Engineer Associate or Professional certification * Experience with Structured Streaming, Auto Loader, Kafka, or Kinesis * Experience working with marketing or customer data platforms such as Braze * Retail, grocery, or consumer packaged goods (CPG) industry experience * Experience supporting Power BI reporting and self-service analytics initiatives ## Description * Design, build, and maintain scalable batch and streaming pipelines on Databricks using PySpark and Spark SQL within a medallion (Bronze/Silver/Gold) architecture * Ingest and unify data from operational databases, cloud storage, APIs, and marketing platforms such as Braze * Model and optimize Delta Lake tables for performance, reliability, and cost efficiency using techniques such as partitioning, liquid clustering, OPTIMIZE, and VACUUM * Deliver analytics-ready data for reporting on sales, shipments, orders, and other key business metrics, supporting Power BI dashboards and downstream analytics * Orchestrate jobs using Databricks Workflows, managing dependencies, retries, alerting, and SLAs * Apply Unity Catalog for data governance, lineage, and access control * Work to a high engineering standard using Git, CI/CD, testing, code reviews, and clear documentation * Monitor pipeline health, troubleshoot issues, and tune Spark jobs and cluster configurations for speed and cost optimization * Partner with business and technical stakeholders to deliver reliable, high-quality data products ## Related Videos - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)