> Markdown version of [/jobs/ext/2277867-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2277867-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Head Resourcing Ltd - **Location:** Glasgow, UK - **Experience:** Expert - **Salary:** £40,000.0 - £70,000.0 - **Contract:** Permanent contract - **Skills:** Microsoft Excel, Application Programming Interfaces (APIs), Data Analysis, Microsoft Azure, Software as a Service, Cloud Computing, Code Review, Continuous Integration, Data Architecture, Information Engineering, Software Design Patterns, DevOps, Dimensional Modeling, Distributed Computing Environment, Apache Hive, Python (Programming Language), Power BI, Standard Sql, Microsoft SharePoint, SQL Databases, File Transfer Protocol (FTP), Azure Data Factory, Apache Spark, Git, Data Lakes, Pyspark, Deployment Automation, Terraform, Data Pipelines, Key Vault, Legacy Systems, Databricks - **Published:** August 28, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5858150571 ## About the Role * Significant commercial experience in Data Engineering, specifically with production workloads on Azure and Databricks. * Strong proficiency in PySpark/Spark SQL and distributed data processing. * Proven experience in delivering solutions based on Medallion/Lakehouse architecture using Delta Lake. * Solid understanding of dimensional modeling (Kimball), including surrogate keys, SCD types 1 and 2, and merge strategies. * Operational experience with SLAs, observability, idempotent pipelines, reprocessing, and backfills. * Strong grounding in secure Azure Landing Zone patterns. * Proficiency with Git, CI/CD, automated deployments, and modern engineering standards. * Effective communication skills to translate technical decisions into business outcomes. * ### ## Description * Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark, and Spark SQL following a full Medallion Architecture (Bronze - Silver - Gold). * Implement ingestion patterns for files, APIs, SaaS platforms (e.g., subscription billing), SQL sources, SharePoint, and SFTP employing ADF and metadata-driven frameworks. * Apply Lakeflow expectations for data quality, schema validation, and operational reliability. * Build clean, conformed Silver/Gold models aligned to enterprise business domains such as customers, subscriptions, deliveries, finance, credit, logistics, and operations. * Deliver star schemas, harmonization logic, SCDs, and business marts to enhance high-performance Power BI datasets. * Apply governance, lineage, and fine-grained permissions via Unity Catalog. * Design and optimize orchestration using Lakeflow Workflows and Azure Data Factory. * Implement monitoring, alerting, SLAs/SLIs, runbooks, and cost optimization across the platform. * Build CI/CD pipelines in Azure DevOps for notebooks, Lakeflow pipelines, SQL models, and ADF artifacts. * Ensure secure, enterprise-grade platform operation utilizing private endpoints, managed identities, and Key Vault. * Contribute to platform standards, design patterns, code reviews, and future roadmap. * Collaborate closely with BI/Analytics teams to deliver curated datasets that support dashboards across the organization. * Influence architecture decisions and elevate engineering maturity within our growing data function. * ### Technologies: * Azure * CI/CD * Databricks * DevOps * Git * Support * Power BI * PySpark * SQL * SharePoint * Spark * Unity * Cloud * GameDev * Excel * Fabric * Python * Security * Terraform More: We are a leading UK consumer business currently on an exciting journey of data modernization, moving away from legacy systems and manual reporting. We are building a fully automated Azure Enterprise Landing Zone combined with Databricks Lakehouse, and we want your expertise to shape our new data platform. This is a rare opportunity for you to join early in the process, influence architecture, and define engineering standards that will support our Operations, Finance, Sales, Logistics, and Customer Care functions. If youre passionate about creating a best-in-class lakehouse from the ground up, we would love to hear from you! ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)