> Markdown version of [/jobs/ext/2841857-data-engineer](https://www.wearedevelopers.com/jobs/ext/2841857-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** HGF Limited - **Location:** Leeds, UK - **Experience:** Expert - **Salary:** £76,293.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Microsoft Azure, Business Systems, Code Review, Cyber Security, System Configuration, Continuous Integration, Data Governance, Data Infrastructure, Data Security, Data Systems, Data Warehousing, Python (Programming Language), Operational Databases, Power BI, SQL Databases, Azure Data Factory, System Availability, Data Strategy, Microsoft Fabric, Pyspark, Data Lineage, Enterprise Integration, Real Time Data, Data Management, Software Version Control, Data Pipelines - **Published:** September 11, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5878899693 ## About the Role Commercial data engineering experience (4+ years) SQL (4+ years) Python and PySpark (3+ years) Microsoft Fabric or an equivalent Azure data platform (2+ years) Medallion / layered lakehouse architecture (2+ years) Dimensional data modelling for analytics (2+ years) Building and maintaining production data pipelines (3+ years) Azure DevOps, including source control and CI/CD practices (2+ years) Master Data Management (MDM) and entity resolution tools such as Aptitude or Senzing (2+ years) This is a mid to senior level role. We don't expect every box ticked if you're close, get in touch. About You You're an experienced Data Engineer who knows Microsoft Fabric or the Azure data stack well, and you've built, optimised and supported pipelines that people actually depend on. Python, PySpark and SQL are comfortable territory, and you've designed lakehouse architectures that hold up under real business intelligence workloads. You're a proactive problem-solver who works methodically and delivers to a high standard, with data quality, security and platform reliability built into everything you do. You can talk to technical and non-technical colleagues just as easily, and you enjoy an environment where collaboration, ownership and continuous improvement genuinely matter. Experience with master data management, data governance, Azure security or real-time data solutions would be a real bonus. ## Description This is a brand-new role, created to build and run the technical foundation of HGF's data strategy. You'll bring data from our source systems into our Lakehouse on Microsoft Fabric, process it through a medallion architecture, and deliver modelled, trusted data that's ready for reporting, AI and machine learning. You'll sit in our Data team reporting to the Head of Data, working with our Data Analysts on modelling, and with IT, Information Security and system owners on secure integration. It's a small team and a genuine build role, which means real influence over the architecture, the tooling and the standards we adopt. As the data platform grows so will the role and there's scope to move into advanced analytics, and AI-assisted tooling such as Fabric MCP, Fabric IQ, Copilot and data agents. Key Responsibilities Building and maintaining scalable data pipelines, pulling data in from core business systems and third-party sources. Designing and developing data solutions with Python, PySpark, SQL, notebooks and dataflows across a modern medallion architecture (Bronze, Silver, Gold). Creating and optimising data models that power high-quality Power BI reporting and analytics. Putting data quality, validation and reconciliation processes in place so the numbers can be trusted. Monitoring, troubleshooting and keeping the platform fast, reliable and available. Configuring and supporting master data management and entity resolution, so data stays consistent across systems. Owning our master data management approach defining golden records for clients, contacts and matters, setting matching, de-duplication and survivorship rules, and using entity resolution tooling such as Aptitude or Senzing to link records across systems. Owning source control, code reviews and CI/CD pipelines through Azure DevOps. Implementing secure, compliant data access role-based security, auditing and governance controls. Keeping technical documentation, data lineage and platform standards in good order. Helping shape where the platform goes next, including future analytics, AI and machine learning capability. ## Related Videos - [Are Code Reviews Worth It? Insights from 16 Years of Review Data](https://www.wearedevelopers.com/videos/1135-are-code-reviews-worth-it-insights-from-16-years-of-review-data) - [Beyond Dashboards: Fixing Text-to-SQL with Semantic RAG](https://www.wearedevelopers.com/videos/2036-beyond-dashboards-fixing-text-to-sql-with-semantic-rag) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Data Analytics with Microsoft Fabric: End-to-End Use Case with Data Agents](https://www.wearedevelopers.com/videos/1547-data-analytics-with-microsoft-fabric-end-to-end-use-case-with-data-agents) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)