Data Engineer

This is a brand-new role, created to build and run the technical foundation of HGF’s data strategy. You’ll bring data from our source systems into our Lakehouse on Microsoft Fabric, process it through a medallion architecture, and deliver modelled, trusted data that’s ready for reporting, AI and machine learning.

You’ll sit in our Data team reporting to the Head of Data, working with our Data Analysts on modelling, and with IT, Information Security and system owners on secure integration. It’s a small team and a genuine build role, which means real influence over the architecture, the tooling and the standards we adopt.

As the data platform grows so will the role and there’s scope to move into advanced analytics, and AI-assisted tooling such as Fabric MCP, Fabric IQ, Copilot and data agents.

Key Responsibilities

Building and maintaining scalable data pipelines, pulling data in from core business systems and third-party sources.

Designing and developing data solutions with Python, PySpark, SQL, notebooks and dataflows across a modern medallion architecture (Bronze, Silver, Gold).

Creating and optimising data models that power high-quality Power BI reporting and analytics.

Putting data quality, validation and reconciliation processes in place so the numbers can be trusted.

Monitoring, troubleshooting and keeping the platform fast, reliable and available.

Configuring and supporting master data management and entity resolution, so data stays consistent across systems.

Owning our master data management approach defining golden records for clients, contacts and matters, setting matching, de-duplication and survivorship rules, and using entity resolution tooling such as Aptitude or Senzing to link records across systems.

Owning source control, code reviews and CI/CD pipelines through Azure DevOps.

Implementing secure, compliant data access role-based security, auditing and governance controls.

Keeping technical documentation, data lineage and platform standards in good order.

Helping shape where the platform goes next, including future analytics, AI and machine learning capability.

Commercial data engineering experience (4+ years)

SQL (4+ years)

Python and PySpark (3+ years)

Microsoft Fabric or an equivalent Azure data platform (2+ years)

Medallion / layered lakehouse architecture (2+ years)

Dimensional data modelling for analytics (2+ years)

Building and maintaining production data pipelines (3+ years)

Azure DevOps, including source control and CI/CD practices (2+ years)

Master Data Management (MDM) and entity resolution tools such as Aptitude or Senzing (2+ years)

This is a mid to senior level role. We don’t expect every box ticked if you’re close, get in touch.

About You

You’re an experienced Data Engineer who knows Microsoft Fabric or the Azure data stack well, and you’ve built, optimised and supported pipelines that people actually depend on. Python, PySpark and SQL are comfortable territory, and you’ve designed lakehouse architectures that hold up under real business intelligence workloads.

You’re a proactive problem-solver who works methodically and delivers to a high standard, with data quality, security and platform reliability built into everything you do. You can talk to technical and non-technical colleagues just as easily, and you enjoy an environment where collaboration, ownership and continuous improvement genuinely matter. Experience with master data management, data governance, Azure security or real-time data solutions would be a real bonus.