Data Engineer IT · London ·
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+23 more
Job description
Data sits at the heart of how we understand our customers, plan our stock and run our operations. The Data Engineer builds the foundation that makes this possible: a reliable, well governed data platform on Microsoft Fabric that brings together our eCommerce and wholesale data and makes it ready for analytics, reporting and machine learning.
This is a hands on role with real ownership. You will take responsibility for defined areas of the platform, from ingestion through to modelled, trusted data, with guidance and support, widening your scope as the business and the platform grow.
You’ll be;
Designing, building and maintaining scalable, reliable data pipelines in Microsoft Fabric that ingest data from D365 F&O, our eCommerce platform and other source systems.
Developing and maintaining well-structured data models in OneLake using Delta tables, PySpark and SQL, following a medallion approach and dimensional modelling principles.
Working closely with analytics engineers, data analysts, data scientists and business stakeholders to understand data requirements and turn them into reusable, well-documented datasets.
Building data quality checks, testing and monitoring into pipelines so issues are caught early, as well as investigating and resolving data and pipeline issues promptly and addressing root causes.
Following data governance standards, including clear data definitions, lineage, documentation and access controls.
Using Git and CI/CD practices, such as Azure DevOps or GitHub, for version control, code review and the deployment of pipelines and notebooks.
Monitoring and optimising pipeline performance and Fabric capacity usage, balancing speed, reliability and cost.
Contributing to team standards, documentation and knowledge sharing.
Requirements
2+ years’ proven experience as a Data Engineer or in a similar role, working across areas such as data ingestion, transformation, modelling, data quality and performance optimisation.
Strong hands-on proficiency across the core data engineering languages:
PySpark: experience building and optimising Spark transformations at scale, ideally in notebook-based environments such as Fabric or Databricks.
Python: experience with scripting and automation around data workflows, including API ingestion, testing and orchestration.
SQL: experience writing queries and transformations, and modelling data in relational and warehouse environments.
Hands on experience with a modern cloud data platform, such as Microsoft Fabric, Azure Synapse, Azure Data Factory or Databricks, with a willingness to deepen your Fabric skills.
A good understanding of data warehousing and Lakehouse concepts, including ELT patterns, dimensional modelling and medallion architecture.
Experience using Git, including working through pull requests and code reviews.
Strong problem-solving skills and attention to detail, with a methodical approach to debugging data issues.
Clear communication skills, with the ability to work effectively with both technical and non-technical colleagues, independently and as part of a team.
Desirable You’ll ideally also have:
Experience with Microsoft Fabric components, such as:
Data Factory, including pipelines and orchestration.
Data Engineering, including Lakehouse, notebooks and Apache Spark.
Data Warehouse, including SQL endpoints and schemas.
Real-Time Intelligence, including KQL databases and event ingestion.
OneLake and Delta tables, including security and governance within Fabric.
An understanding of D365 F&O data structures, for example through Fabric Link or Synapse Link.
Power BI knowledge, including semantic models and basic DAX, allowing you to collaborate effectively with analysts.
Experience working with eCommerce, retail or wholesale data.
Familiarity with data governance and cataloguing tools, such as Microsoft Purview.
Exposure to streaming or real-time data technologies, such as Event Hubs or Kafka.
Exposure to data science or machine learning workflows.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…