Senior Data and Machine Learning Engineer

The Senior Data Engineer is responsible for designing, building, and optimising enterprise-grade data platforms and analytics assets across Microsoft Fabric, Power BI, and Qlik. This role leads the development of scalable data pipelines, ensures high-quality data models, and provides technical leadership and oversight for AI and data science enablement.

Key Deliverables

• Act as the technical owner for Microsoft Fabric, shaping strategy, architecture, and bestpractice use of OneLake, Data Factory, Lakehouse, Warehouses, and RealTime Analytics.
• Develop and maintain endtoend data architecture, ensuring consistency, scalability, reusability, and alignment with enterprise data strategy.
• Lead the transition from legacy and hybrid environments to Fabric-centric modern data architecture.
• Design, build, and optimise robust, automated data pipelines using Fabric Data Factory, Spark, SQL, and other relevant tools.
• Build scalable feature engineering pipelines to support machine learning workloads.
• Implement CI/CDbased deployment patterns for pipelines, notebooks, and semantic models.
• Ensure pipelines are costoptimised, monitored effectively, and resilient to failures.
• Provide technical direction and support for enterprise reporting platforms including Power BI and Qlik.
• Create and maintain centrally governed semantic models, shared datasets, and data products.
• Collaborate with BI analysts to optimise performance, automate refreshes, and implement data modelling best practices.
• Provision and maintain environments, compute clusters, Python/R packages, and secure data access for experimentation and model training.
• Enable MLOpsstyle workflows within Fabric (or other platforms), including model versioning, monitoring, and pipeline integration.
• Integrate AI/ML services (Azure AI, Azure ML, Azure OpenAI, custom APIs) into data products.
• Provide oversight on feature engineering pipelines and productionisation of machine learning models.
• Embed data quality frameworks, documentation standards, and lineage tracking across all pipelines and datasets.
• Ensure compliance with organisational data governance policies, including access controls, data retention, and GDPR considerations.
• Work closely with Data Governance functions to reinforce metadata standards and promote high data literacy.
• Lead a team of data engineers, providing coaching, mentoring, and capability uplift.
• Work collaboratively with product owners, BI teams, data scientists, IT, and business stakeholders.
• Translate business needs into scalable technical solutions and provide expert guidance on platform capabilities and limitations., We actively promote equal opportunities and are dedicated to ensuring that our recruitment, selection, training, and promotion decisions are made based on qualifications, and experience only.

If you require any reasonable adjustments to support your application or to attend an interview, if shortlisted, please let us know and we will be happy to assist.

• Expertise in Microsoft Fabric, including Data Factory, Lakehouse, OneLake, Pipelines, and SQL Warehouse.
• Advanced skills in data engineering: SQL, PySpark, Python, Delta Lake, data modelling, pipeline orchestration, and ETL/ELT best practices.
• Experience implementing CI/CD for analytics using Azure DevOps or GitHub.
• Experience integrating AI platforms such as Azure AI, Azure Machine Learning, or Azure OpenAI.
• Knowledge of real-time data streaming (Event Hub, Fabric Real-Time Analytics, Kafka, etc.).
• Experience leading or mentoring data engineering teams.
• Relevant certifications (e.g., DP203, Fabric Analytics Engineer, Azure Solutions Architect).
• Strong experience with Power BI and Qlik at enterprise scale (semantic models, governance, performance optimisation).
• Handson experience supporting and building AI/ML environments and models, data science workflows, or MLOps frameworks.
• Excellent understanding of cloud platforms (Azure preferred) including storage, compute, identity/security management.
• Strong stakeholder communication skills and the ability to lead technical discussions.

Attributes (who I am)

• Strategic thinker with a strong engineering mindset.
• Proactive, highly organised, and able to prioritise competing demands.
• Collaborative leader able to influence without authority.
• Committed to continuous improvement and innovation.
• Strong project management skills to ensure that information is delivered on time with a high level of professionalism.
• Experience of having worked closely with senior management