> Markdown version of [/jobs/ext/3520126-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3520126-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Quilter plc - **Location:** Southampton, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Automation of Tests, Microsoft Azure, Big Data, Code Review, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Data Retention, Data Systems, Data Vault Modeling, Data Warehousing, Database Development, Software Design Patterns, DevOps, Distributed Systems, Python (Programming Language), Meta-Data Management, Pair Programming, Performance Tuning, Azure Data Lake, Azure Data Factory, Apache Spark, Data Strategy, Git, Event Driven Architecture, Microsoft Fabric, Data Lakes, Kubernetes, Infrastructure Automation Frameworks, Information Technology, Data Lineage, Deployment Automation, Data Management, Stream Processing, Software Version Control, Data Pipelines, Key Vault, Databricks - **Published:** October 2, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=b01d48180f70283b ## About the Role * Degree in a technical discipline (e.g. computer science, engineering, maths, physics) or evidence of equivalent practical experience Knowledge * Proven experience as a senior or lead data engineer within a complex enterprise environment, delivering large-scale data solutions. * Strong hands-on experience designing, developing, and supporting data platforms and pipelines using Databricks within Azure. * Experience implementing both batch and real-time data processing solutions, including event-driven and streaming architectures. * Demonstrable experience leading the technical design and delivery of data products and platform capabilities across one or more business domains. * Experience implementing and promoting data quality frameworks, service level objectives (SLOs), data contracts, and observability tooling. * Strong understanding of data governance principles, including data lineage, metadata management, cataloguing, security, and access controls. * Experience ensuring compliance with regulatory and organisational requirements, including GDPR, data retention, and information security controls. * Proven ability to define technical standards, engineering patterns, and best practices and drive their adoption across engineering teams. * Experience mentoring and coaching engineers, conducting code reviews, and promoting engineering excellence. * Strong experience designing and implementing data models across multiple architectural layers, including Kimball dimensional modelling and Data Vault methodologies. * Experience working within Agile delivery teams and collaborating with architects, product owners, analysts, and business stakeholders to deliver business outcomes. Skills * Strong technical leadership skills with the ability to influence engineering direction and drive best practices across teams. * Strategic thinker with a curious mindset, strong analytical capability, and a passion for continuous improvement. * Excellent problem-solving skills with the ability to resolve complex technical challenges and make sound engineering decisions. * Strong communication skills, with the ability to explain complex technical concepts clearly to both technical and non-technical audiences. * Databricks: Delta Lake, Unity Catalog, Workflows, Asset Bundles, Notebooks, Compute Management, Delta Live Tables, Spark Declarative Pipelines and platform optimisation. * Microsoft Fabric: Lakehouse, Data Warehouse, Real-Time Intelligence, Semantic Models, and Fabric administration concepts. * Azure: ADLS Gen2, Azure Data Factory, Key Vault, Functions, Azure Monitor, and broader Azure data services. * Spark: Deep understanding of Spark internals, distributed computing principles, optimisation techniques and features, performance tuning. * Languages: Advanced Python and SQL development skills; Scala experience beneficial. * CI/CD & DevOps: Azure DevOps, Git-based development workflows, Infrastructure as Code, automated testing, and deployment automation. * Orchestration: Databricks Workflows, Azure Data Factory, Fabric pipelines, and other enterprise scheduling and orchestration frameworks. * Data Architecture & Modelling: Strong knowledge of Kimball, Data Vault, medallion architecture, and modern lakehouse design patterns. * Data Governance & Security: Experience implementing secure-by-design data solutions, platform governance controls, and operational monitoring practices. Desired Attributes * Passionate about engineering excellence and continuous learning. * Acts as a trusted technical advisor and subject matter expert within their domain. * Proactively identifies opportunities to improve platform capability, performance, reliability, and security. * Collaborative and supportive team player who enjoys mentoring others and sharing knowledge. * Takes ownership and accountability for technical outcomes while maintaining a pragmatic delivery-focused approach. ## Description The role sits within the Chief Data Office (CDO) and reports to the Data Engineering Manager. This role accelerates Quilter's data strategy by providing deep, technical leadership across one or more data domains; defining engineering standards, and delivering robust, scalable and secure solutions. The Lead Data Engineer plays a pivotal role in enabling business value through the design, development, and continuous improvement of data products and platforms that support actionable insights across the organisation. Working closely with engineers, architects, analysts, platform teams, and business stakeholders, the Lead Data Engineer is responsible for translating business requirements into robust technical solutions while driving engineering best practices and fostering a culture of technical excellence. The role requires deep technical expertise, strong problem-solving capabilities, and the ability to influence and guide both technical and non-technical stakeholders. The key accountabilities for the role are as follows: Technical Leadership & Domain Ownership * Provide technical leadership for one or more data domains, acting as the primary engineering authority and subject matter expert. * Define, champion, and evolve data engineering standards, patterns, and best practices to ensure robust, efficient, secure, and maintainable solutions. * Lead solution design activities and guide the implementation of scalable data pipelines, data products, and platform capabilities. * Review code, architecture, and engineering approaches to ensure consistency, quality, and alignment with strategic objectives. * Remain hands-on when required, supporting the resolution of complex technical challenges and critical delivery issues. Engineering Delivery * Lead the technical delivery of key initiatives and ensure solutions are delivered to a high standard, meeting business requirements and engineering principles. * Translate business and product requirements into detailed technical designs and implementation plans. * Collaborate with Data Engineering Managers, Product Owners, Architects, and stakeholders to prioritise and deliver work effectively. * Drive adoption of modern engineering practices including automation, testing, CI/CD, source control, observability, and infrastructure-as-code. * Contribute to roadmap planning by identifying technical opportunities, risks, dependencies, and improvement initiatives. Mentoring & Capability Development * Provide coaching, mentoring, and technical guidance to data engineers across the team. * Support the development of engineering capability through knowledge sharing, technical reviews, pair programming, and communities of practice. * Promote a collaborative and inclusive engineering culture focused on quality, learning, and continuous improvement. * Act as a role model for engineering excellence and professional standards. Cross-Functional Collaboration * Partner with product managers, analysts, data scientists, platform teams, and business * stakeholders to understand requirements and deliver valuable outcomes. * Work closely with architecture, platform, and governance teams to ensure solutions align with enterprise standards and strategic direction. * Communicate technical concepts clearly to both technical and non-technical audiences. * Influence decision-making by providing expert technical advice and recommendations. Operational Excellence * Promote operational stability, reliability, and performance of data products and pipelines through monitoring, alerting, automation, and incident management practices. * Champion high standards of data quality, security, governance, and regulatory compliance. * Identify and implement opportunities to improve platform efficiency, engineering productivity, and service resilience. * Support root cause analysis and continuous improvement activities to enhance operational performance and service reliability., We value diversity and strive to promote inclusivity in all aspects of our culture. We believe in equal opportunities for all, ensuring that no applicant encounters less favourable treatment based on anything but their skills, qualifications, experience, and potential. We celebrate the unique contributions of a diverse workforce and create a respectful, nurturing environment where every colleague can thrive.