Senior Data Engineer

Here at Serco, we're looking for a Senior Data Engineer to design, build and optimise robust data pipelines and products across a modern multi-cloud ecosystem spanning Azure, AWS, and Databricks. Databricks is central to our platform, and you'll play a key role in leveraging the Lakehouse architecture to power analytics, data science, and operational insight.

This is a deeply technical, hands-on role for a Senior Data Engineer who thrives on building high-quality data solutions at scale. Alongside hands-on delivery, you'll act as a senior technical contributor - guiding other engineers, shaping engineering standards, and influencing how data engineering is delivered across the organisation.

As part of this you'll:

• Lead the design and delivery of complex data pipelines and products (batch and streaming), ensuring scalability, resilience, and long-term maintainability.
• Define and assure data integration architectures, aligning to enterprise standards and evolving business needs.
• Build and optimise data solutions on Databricks, leveraging Delta Lake, Delta Live Tables, Unity Catalog, Structured Streaming, and workflows.
• Develop and optimise ETL/ELT pipelines using PySpark, SQL, and orchestration frameworks, with a strong focus on performance tuning and efficiency.
• Drive cost-effective engineering practices across Azure and AWS cloud platforms.
• Re-engineer legacy or manual processes into automated, reusable, and scalable solutions, defining patterns and accelerators.
• Design and implement data models and structures, including schema evolution, metadata, lineage, and lakehouse standards.
• Embed data quality, testing, and observability frameworks to ensure reliable and trusted data products.
• Ensure data security and compliance, including access control, encryption, and lifecycle management.
• Provide technical leadership through mentoring, coaching, and design reviews, raising engineering standards across teams.
• Collaborate across a matrix organisation (Data Science, Analytics, Architecture, Cyber, Product), translating technical decisions into clear business outcomes.
• Contribute to the evolution of the data platform, recommending tools, patterns, and improvements to engineering maturity.

• Strong experience designing and building data pipelines and products at enterprise scale.
• Advanced hands-on expertise in PySpark and SQL, including performance optimisation.
• Proven experience in data modelling, integration design, and scalable data architectures.
• Deep practical experience with Databricks (essential), including Delta Lake, Unity Catalog, streaming, and workflow orchestration.
• Solid experience across Azure (ADF, Synapse, ADLS, Event Hub, Purview) and/or AWS (S3, Glue, Kinesis, Athena, EMR, Lake Formation).
• Strong understanding of CI/CD for data, test automation, and modern engineering practices.
• Experience implementing data quality frameworks, observability, and monitoring solutions.
• Familiarity with Git, Infrastructure-as-Code (Terraform/Bicep), and cloud-native tooling.
• Strong communication skills, with the ability to explain technical decisions and trade-offs clearly.
• Experience mentoring or guiding other engineers and contributing to technical best practice.
• 5+ years' experience in data engineering within enterprise-scale environments.
• Relevant certifications (desirable): Azure Data Engineer Associate, AWS Data Analytics Specialty, Databricks Data Engineer (Associate/Professional)
• SC clearance may be needed (or willingness to obtain it)

What we offer:

• Flexible working considered
• Pension - 6%
• Chance to contribute to innovation in the public services
• A company passionate about diversity and inclusion
• Serco discounts which include cinema, merlin entertainment and online shopping discounts, and discounts on mobile phone plans and leisure centre memberships.
• A range of benefits to support the health and wellbeing of you and your family such as Employee Assistance Programme, Simply Health Cash Plans, and more.
• A wealth of career development training to suit your future aspirations. These range from role specific training, leadership coaching, formal study and much more to support you to build your career with Serco.
• A safe and supportive culture.

At Serco, everyone plays a vital part in delivering essential public services that make a real difference. As a global team of 50,000 people across Defence, Transport, Justice, Immigration, Healthcare, Leisure and Citizen Services, we're united by a shared purpose: to improve the services that communities rely on. Guided by our values of Trust, Care, Innovation and Pride, our people make meaningful impact, every single day. Our inclusive recruitment commitment: We prioritise your skills and potential. As a Disability Confident Leader, the Top Great British Employer of Veterans 2026, and a Gold Inclusive Employer Standard holder, we champion equal opportunities for all. We offer flexible and hybrid working where possible and make reasonable adjustments throughout recruitment and employment.