Senior Software Engineer

• Lead the design and development of scalable, cloud-native data services supporting a federated platform.
• Define architectural patterns, engineering standards, and best practices that will scale across the wider organisation.
• Design and build distributed server-side services using modern software engineering practices.
• Architect and develop federated data services capable of securely querying and processing data across multiple environments.
• Design high-performance data processing and streaming workflows using technologies such as Apache Spark, Apache DataFusion, or similar distributed query frameworks.
• Design and implement Kubernetes-based services and containerised workloads.
• Build secure APIs and backend services supporting authentication, authorisation, governance, and audit requirements.
• Contribute to the design of distributed systems, data pipelines, and platform architecture.
• Establish CI/CD pipelines, Infrastructure as Code, automated testing, and deployment practices.
• Evaluate and integrate open-source technologies and distributed data frameworks.
• Mentor engineers and promote engineering excellence across the team.
• Work closely with architecture, product, security, and external stakeholders.

Technologies:

• AWS
• Architect
• Azure
• Backend
• C#
• CI/CD
• Cloud
• Docker
• Flink
• GCP
• Support
• Java
• Kafka
• Kubernetes
• Python
• Scala
• Security
• Spark
• DevOps, We are looking for an experienced Lead Backend Software Engineer to join a high-profile programme building secure, cloud-native data services that will support data-driven research and innovation at national scale. This is a founding engineering role where we will shape the architecture, engineering standards, and core data services from the ground up. You will work closely with the Engineering Manager and a small team of senior engineers to establish the technical foundations for a complex federated data platform operating within a highly regulated environment. The role is based in London with two days per week onsite. It is an initial 6-month contract with likely extension, offering a day rate of £700 to £800 outside IR35.

• Experience as a Lead Engineer, Principal Engineer, Staff Engineer, Software Architect, or in a similar senior technical role.
• Strong server-side software engineering experience, with flexibility across programming languages such as Python, Java, Go, Scala, C#, or similar.
• Experience designing and building distributed data services or large-scale backend platforms.
• Experience with Apache Spark, Apache DataFusion, Trino, Presto, or similar distributed data processing technologies.
• Strong architecture and design experience across distributed systems and data services.
• Experience designing scalable data processing or data streaming workflows.
• Experience building cloud-native applications on AWS, Azure, or GCP.
• Experience with Kubernetes, Docker, and containerised environments.
• Experience building and operating CI/CD pipelines and Infrastructure as Code.
• Strong understanding of distributed systems, scalability, resilience, and secure-by-design principles.
• Comfortable making architectural decisions within evolving, fast-paced environments.
• Excellent communication skills with the ability to work across technical and non-technical teams.
• Experience building federated data services or federated data platforms.
• Experience within healthcare, life sciences, research, government, or other highly regulated environments.
• Knowledge of Trusted Research Environments (TREs), Secure Data Environments (SDEs), or similar secure data platforms.
• Experience with data governance, metadata services, data catalogues, or secure data access models.
• Exposure to event-driven architectures and streaming technologies such as Kafka, Flink, or Spark Streaming.
• Experience evaluating or contributing to open-source technologies.