Lead Data Engineer

We are looking for a motivated individual with excellent data engineering and infrastructure skills, who can be independently forward deployed across NHS hospitals in London to lead development of infrastructure and data pipelines, working in SQL/dbt and using language AI for curating unstructured data. You will work on a pan-London Snowflake platform, solving key technical challenges to enable data-driven value for a population of >10 million., The Lead Data Engineer is a senior technical role that will:

• Design and lead on technical objectives for the AI Centre and Health Data for London
• Lead development of cloud data infrastructure and ELT pipelines across London hospitals and platforms
• Work with a dedicated AI Centre team to deploy language AI technologies for extracting and standardising unstructured clinical records
• Provide expert technical support for the standardisation of London data into research data models
• Co-ordinate, support, and upskill local analysts/engineers in cross-London collaborations to ensure alignment on projects and timelines
• Build robust technical solutions for automation of data pipelines and cohort creation for London research data delivery
• Contribute to deployment architectures for live tools built on top of London data platforms
• Contribute to academic publications, stakeholder presentations, and help to produce materials that support public, patient, and community engagement, such as blog posts

• Relevant technical degree
• Proficient in SQL, dbt, Python, and orchestration frameworks
• Proficient in at least one modern cloud data platform
• Expertise in on prem/cloud infrastructure management
• Experience working in agile development teams with good development practices
• Expertise in NHS data models / data standards
• Ability to effectively break down complex analyses for non-technical stakeholders
• A strong desire to create real-world positive impacts for patients and the NHS

Desirable Criteria

• Expertise in software engineering, including RESTful API development
• Expertise in FHIR / HL7 development
• Expertise working with NLP pipelines for unstructured medical records.
• Experience working with Real-World Data or EHR databases
• Experience building OMOP common data model pipelines

This is an exciting opportunity to join a leading health data engineering team, based at GSTT, but operating across Health Data for London and the AI Centre., AI Centre for Value-Based Healthcare The AI, Data & Digital Innovation directorate is made up of data and technology experts - based in GSTT but working closely as a team with KCH and KCL. The team forms part of the Artificial Intelligence Centre for Value-Based Healthcare - a consortium of NHS, academic, and industry partners from across the UK. This consortium offers expert professional technical delivery across data engineering, data science & AI development, and software engineering. Programmes include region-wide infrastructure delivery of cloud and federated platforms, multi-modal Real-World Data engineering, foundation model development, and development of different Language AI solutions. London / GSTT Snowflake Platform A secure data and research cloud platform that provides access to some of the broadest and deepest data in the NHS, including low latency patient-level data flows from primary care, linked to Acute Trust data. Secure Data Environment (SDE) for London The London SDE is a data, research, and analytics ecosystem that unites data across the London region. It includes Health Data for London are the next iteration of this programme, delivering one of the best and most diverse research data assets in the world that links data across care pathways for more than 10 million patients. The AI Centreis commissioned to deliver multi-modal data integrations for Health Data for London.