Senior Data Engineer
Role details
Job location
Tech stack
Job description
In this role, you will be expected to develop modern data pipelines built to production standards (using Snowflake / SQL / dbt / Python) to harmonise data from highly complex transactional data, including from Cerner and Epic systems. You will be responsible for owning pipeline functions, and regularly engaging with data product end-users to update and improve functionality. Understanding of clinical data quality, best practices in testing and monitoring, and ability to build visualisation applications, are all highly desirable. Your work will be the foundation for a wide variety of valuable use-cases, from clinical care to research, and from audit to innovation and AI projects. You will have the opportunity to deploy Large Language Model and NLP tooling orchestration frameworks as part of standard data transformation pipelines, for extracting information from unstructured free text. You will be supported in continuing your personal development within a friendly and highly expert team, particular in areas such as data orchestration, cloud infrastructure, and continuous integration/deployment. There will also be opportunity to mentor and supervise more junior technical staff.
Working for our organisation The AI, Data & Digital Innovation directorate is made up of data and technology experts - based in GSTT but working closely as a team with KCH and KCL. The team forms part of the Artificial Intelligence Centre for Value-Based Healthcare - a consortium of NHS, academic, and industry partners from across the UK. This consortium offers expert professional technical delivery across data engineering, data science & AI development, and software engineering. Programmes include region-wide infrastructure delivery of cloud and federated platforms, multi-modal Real-World Data engineering, foundation model development, and development of different Language AI solutions., + Developing modern data pipelines built to production standards, that harmonises data from highly complex transactional data, including from Cerner and Epic systems
- Designing and building data products that serve frequent end-user needs across clinical analytics, research, and population health
- Develop pipelines that drive value for a wide range of end-user stakeholders, including clinicians, researchers, delivery teams, and population health analysts
- To use expertise in data modelling and health data to build deep insights into health data quality, including implementing tests and real-time monitoring, and producing impactful visualisations and reports
- Lead collaborations with multi-disciplinary teams to discover and extract data from new sources
- Co-ordinate and support local analysts to ensure alignment on projects and timelines
- Develop, deploy, and maintain Large Language Model and NLP tooling orchestration frameworks as part of standard data transformation pipelines, for extracting information from unstructured free text
- Contribute to a culture of shared learning, and support capability building around engineering and analytics within the team
- To use understanding of health data to act as subject matter expert for other teams
- Help to scale exemplar product solutions across other NHS Trusts
- Conduct stakeholder presentations and help to produce materials that support engagement across the Trust and the wider region Guy's and St Thomas' celebrates, respects and values the diversity of its staff and patients. We review our policies, procedures and practices to ensure that all employees, patients and carers are treated equitable according to their needs. We are actively committed to ensuring that no one who applies for a job, works or study's at the Trust, or accesses our services is discriminated against on the grounds of race, ethnicity, nationality, disability, religion or belief, age, gender identity , gender reassignment, sexual orientation, pregnancy and maternity/paternity, or marital/civil partnership. Applications are welcomed from applicants with a disability. We can make reasonable adjustments and offer support and advice in a variety of ways throughout the application process. Equality of opportunity is our policy. As an organisation we are committed to developing our services in ways that best suit the needs of our patients. This means that some staff groups will increasingly be asked to work a more flexible shift pattern so that we can offer services in the evenings or at weekends. Flexible working We are committed to supporting all employees to achieve a healthy work life balance and to work in a way that is best for them and our patients. We will consider all requests to work flexibly, taking in to account the individual's personal circumstances as well the needs of the service. We encourage all prospective applicants to discuss their individual circumstances with the recruiting manager as part of the on-boarding process. Due to recent changes in the UK immigration rules which affect Skilled Worker Visas, Global Business Mobility, Higher Skill Level and Increased Salary Thresholds, please ensure that you are able to meet the requirements to live and work in the UK before applying. Further information about eligibility is available on the UK Government website. Your e-mail address is important to us - We communicate to all job applicants via the e-mail address which has been provided on the application form. Please ensure that you check your e-mail on a regular basis. Please apply for this post by clicking "Apply Online Now."
Requirements
We are looking for a motivated individual with excellent data engineering skills, with enthusiasm and passion to learn. The successful applicant will use a professional tool stack (including Snowflake, SQL, dbt, Python, and Natural Language Processing/Language AI tools) for building and orchestrating data pipelines, from source systems to analyst ready marts which will be used in clinical analytics, research, population health, and to support Real-World Data functions. The postholder will work closely with a range of key stakeholders across GSTT and other NHS Trusts (including Lewisham and Greenwich, King's College Hospital, and others). Essential Criteria
-
Proficient in SQL for data engineering and analytics
-
Experience of using dbt
-
Experience of EHR data and its meaning in clinical contexts
-
Experience with best development practices, CI, and tools such as git Desirable Criteria
-
Experience in Snowflake or other cloud platforms
-
Proficiency in Python
-
Experience of NLP / LLM usage in production settings
-
Experience in data pipeline orchestration