> Markdown version of [/jobs/ext/95156-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/95156-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Secret Intelligence Service - **Location:** London, UK (Remote available) - **Experience:** Expert - **Salary:** £74,624.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Information Engineering, Python (Programming Language), Large Language Models, Data Pipelines - **Published:** May 14, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=b80038add2360088 ## About the Role Do you have experience in Python?, You're a curious, motivated person who enjoys solving problems in unstructured environments where no two challenges look the same. You're comfortable working independently, building data pipelines and developing robust data models, with strong core technical skills grounded in areas like data modelling or Python-based ecosystems rather than low-code tools. You bring a creative, inquisitive mindset and a habit of learning by doing - prototyping, experimenting, and continuously building on your understanding. You feel comfortable engaging with both technical and non-technical audiences, translating complex ideas into clear, actionable insights, and you enjoy shaping solutions collaboratively. At this level, you'll bring either deep technical expertise or strong leadership experience, whether that's mentoring others or managing people. You'll also have exposure to modern AI and LLM-enabled workflows, including testing, evaluation, and ethical considerations., * You'll be required to reach the minimum pass mark for the Online Situational Judgement Test (SJT), which assesses your ability to problem-solve. * Demonstrate technical expertise in data engineering, which consistently demonstrates a high level of complexity - to be assessed at application sift * Demonstrate the ability to provide direct engineering leadership, support junior engineers and shape best practice across teams - to be assessed at application sift * Demonstrate developed understanding and adoption of AI with an ability to work with a range of models, where performance is evaluated and managed - to be assessed at application sift. ## Description Enhancing how we operate today will better prepare us for the challenges of tomorrow. That's why we're creating dedicated teams focused on bringing Artificial Intelligence (AI) into some of our most mission critical areas. As a Lead Data Engineer, you'll work in a truly unique environment where data supports MI6's frontline teams and core enterprise operations. Day to day, you'll lead the design, build and optimisation of secure, scalable data pipelines across complex environments, ensuring sensitive, high-volume data is onboarded, processed and used safely and effectively. You'll work closely with operational officers, analysts, engineers and partners to integrate new data sources and enable advanced, AI-driven workflows. Acting as a bridge between technical and non-technical stakeholders, you'll help translate requirements into practical, high-impact solutions. Along with leading technical delivery, you'll provide direct engineering leadership, mentoring junior engineers and shaping best practice across teams. You'll also support the evaluation and integration of LLMs and agentic AI systems, including testing, ethics and policy considerations, while driving exploration and prototyping of emerging technologies to build future capability., We take professional development seriously, and you'll be joining a community that invests in helping people grow their skills. You'll have access to a broad mix of learning opportunities, including in-house and online training, technical deep dives and development pathways shaped around your interests and the needs of the organisation. This includes support to strengthen your capability across modern AI techniques and wider security practices, with a variety of different courses on offer. We use the Government Digital and Data Profession Capability Framework (formerly known as DDaT) as a compass to provide guidance and tools to support you. You can find out more about the framework here: Government Digital and Data Profession Capability Framework - https://ddat-capability-framework.service.gov.uk/. You'll also have opportunities to work with external professional bodies and training providers who offer routes to specialist accreditation, advanced AI training or broader professional recognition. Whether you choose to deepen your expertise in a specific technical area or broaden your capability across new domains, you'll have the guidance and support you need to progress. Alongside formal training, you'll benefit from mentoring, peer support and a collaborative community that shares knowledge openly. You'll have opportunities to rotate across different teams, explore emerging technologies and contribute to shaping how MI6 evolves its AI and data-science capability. As you develop, you'll have the space to experiment, expand your expertise and play an active role in strengthening our AI practice across the organisation., * An initial online application form including pre-screening questions to ensure you meet our eligibility criteria * Online Situational Judgement Test (SJT) rating the appropriateness of your responses to a series of short scenarios. The SJT tests criteria important for all roles in our organisation * Application sift looking at your skills and motivation for the role and organisation * If you pass, you'll be invited to an interview where you'll be asked questions relating to your motivations and our core competencies * If you pass, you'll be invited to an in-person technical skills-based interview, based on the Digital and Data Profession Capability Framework for the role * If successful you will receive a conditional offer of employment Please note, you must successfully pass each stage of the process to progress to the next. Your application may take around 6 - 9 months to process including vetting, so we advise you continue any current employment until you have received your final job offer. ## Related Videos - [Why and when should we consider Stream Processing frameworks in our solutions](https://www.wearedevelopers.com/videos/1085-why-and-when-should-we-consider-stream-processing-frameworks-in-our-solutions) - [Creating Industry ready solutions with LLM Models](https://www.wearedevelopers.com/videos/899-creating-industry-ready-solutions-with-llm-models) - [Edit Your Future: Queerverse Radical AI](https://www.wearedevelopers.com/videos/909-edit-your-future-queerverse-radical-ai) - [Data Science, ML & AI in the Oil and Gas Industry at NDT Global - Dr. Katja Träumner](https://www.wearedevelopers.com/videos/1308-data-science-ml-ai-in-the-oil-and-gas-industry-at-ndt-global-dr-katja-traumner) - [Python-Based Data Streaming Pipelines Within Minutes](https://www.wearedevelopers.com/videos/1233-python-based-data-streaming-pipelines-within-minutes) - [Behind the Curtain: How Recruiters Shortlist Devs in an AI-Driven Market and How You Stand Out](https://www.wearedevelopers.com/videos/100229-behind-the-curtain-how-recruiters-shortlist-devs-in-an-ai-driven-market-and-how-you-stand-out) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Prompt Engineering is a Job of the Past](https://www.wearedevelopers.com/magazine/342-prompt-engineering-is-a-job-of-the-past) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)