Consultant - Manager, Data Engineer, AI & Data, Defence & Security

HM Revenue & Customs
London, UK
1 day ago
Apply on www.collegerecruiter.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
3 years minimum
Working hours
Regular working hours

Tech stack

Java (Programming Language) Artificial Intelligence Airflow Amazon Web Services Audit Trail Microsoft Azure BigQuery Cloud Database Code Reuse Computer Programming Continuous Integration Data as a Services
+44 more
Data Architecture Information Engineering Data Governance Extract Transform Load (ETL) Data Security Data Stores Data Systems Data Warehousing Database Design Database Queries DevOps Digital Assets Distributed Computing Environment Github Python (Programming Language) Machine Learning Metadata Meta-Data Management Scala (Programming Language) Software Engineering Data Streaming Technical Data Management Systems Unstructured Data Workflow Management Systems Azure Service Bus Google Cloud Snowflake Apache Spark Gitlab Containerization Pyspark Kubernetes Apache Flink Apache Kafka Apache Nifi Data Management Video Streaming Terraform Azure Synapse Analytics Software Version Control Data Pipelines Docker Amazon Redshift Databricks

Job description

We are seeking talented data engineers to join our multidisciplinary Engineering, AI and Data team, with a focus on clients in the Defence and Security sector.

As a data engineer, you will help clients turn complex, high-volume and sensitive data into trusted, governed and reusable information assets. You will design and build the data pipelines, platforms and products that enable analytics, AI, reporting and operational decision-making.

You will work in multidisciplinary teams across a range of client engagements, often in complex and secure environments. Your role may span discovery, architecture, engineering, delivery and ongoing improvement, depending on the needs of the project and your individual skillset.

As a practitioner in our team, you will:

  • Design, build and maintain high-quality data pipelines, data stores and data products.
  • Ingest, transform, enrich and integrate data from diverse sources, including structured, semi-structured, streaming and unstructured data.
  • Create trusted, standardised and reusable data assets for downstream use cases such as analytics, AI, reporting and operational services.
  • Work with engineers, data scientists, analysts, architects and client stakeholders to deliver scalable data solutions.
  • Apply engineering good practice, including version control, testing, automation, CI/CD and documentation.
  • Help design data architectures that are secure, reliable, scalable and maintainable.
  • Support data governance, including metadata, data quality, lineage, access control and appropriate handling of sensitive data.
  • Monitor, troubleshoot and optimise data systems for performance, reliability and operational use.
  • Bring innovation and practical problem-solving to complex data engineering challenges.
  • Contribute to client delivery, stakeholder relationships, practice development and future propositions.

Requirements

All applicants must hold active UK Developed Vetting clearance.

We are looking for candidates who combine strong hands-on data engineering capability with the ability to communicate clearly, work collaboratively and explain the value of their work to clients.

Essential skills and experience

  • Hands-on experience designing, building and maintaining data pipelines, ETL/ELT processes or workflow orchestration.
  • Practical coding experience in Python, Scala, Java or equivalent, with strong SQL skills.
  • Experience working with diverse data sources and data types, such as batch, streaming, structured, semi-structured or unstructured data.
  • Understanding of data modelling, database design, data warehousing or lakehouse patterns.
  • Experience applying software engineering practices such as version control, testing, reusable code, documentation and CI/CD.
  • Ability to design data solutions that are scalable, reliable, performant and maintainable.
  • Understanding of data security, access control, data quality, metadata, lineage or broader data governance principles.
  • Strong communication skills, including the ability to understand user and business needs and translate them into technical data solutions.
  • Ability to work effectively in multidisciplinary teams and build productive relationships with colleagues and clients.

Desirable skills and experience

We do not expect candidates to have experience across all the areas below, but we are particularly interested in people who can bring depth in one or more of them:

  • Distributed data processing using tools such as Spark, PySpark, Flink or equivalent.
  • Workflow orchestration using tools such as Airflow, Dagster, dbt, NiFi or equivalent.
  • Cloud-based data engineering services across AWS, Azure, GCP or equivalent platforms.
  • Data platforms such as Databricks, Snowflake, BigQuery, Redshift, Synapse or similar.
  • Streaming technologies such as Kafka, Kinesis, Event Hubs or equivalent.
  • Containerisation, DevOps or platform engineering practices using tools such as Docker, Kubernetes, Terraform or GitHub/GitLab pipelines.
  • Building data products or reusable data services for analytics, AI or machine learning use cases.
  • Metadata management, data catalogues, lineage tooling, master data management or data quality frameworks.
  • Working with sensitive, classified or mission-critical data in secure environments.
  • Designing architectures that support auditability, resilience, security and operational monitoring.

About the company

Distinctive thinking, deep expertise, innovation and collaborative working. That’s what connects us. That’s what makes us Deloitte. If you want to help solve some of the biggest tech and transformational challenges around, join us. Together, we’ll make an impact that matters., Deloitte drives progress. Using our vast range of expertise, we help our clients’ become leaders wherever they choose to compete. To do this, we invest in outstanding people. We build teams of future thinkers, with diverse talents and backgrounds, and empower them all to reach for and achieve more.

What brings us all together at Deloitte? It’s how we approach the thousands of decisions we make every day. How we behave, our beliefs and our attitudes. In other words: our values. Whatever we do, wherever we are in the world, we lead the way, serve with integrity, take care of each other, foster inclusion, and collaborate for measurable impact. These five shared values lead every decision we make and action we take, guiding us to deliver impact how and where it matters most., “What attracted me to Deloitte were the endless opportunities and the collective experience of other like- minded individuals. Deloitte’s clients include many of the world’s largest organisations; I wanted to be part of a team that made a difference that I could be proud of.” -Dan, Technology and Transformation

Our commitment to you

Making an impact is more than just what we do: it’s why we’re here. So we work hard to create an environment where you can experience a purpose you believe in, the freedom to be you, and the capacity to go further than ever before.

We want you. The true you. Your own strengths, perspective and personality. So we’re nurturing a culture where everyone belongs, feels supported and heard, and is empowered to make a valuable, personal contribution. You can be sure we’ll take your wellbeing seriously, too. Because it’s only when you’re comfortable and at your best that you can make the kind of impact you, and we, live for.

Your expertise is our capability, so we’ll make sure it never stops growing. Whether it’s from the complex work you do, or the people you collaborate with, you’ll learn every day. Through worldclass development, you’ll gain invaluable technical and personal skills. Whatever your level, you’ll learn how to lead.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.collegerecruiter.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

2:07 min

Inspecting default bridge architectures and custom Docker networks

Oliver Seitz Oliver Seitz · World Congress 2025

6:14 min

Structuring CI/CD pipelines with integrated security and quality checks

Christoph Ruggenthaler · LIVE

6:36 min

Funding open source through GitHub Accelerator and Sponsors

Stormy Peters · World Congress 2023

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

2:34 min

Docker sandbox architecture and microVM environment integration

Manuel de la Peña Manuel de la Peña · World Congress 2026 Europe

Videos

See all

Related articles

See all