Data Engineer

This is more than a job - it's a mission. You will be part of a high-impact, collaborative environment, where we expect everyone to live the values and standards of the British Army. Every person in our team plays a critical role in delivering OMNIA's vision; designing, delivering, and transforming collective training so the British Army is ready to fight and win. You'll work in a matrix organisation and report operationally through OMNIA Training and functionally to the Data Lead. Ultimately, you'll work for the British Army, championing innovation, and helping shape the future of military collective training., Collaborate with engineering, simulation, customer, third party and training teams to ensure seamless data management and integration across military training environments. Ensuring the security and compliance of data environments through the implementation of appropriate security controls and governance frameworks. Author, review and contribute to technical documentation. Coordinate with cross-functional engineering teams-including QA, development, operations, and business SMEs. Assist in the design and implementation of scalable, high-performance data platforms and pipelines. Define and enforce data engineering standards, best practices, and governance frameworks including data lifecycle design, implementation and management. Implement the design and optimisation of data storage, processing, and retrieval strategies. Collaborate with cross-functional teams (data scientists, analysts, software engineers) to align data solutions with business needs. Drive adoption of modern data technologies (e.g., cloud-native platforms, streaming, orchestration tools). Contribute to data quality, security, compliance, and lineage practices across the organisation. Provide technical input and guidance to data engineering team members. Evaluate, select, and integrate new tools, frameworks, and platforms for the data ecosystem. Act as a subject matter expert in data strategy, influencing design decisions. Any other duties required to meet the needs of the programme., The OMNIA Data Engineer will provide hands-on technical oversight and management across the Army Collective Training Service (ACTS) data solutions. These will be prominently MODCloud hosted utilising Cloud Services from AWS, Azure, OCP where appropriate. Reporting to the Data Architect, you will play a key role in ensuring the effective design, assurance, transformation, and delivery of data across ACTS services and capabilities. This includes supporting the integration and operational use of data from legacy systems through API-driven solutions.

This role requires a systems-thinking mindset, strong stakeholder engagement skills, and the ability to work across engineering teams in a complex and evolving environment. Given the programme's focus on modelling and simulation, familiarity with relevant standards and technologies is highly desirable. Essential Skills and Experience: Hands-on experience with cloud platforms (AWS, Azure, GCP, OCI) and cloud-native data services. Extensive experience designing and building large-scale data pipelines and platforms. Strong expertise in SQL, data modelling, and database optimisation (relational, vector and NoSQL). Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Deep knowledge of cloud data platforms (AWS, Azure, or GCP) and associated services. Strong programming skills in Python, Java, or Scala for data engineering. Hands-on experience with data orchestration and workflow management tools (e.g., Airflow, Dagster, Prefect). Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion / ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge of data transformation and ETL pipelines and APIs. Desirable Skills and Experience: Degree in Data Engineering or equivalent professional accreditation such as CEng. Experience with graph databases and advanced query languages (e.g., Neo4j, Gremlin). Knowledge of machine learning data pipelines and MLOps practices. Familiarity with data virtualisation and data mesh concepts. Hands-on experience with containerisation and orchestration (Docker, Kubernetes, Red Hat OpenShift and Ceph). Exposure to infrastructure-as-code tools (Terraform, CloudFormation). Experience with BI/visualisation tools (e.g., Tableau, Power BI, Looker, Elastic Stack). Knowledge of compliance frameworks (GDPR, HIPAA, CCPA) and their impact on data systems. Strong background in performance tuning for high-throughput, low-latency data systems. Microsoft Certified: Azure Data Engineer Associate, AWS Certified Data Engineer, IBM Data Engineering Professional Certificate or similar.

At OMNIA® Training, we've brought together some of the UK's most innovative defence training organisations under one powerful mission: to transform the British Army's training system and create the best-trained Army in the world. OMNIA are redefining the British Army's collective training. To do that, we are looking for the best and brightest minds from across the UK. We are backed by British innovation and powered by world-class experts, like you. OMNIA is at the heart of the UK's bold Land Industrial Strategy.