Data Engineer

Data Careers Ltd
London, UK
24 days ago
Apply on www.totaljobs.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Compensation
£182,000.0 - £208,000.0
Working hours
Regular working hours

Tech stack

Airflow Microsoft Azure Big Data Information Engineering Data Governance Data Systems Python (Programming Language) Operational Databases Apache Spark Kubernetes Data Pipelines

Job description

We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining an established defence programme requiring additional engineering support across an existing data platform.

You will be responsible for building, maintaining and improving new and existing data pipelines within a secure MoD environment., * Build and maintain robust data pipelines using Python

  • Work across Azure environments on the low side
  • Support data engineering workloads within Kubernetes environments on the high side
  • Develop and maintain workflows using Apache Airflow
  • Work with Apache Spark for large-scale data processing
  • Support the ongoing development and improvement of established data pipelines
  • Help ensure data remains within MoD-owned infrastructure in line with data governance requirements
  • Work within an existing long-running programme requiring additional delivery capacity

Requirements

  • Active DV Clearance
  • Strong Python data engineering experience
  • Experience with Azure
  • Strong knowledge of Kubernetes
  • Experience with Apache Airflow
  • Experience with Apache Spark
  • Strong background building and maintaining production data pipelines

Desirable

  • Previous experience working within a Palantir environment
  • Experience delivering data solutions within secure defence or government environments

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.totaljobs.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:19 min

Introduction to Apache Airflow for advanced orchestration

Alan Mazankiewicz · LIVE

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

1:15 min

Deploying local container pods to Kubernetes clusters

Stevan Le Meur Stevan Le Meur · World Congress 2024

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

1:11 min

Deploying and running Airflow in cloud environments

Alan Mazankiewicz · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

Videos

See all

Related articles

See all