Senior Data Engineer | Python | Azure Data Factory | Kafka | PostgreSQL | Elasticsearch | Active SC

Senior Data Engineer

Ledbury/Malvern/Gloucester/Cheltenham Area On-site - 5 Days Per Week Up to £85,000 + Benefits Security Clearance: Active SC Clearance Required

Methods Business and Digital Technology is a £100M+ IT Services Consultancy delivering complex digital transformation programmes across government and regulated sectors. We are seeking an experienced Senior Data Engineer to join a growing team supporting the delivery of secure, mission-critical data platforms.

This role requires an individual who is comfortable operating across hybrid environments, combining on-premises infrastructure with Azure cloud technologies to deliver scalable, secure and high-performing data solutions.

The Role:

As a Senior Data Engineer, you will design, build and maintain sophisticated data infrastructure supporting analytics, reporting and operational decision-making. You will work across the full data life cycle, developing robust data pipelines, implementing automation and supporting Real Time data streaming capabilities.

Responsibilities:

Design, develop and maintain scalable data pipelines using Python, Go and Azure Data Factory Develop ETL and ELT processes supporting structured, semi-structured and unstructured data Implement and manage hybrid data solutions across on-premises and Azure environments Build and maintain PostgreSQL database solutions Manage Elasticsearch deployments and optimisation Implement workflow automation using Azure Data Factory Develop and support Real Time event streaming solutions using Kafka and NATS Support security, authentication and access management using technologies such as Keycloak Collaborate with technical teams to deliver resilient, secure and scalable data platforms

Skills:

Strong commercial Python development experience Experience developing ETL/ELT solutions within enterprise environments Azure Data Factory expertise Experience with Kafka and/or NATS event streaming technologies PostgreSQL administration and development experience Elasticsearch experience Strong understanding of data security, access controls and platform governance Experience operating within hybrid cloud and on-premises environments Active SC Clearance

Desirable:

Azure certifications Experience with Go development DevOps and automation experience Experience supporting highly secure government or defence environments Willingness and eligibility to progress to DV Clearance

Benefits:

Up to £85,000 salary Work on nationally significant programmes Join a growing and highly respected consultancy Exposure to modern cloud, data and platform technologies Career development and certification support Opportunity to progress through higher levels of security clearance

Please note: Active SC Clearance is mandatory for this position. Candidates must also be willing to undergo DV clearance as part of future project requirements.

Strong commercial Python development experience Experience developing ETL/ELT solutions within enterprise environments Azure Data Factory expertise Experience with Kafka and/or NATS event streaming technologies PostgreSQL administration and development experience Elasticsearch experience Strong understanding of data security, access controls and platform governance Experience operating within hybrid cloud and on-premises environments Active SC Clearance

Desirable:

Azure certifications Experience with Go development DevOps and automation experience Experience supporting highly secure government or defence environments Willingness and eligibility to progress to DV Clearance

Up to £85,000 salary Work on nationally significant programmes Join a growing and highly respected consultancy Exposure to modern cloud, data and platform technologies Career development and certification support Opportunity to progress through higher levels of security clearance

Please note: Active SC Clearance is mandatory for this position. Candidates must also be willing to undergo DV clearance as part of future project requirements.