Senior Data Engineer - DV Cleared

As a Senior Data Engineer, you will play a key role in designing, building, and maintaining robust data pipelines and platforms. You will work closely with architects, engineers, and security teams to deliver high-performance data solutions that meet stringent security and governance requirements., * Design, develop, and maintain secure, scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi

• Build and manage data ingestion, transformation, and integration processes
• Ensure data quality, integrity, security, and compliance with governance standards
• Collaborate with data architects and security teams within highly secure environments
• Monitor and manage large-scale, Real Time data flows, optimising performance and reliability
• Develop robust data models to support analytics and reporting capabilities
• Create dashboards and visualisations using Kibana to drive operational insights
• Troubleshoot, debug, and tune data pipelines and platform performance
• Implement high availability, backup, and disaster recovery strategies
• Produce clear technical documentation, including architecture, workflows, and security protocols

• Strong experience with the Elastic Stack (Elasticsearch, Logstash, Kibana)
• Proven expertise with Apache NiFi
• Extensive experience designing and building enterprise-scale data pipelines
• Strong understanding of secure data architectures and data governance
• Experience operating within regulated or security-conscious environments
• Ability to design and deliver end-to-end data engineering solutions
• Excellent troubleshooting and performance optimisation skills

Desirable Experience

• Experience working within government, defence, or other highly regulated sectors
• Knowledge of Apache Kafka, Spark, or Hadoop
• Experience with Docker and Kubernetes
• Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK
• Understanding of machine learning algorithms and their implementation
• Experience with Infrastructure as Code tools such as Terraform and Ansible

? £70-£75 per hour ? Outside IR35 ? 12-month rolling contract ? Compressed working available (3 long days, paid for 5) ? Opportunity to work on complex, mission-critical data platforms ? Long-term contract stability within a high-profile programme