DataOps Engineer - Data Platform Reliability

Valarian Technologies
UK
1 day ago
Apply on startup.jobs
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Airflow Cyber Security Databases Continuous Integration Data as a Services Data Integrity Linux Graph Database Python (Programming Language) Key Management PostgreSQL
+13 more
Enterprise Messaging Systems Octopus Deploy OpenShift Prometheus Perplexity AI Shell Script SQL Databases Grafana Database Performance Git Kubernetes Data Management Terraform

Job description

We are bringing a frontier AI experience inside secure boundaries for the first time. Our technology supports mission critical workflows across government, healthcare, and enterprise, including:

  • Scaling open-weight models without external inference endpoints.
  • Putting frontier class AI to work across organisations through partnerships with other leading AI startups such as Perplexity.
  • Increasing operational resiliency for air gapped, on prem, and cloud based systems.
  • Securing intellectual property and maintaining auditability for high stakes enterprise environments.
  • Facilitating secure information flow during disaster response, M&A due diligence, and privileged legal communications.

We’ve only scratched the surface. Built in Britain by veterans of the Ministry of Defence, British Forces, Darktrace, and elite engineering backgrounds, we are driven by a shared mission to safeguard western democracy . Whether you join our London HQ or our wider global organisation, you’ll be part of a collaborative, high performing team creating the cutting edge software and infrastructure that defines the frontier of enterprise AI.

The Role

You will make the data services and processing workloads supporting ACRA and Valarie reproducible to deploy, observable in production and safe to recover. You will work closely with data engineers and ACRA platform engineers across customer-controlled environments.

What you’ll do:

  • Build and maintain CI/CD and GitOps workflows for data applications, pipelines, configuration and database changes, including validation, promotion and rollback.
  • Develop reusable tooling for scheduling, dependency management, retries, checkpoints, replay and backfills.
  • Establish monitoring and actionable alerts for pipeline freshness, completeness, processing latency, failures and resource usage.
  • Operate workload-owned data services and coordinate with ACRA platform owners for shared database, messaging and storage services.
  • Implement and exercise recovery procedures, including backup restoration, interrupted processing and failed upgrades.
  • Integrate data workloads with ACRA’s identity, secrets and network controls, preserving enclave isolation and authorised access.
  • Investigate production incidents, improve runbooks and automate recurring operational work.

Requirements

  • Demonstrable experience operating production data pipelines or data platforms, with ownership of reliability and incident resolution.
  • Strong practical Kubernetes, container and Linux skills, including troubleshooting workloads, networking, resource limits and persistent storage.
  • Experience building automated delivery workflows using Git, CI/CD and declarative configuration; familiarity with Helm, GitOps or equivalent approaches.
  • Python and shell scripting for automation, plus sufficient SQL to investigate processing failures and database performance.
  • Experience with workflow orchestration, such as Airflow, Dagster, Prefect or an equivalent system.
  • Understanding of reliable processing patterns: idempotency, bounded retries, duplicate delivery, incremental processing and safe replay.
  • Experience implementing metrics, logs, dashboards and alerting, with meaningful service and data reliability targets.
  • Practical experience with stateful systems, controlled migrations, backup and recovery, secrets management and least-privilege access.

Nice to have:

  • OpenShift, on-premises deployments or environments with restricted connectivity.
  • ArgoCD, Terraform, Prometheus, Grafana or OpenTelemetry.
  • Operating PostgreSQL, graph databases, document stores or messaging systems.
  • Delivering software in regulated or security-sensitive customer environments.

Benefits & conditions

Our benefits are designed to ensure our employees feel taken care of and are proud to be a part of the Valarian team. We are committed to consistently enhancing our benefit package, taking into account the overall well-being and needs of our teammates. Here are the key benefits accessible to all employees at Valarian Technologies:

  • Equity - because you have the right to own what you’re building
  • A competitive salary - because we value your unique skills
  • Employer pension contributions - because you deserve a secure future
  • Private health insurance - because your health is important to us
  • Hybrid work setup - because everyone has different needs
  • Gym memberships - because staying active fuels productivity
  • Rewarding company retreats and meetups that respect your work/life balance - because we love getting to know each other

Life at Valarian

Our culture is built on inclusivity, compassion and flexibility - we want everyone to be empowered to achieve their goals at Valarian.

The work we do is vital, but so are the connections that make it happen. We thrive on the shared energy, spontaneous conversations, and mutual trust built when we spend time together. We operate on an office first hybrid model, regularly gathering in our London office to support one another and collaborate.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on startup.jobs
Prepare application

Good distractions

Loading talks and stories from around this role…