Senior ML Infrastructure Engineer

Chemify Limited
Glasgow, United Kingdom
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Glasgow, United Kingdom

Tech stack

Artificial Intelligence
Amazon Web Services (AWS)
Cloud Computing
Computer Clusters
Computer Security
Linux
Distributed Systems
Job Scheduling
Python
Linux System Administration
Machine Learning
Molecular Modelling
Azure
Scientific Computating
Server Administration
Management of Software Versions
Workflow Management Systems
Data Logging
Graphics Processing Unit (GPU)
Autoscaling
Delivery Pipeline
Containerization
Data Lake
Kubernetes
Slurm
Machine Learning Operations
Hardware Infrastructure
Data Pipelines
Serverless Computing

Job description

We are hiring a Senior ML Infrastructure Engineer to build, enable and operate the core platform that powers Chemifys machine learning and scientific AI computing workloads. This role sits at the intersection of distributed systems engineering, machine learning infrastructure, scientific computing, and platform engineering.

You will build and operate the operational backbone of the ML platform, ensuring that pipelines run reliably across Kubernetes clusters, onpremise GPU infrastructure, and serverless compute environments. The systems you build will support ML engineers and computational chemists running workloads from largescale model training to molecular simulation.

If you enjoy building complex technical systems at the intersection of ML and scientific computing, working on platform problems that combine distributed systems, cloud and onpremise GPU infrastructure, and real-world scientific workloads, youll thrive here., * ML Pipeline Orchestration : implement routing logic dispatching workloads to appropriate compute backends; maintain workflow reliability including retries, dependency management, and failure recovery.

  • ML Pipeline Orchestration : implement routing logic dispatching workloads to appropriate compute backends; maintain workflow reliability including retries, dependency management, and failure recovery.
  • Linux Administration : Server administration and support including security and scaling.
  • Kubernetes Platform Operations : Operate clusters for ML training, inference, and batch workloads; maintain container build pipelines and GitOps deployment workflows; optimise cluster scheduling, autoscaling, and GPU utilisation.
  • HPC / GPU Compute Integration : Integrate orchestration systems with HPC job schedulers; maintain execution paths for workloads running on GPU clusters; ensure artifacts and results from HPC jobs are captured and versioned.
  • Model & Experiment Lifecycle : Operate model registry and experiment tracking platforms; ensure training runs are reproducible and linked to code and datasets; support promotion of models from staging to production.
  • Data Versioning & Pipeline Traceability : Implement dataset versioning and lineage tracking across ML pipelines; ensure predictions are traceable to model versions and datasets; maintain reproducible ML training pipelines.
  • Platform Tooling & Developer Experience : Develop platform CLI tools and pipeline templates; maintain base container images used for ML workloads; improve developer workflows for ML engineers and scientists.
  • Observability, Security & Governance : Implement monitoring, logging, and alerting across orchestration systems; maintain infrastructure as code for platform resources; ensure workloads are traceable to source code, container images, and execution environments., You will lead a team of Lab Coordinators and Production Technicians responsible for maintaining and improving the lab space at the first Chemifarm, laying the foundation for how future Chemifarms will operate., Reporting to the Inventory and Materials Manager, you will own the lab operations strategy, improve processes and workflows, and have meaningful impact over the at how Chemify labs run and operate.

Requirements

  • Degree in Science, Engineering or related field (or equivalent practical experience).

  • Strong Python engineering skills.

  • Experience operating workflow orchestration platforms.

  • Strong Kubernetes platform experience.

  • Experience with containerisation and CI/CD pipelines .

  • Experience with cloud infrastructure such as AWS & GCP .

  • Experience operating distributed systems in production.

  • Strong Linux systems engineering skills. Beneficial Skills

  • Argo Workflows or Kubernetes workflow engines.

  • SLURM or other HPC job schedulers.

  • ML experiment tracking tools such as Weights & Biases or MLflow.

  • Data versioning or lakehouse technologies such as LakeFS, Iceberg, or Delta Lake.

  • Scientific computing environments.

  • Internal developer platform or CLI tooling experience.

  • Experience in Cyber Security and operating in regulated environments.

About the company

Chemify is revolutionising chemistry. We are creating a future where the synthesis of previously unimaginable molecules, drugs, and materials is instantly accessible. By combining AI, robotics, and the worlds largest continually expanding database of chemical programs, we are accelerating chemical discovery to improve quality of life and extend the reach of humanity., We are a Series B deep-tech company investing in world-class infrastructure and tackling problems at the frontier of AI, robotics, and chemistry. You will have the resources, the data, and the mandate to shape how laboratories at Chemfiy run and are maintained.

Apply for this position