> Markdown version of [/jobs/ext/4549826-senior-staff-backend-engineer-distributed-system](https://www.wearedevelopers.com/jobs/ext/4549826-senior-staff-backend-engineer-distributed-system). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior/Staff Backend Engineer - Distributed System

- **Company:** ZETTABYTES, INC.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Salary:** $200,000.0 - $265,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Software Applications, Systems Engineering, Cloud Computing, Computer Clusters, Data Centers, Linux, Distributed Systems, Memory Management, Design of User Interfaces, Python (Programming Language), Machine Learning, Performance Tuning, AI Infrastructure, Graphics Processing Unit (GPU), Computer Network Operations, Backend, Usage Tracking, Containerization, Kubernetes, Graphql, Machine Learning Operations, Front End Software Development, Api Design, Model Inference, Scheduling Algorithms, Docker
- **Published:** November 3, 2026
- **Apply:** https://www.careerbuilder.com/job-details/senior-staff-backend-engineer-distributed-system-new-york-ny--54e8feb2-e9ad-4156-a45f-a182160189d2

## About the Role

Distributed Computing, Systems Engineering, Artificial Intelligence (AI), High Tech Industry, GPU (Graphics Processing Unit), Computer Software, Network Operations Center, Cloud Computing, OEM (Original Equipment Manufacturer), Application Programming Interface (API), * 5+ years of backend engineering experience with distributed systems

 * Strong proficiency in Go, Python or similar backend languages
 * Experience with resource scheduling, orchestration and API design (REST, GraphQL, gRPC)
 * Understanding of hardware constraints and system optimisation
 * Linux systems knowledge and containerisation experience (Docker, Kubernetes)
 * Comfort working with expensive resources where efficiency directly affects cost
 * Excitement about novel problems in AI infrastructure and a start-up mindset, comfortable with ambiguity and rapid iteration

Bonus Qualifications

 * GPU or HPC cluster management, or GPU virtualisation and sharing technologies
 * Understanding of ML and AI workload patterns and requirements
 * Background in performance optimisation for compute-intensive workloads
 * Experience with high-value resource allocation, billing or metering systems

## Description

* AI computing company: we turn GPU hardware into reliable, high-utilisation AI compute through software, orchestration and full lifecycle operations
 * What we offer: enterprise GPU cloud and AI inference, software for data centre owners to run their own GPU clouds, turnkey and sovereign AI data centres, edge computing and GPU financing
 * Who we work with: GPU owners, data centre operators, enterprises, OEMs, integrators and financiers across multiple markets
 * How we work: fast-moving and high-accountability; we value speed, ambition, clarity, ownership and measurable outcomes

Role Summary

Zettabyte's mission is to make AI compute ubiquitous, seamless and limitless: a cloud where AI just works, anywhere, anytime.

The Senior/Staff Backend Engineer leads the design and implementation of the high-scale backend services and distributed architectures that orchestrate GPU clusters for AI workloads. The role creates APIs that handle GPU allocation, memory management, compute scheduling and multi-tenant isolation.

It tackles problems such as sharing expensive GPU resources efficiently across users, handling GPU memory constraints for large models, and keeping quality of service when workloads compete for compute.

Reporting Line

Reports to: Co-CTO and CPO, * Developer APIs: design APIs that abstract complex GPU operations into simple developer experiences.

 * Scheduling: build scheduling algorithms that maximise GPU utilisation while meeting SLAs.
 * Resource management: develop resource management for GPU lifecycle, provisioning, allocation, scheduling and release.
 * Usage and billing: create usage tracking and billing systems for GPU-hours, memory usage and compute utilisation.
 * Monitoring and recovery: implement GPU-specific metrics, health checks and automatic failure recovery.
 * Multi-tenancy: build resource isolation, quota management and fair scheduling.
 * Model serving: optimise cold starts and implement efficient model loading strategies.
 * Interfaces: expose complex infrastructure through intuitive interfaces with frontend engineers, using AI-assisted coding tools.