Model Routing Engineer, zPLATFORM
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+10 more
Job description
Model Routing Engineer, zPLATFORMSkip to main content, * AI computing company: we turn GPU hardware into reliable, high-utilisation AI compute through software, orchestration and full lifecycle operations
- What we offer: enterprise GPU cloud and AI inference, software for data centre owners to run their own GPU clouds, turnkey and sovereign AI data centres, edge computing and GPU financing
- Who we work with: GPU owners, data centre operators, enterprises, OEMs, integrators and financiers across multiple markets
- How we work: fast-moving and high-accountability; we value speed, ambition, clarity, ownership and measurable outcomes
Role Summary
The Model Routing Engineer, zPLATFORM builds the services that route each inference request on zPLATFORM to the right model and capacity, balancing quality, latency and cost.
The role works on the request path, the policies that guide routing decisions and the measurement that shows whether those decisions are good.
The Model Routing Engineer partners with inference, platform and product teams to give customers one simple interface across many models.
Reporting Line
Reports to: Co-CTO and CPO, * Routing services: build and operate low-latency, highly available services that direct inference requests across models and regions.
- Routing policies: implement policies that weigh quality, latency, cost and customer preferences, and make them configurable.
- Evaluation: build evaluation sets and online metrics that show the effect of routing choices on output quality and cost.
- Model onboarding: add new open and commercial models to zPLATFORM with consistent APIs and documentation.
- Reliability: handle failover, rate limits and graceful degradation when models or capacity are unavailable.
- Regional and compliance rules: apply data residency and customer restrictions in routing, and follow export-control and sanctions screening requirements with the Export Compliance team.
- Observability: instrument the request path so that latency, errors and cost per request are visible.
- Customer feedback: work with product and solutions teams to turn customer needs into routing features., * Routing decisions are measured against clear quality, latency and cost metrics
- New models are added to zPLATFORM through a documented, repeatable process
- Failover behaviour is tested and documented
- Customers can set routing preferences through the API
Why This Role Matters
- Product core: routing between models is a defining capability of zPLATFORM.
- Customer value: good routing gives customers better answers for less money without extra work on their side.
Working Norms at Zettabyte
- Bias to action: deliver drafts quickly, then iterate on feedback
- Be precise: double-check numbers and facts; sloppy analysis creates downstream cost
- Make it usable: package outputs for decision-making, not data dumps
- Own the process: whoever drives a deliverable manages stakeholders and timelines
- Learn the ecosystem: understand GPUs, cluster operations and the buyer landscape
Requirements
Network Routers, Cloud Computing, Artificial Intelligence (AI), GPU (Graphics Processing Unit), High Tech Industry, Computer Software, Network Operations Center, OEM (Original Equipment Manufacturer), Network Routers, High Availability, * 4+ years of backend or distributed systems engineering in production
- Strong programming skills in Go, Rust or Python
- Experience building low-latency APIs, gateways or proxies at scale
- Working knowledge of LLM APIs, inference serving and evaluation methods
- Comfort with experimentation, A/B testing and data-driven decisions
- Clear written communication of design trade-offs
Bonus Qualifications
- Experience with LLM gateways or model routing products
- Background in recommendation, ranking or search systems
- Familiarity with inference serving frameworks such as vLLM
- Experience with multi-region service deployments
Benefits & conditions
- Compensation: USD 190,000 to 245,000 a year base salary, plus equity
- Benefits: Medical, dental and vision insurance; 401(k) retirement plan; paid time off and US public holidays; equity
- Work authorisation: Required in the United States
- Equal opportunity: Zettabyte is an equal opportunity employer; qualified applicants with arrest or conviction records will be considered in line with the California Fair Chance Act
Model Routing Engineer, zPLATFORM
About the company
Zettabyte is a supplier of the United Nations Global Marketplace (UNGM #1117361) and is committed to aiding the United Nations in achieving its Sustainable Development Goals.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦