> Markdown version of [/jobs/ext/4639010-ml-inference-engineer-terminal-bench-task-designer-vllm-sglang-tensorrt-llm](https://www.wearedevelopers.com/jobs/ext/4639010-ml-inference-engineer-terminal-bench-task-designer-vllm-sglang-tensorrt-llm). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# ML Inference Engineer - Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

- **Company:** REMOTE JOBS LLC
- **Location:** United States (Remote available)
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Automation of Tests, Command-Line Interface, Databases, JSON, Python (Programming Language), Pytorch, Large Language Models, Git, SGLang, Machine Learning Operations, TensorRT, Virtual Agents, VLLM, Docker, Web Api
- **Published:** November 8, 2026
- **Apply:** https://www.jobmonkeyjobs.com/career/28169687/Ml-Inference-Engineer-Terminal-Bench-Task-Designer-Vllm-Sglang-Tensorrt-Llm-Any-All-Cities-9382

## About the Role

* 3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield
 * Did this work in the last 3 years and can name the systems, tools, data and rules used
 * Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls
 * Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM
 * Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them
 * Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases
 * Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each
Nice-to-have

 * AI lab or LLM infrastructure experience
 * Contributions to PyTorch, vLLM or SGLang
 * GPU profiling experience
 * Docker and Git proficiency
 * Experience writing automated tests
Process

 * Task vetting instead of an interview: complete a vetting task so the client can assess your task design
 * Paid per task; each task takes about 60 to 75 minutes
 * Work is delivered task by task, with reviewer feedback on submissions
 * Expected start in early November 2026

## Description

* Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.
 * Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.
 * Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.
 * Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.
 * Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.
 * Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.