> Markdown version of [/jobs/ext/3883891-devops-engineer-ai-model-evaluator](https://www.wearedevelopers.com/jobs/ext/3883891-devops-engineer-ai-model-evaluator). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# DevOps Engineer - AI Model Evaluator

- **Company:** Mercor, Inc.
- **Location:** San Francisco, CA, United States (Remote available)
- **Experience:** Experienced
- **Salary:** $176,800.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Microsoft Azure, Cloud Engineering, Continuous Integration, Cursor, DevOps, Reliability Engineering, Datadog, Google Cloud, Claude Code, AI Coding Agents, Kubernetes, Infrastructure Automation Frameworks, Google Gemini, Terraform
- **Published:** October 23, 2026
- **Apply:** https://www.thejobnetwork.com/job/20f9e3c1-bb67-4bfc-9ab9-d9e897b296b5/devops-engineer-ai-model-evaluator

## About the Role

Must-Have

 * 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
 * Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
 * Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
 * Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

 * Experience supporting production-scale systems.

## Description

* Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
 * Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
 * Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
 * Compare outputs from multiple frontier models to assess their strengths and weaknesses.
 * Apply professional engineering judgment to realistic infrastructure engineering scenarios., PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.