> Markdown version of [/jobs/ext/1262139-devops-engineer-ai-model-evaluator](https://www.wearedevelopers.com/jobs/ext/1262139-devops-engineer-ai-model-evaluator). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# DevOps Engineer - AI Model Evaluator

- **Company:** Mercor
- **Location:** Wien, Austria (Remote available)
- **Experience:** Experienced
- **Salary:** €176,800.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Microsoft Azure, Cloud Engineering, Continuous Integration, Cursor (Graphical User Interface Elements), DevOps, Reliability Engineering, Datadog, Google Cloud, Kubernetes, Infrastructure Automation Frameworks, Terraform
- **Published:** July 14, 2026
- **Apply:** https://www.adzuna.at/details/5785025805

## About the Role

Must-Have

 * 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
 * Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
 * Regular use of AI coding agents like Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
 * Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

 * Experience supporting production-scale systems.

## Description

* Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
 * Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation.
 * Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
 * Compare outputs from multiple frontier models to assess strengths and weaknesses.
 * Apply professional engineering judgment to realistic infrastructure engineering scenarios.


## Related Videos

- [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Debugging in the Dark](https://www.wearedevelopers.com/videos/1658-debugging-in-the-dark)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Agentic DevOps: How AI-Powered Automation Transforms Software Delivery on GitHub and Azure](https://www.wearedevelopers.com/videos/1539-agentic-devops-how-ai-powered-automation-transforms-software-delivery-on-github-and-azure)
- [DevOps for AI: running LLMs in production with Kubernetes and KubeFlow](https://www.wearedevelopers.com/videos/1222-devops-for-ai-running-llms-in-production-with-kubernetes-and-kubeflow)

## Related Articles

- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)
- [Dev Digest 162: AI careers, MCP, AWS best practices & floppy sweaters](https://www.wearedevelopers.com/magazine/571-dev-digest-162-ai-careers-mcp-aws-best-practices-floppy-sweaters)
- [ Dev Digest 213: Petrol Prices, Agentic Workflows, AI Skills and CODE100!](https://www.wearedevelopers.com/magazine/718-dev-digest-213-petrol-prices-agentic-workflows-ai-skills-and-code100)