DevOps Engineer - AI Model Evaluator

Mercor

Wien, Austria

28 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

2 years minimum

Compensation

€176,800.0

Working hours

Regular working hours

Job source

www.adzuna.at

Tech stack

Artificial Intelligence Amazon Web Services Microsoft Azure Cloud Engineering Continuous Integration Cursor (Graphical User Interface Elements) DevOps Reliability Engineering Datadog Google Cloud Kubernetes Infrastructure Automation Frameworks

+1 more

Terraform

Job description

Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation.
Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
Compare outputs from multiple frontier models to assess strengths and weaknesses.
Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Requirements

Must-Have

2+ years of professional DevOps, SRE, or Cloud Engineering experience.
Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
Regular use of AI coding agents like Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

Experience supporting production-scale systems.

About the company

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D’Angelo, Larry Summers, and Jack Dorsey.