WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

DevOps Engineer - AI Model Evaluator

Mercor Bekijk Alle Vacatures
Amsterdam, Netherlands
1 day ago
Apply on www.careerjet.nl
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
2 years minimum
Compensation
€14,733.0
Working hours
Regular working hours
Job source
www.careerjet.nl

Tech stack

Amazon Web Services Microsoft Azure Cloud Engineering Continuous Integration Cursor (Graphical User Interface Elements) DevOps Reliability Engineering Datadog Google Cloud Kubernetes Infrastructure Automation Frameworks Terraform

Job description

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
  • Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess their strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Requirements

Must-Have

  • 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
  • Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
  • Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.nl
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

Moments

Explore playlists

2:17 min

Mapping the maturity roadmap for scaled devops adoption

Dominik Krichbaum Dominik Krichbaum · World Congress 2026 Europe

1:36 min

Visualizing memory limits and isolating suspicious endpoints

Dina Matveev Dina Matveev · Europe 2026 Virtual

1:34 min

Essential commands for running and testing Terraform configurations

Hennie Francis · LIVE

4:48 min

Automating large language model evaluations in DevOps

Oliver Will Oliver Will · World Congress 2025

3:18 min

Scaling global network engineering through DevOps culture

Stuart Clark · LIVE

3:22 min

Evaluating advanced artificial intelligence platforms for daily recruitment

Rudi Bauer Rudi Bauer +1 · Cappuccino with HR

Videos

See all
Infrastructure as Code: The Developer's Secret Weapon thumbnail
45:04
LIVE • almost 2 years ago

Infrastructure as Code: The Developer's Secret Weapon

Hennie Francis

Debugging in the Dark thumbnail
8:02
World Congress 2025 • 12 months ago

Debugging in the Dark

Nishil Patel

From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform thumbnail
22:57
World Congress 2026 Europe • about 1 month ago

From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform

Dominik Krichbaum

Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps thumbnail
35:19
Coffee With Developers • 3 months ago

Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps

Hazal Mestci, Raph Terrier

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow thumbnail
34:21
LIVE • almost 2 years ago

DevOps for AI: running LLMs in production with Kubernetes and KubeFlow

Aarno Aukia

Implementing Feature Environments with AWS and Terraform thumbnail
42:33
LIVE • almost 4 years ago

Implementing Feature Environments with AWS and Terraform

Thomas Hartenstein

Related articles

See all