AI Eats the Verifiable First

One property predicts which tasks AI can solve best — and the line it draws runs straight through your job

Every few weeks a stronger coding model lands and the same question comes back around: will AI take my job? It’s the wrong question. The honest answer is “parts of it, and at wildly different speeds.” The question worth asking is which parts, and why? One idea helps us answer that. Not model size, not training data, not raw intelligence — but whether a task can be verified.

Jason Wei — first author of the Chain-of-Thought paper — named it Verifier’s Law, which boils down to:

The easier a task is to verify, the easier it is to train an AI to do it.

A task is “easy to verify” when the verification of a candidate solution is:

• objective — everyone agrees what counts as correct
• fast — doesn’t take much time to check
• scalable — you can check many attempts in parallel
• low-noise — verification score tracks the actual solution quality
• continuous — you can easily rank individual solutions

When those line up, AI excels: competitive math, LeetCode, Sudoku. When they don’t — grading an essay, creating an open-ended design — AI performance falls apart. This wildly uneven, often counterintuitive capability profile of modern AI is what Wei calls the jagged edge of intelligence: brilliant on one task, hopeless on the one next to it.

The same lens answers a far more practical question: how will AI actually change software work? Track that one property — verifiability — and the picture comes into focus.

Why coding was the first big bite

Code is probably the most verifiable thing humans ever write down. It ticks every box on Wei’s checklist, which is exactly why it generates such a strong, consistent signal — driving both RL training and test-time compute (spending more compute on retries at inference) inside an agentic loop.

As developers, we’ve relied on this verifiability long before AI showed up — running linters or compilers, executing test suites, fixing errors and warnings, and repeating this over and over until everything goes green.

Coding agents — Claude Code, Cursor, Codex, OpenCode — have just automated us out of this inner loop. They propose a patch, run the suite, keep what’s green, read the errors, and try again. None of us writes correct code on the first try either — that’s what the compilers, type checkers, and tests are for. Ask me to multiply two ten-digit numbers in my head and I’ll fail — give me pen and paper and I might get it right. The agentic loop is that pen and paper, plus the patience to retry.

It’s the algorithmic idea behind AlphaGo Zero, except the game board is your repository — propose, verify against the rules, keep what wins. What makes all of it work is verifiability: in RL it is the learning signal, and in a coding agent it’s what lets the loop converge on a solution.

The line runs through your job, not around it

The popular framing is “AI automates the coding, humans keep the high-level work.” Right shape, wrong cut. The line Verifier’s Law draws isn’t coding vs. design, and it isn’t junior vs. senior. It’s verifiable vs. unverifiable — and it runs straight through your work.

Hand a coding agent a task and its fate depends on which side of that line it lands on:

• Eaten fast — implementing a function against a clear spec and tests; fixing a well-defined bug with a reproducer; building a typed API against fixed constraints.
• Resists the bite — refactoring a legacy module for “maintainability” (no oracle for what that means); chasing a once-a-week production race condition (slow, noisy, non-reproducible); making a screen “feel right.”

Plenty of coding is not cheaply or locally verifiable, and plenty of design — latency budgets, schemas, data normalization — is perfectly verifiable. So the boundary doesn’t respect your job title. It cuts through your sprint — the jagged edge of intelligence, redrawn across your own backlog.

Why design genuinely resists

Here’s a sharper way to see why design resists, through one of the field’s most iconic ideas.Rich Sutton’s Bitter Lesson names two general methodsto leverage computation — learning and search; for ourpurposes: imitation and search.

Pre-training leaned on learning — we taught models to imitate the entire internet, and we’re now running low on fresh, high-quality human text to feed them. Reasoning models — starting with OpenAI o1 and DeepSeek-R1 — turn to search: reinforcement learning that hunts for solutions and keeps the ones a verifier approves.

Map your work onto those two axes and the asymmetry jumps out:

• Implementation often rides the search axis. Once you have tests,compilers, benchmarks, or simulations, the model can generate candidates and test them against feedback — and it isn’t capped by the data wall; give it a checker and it can practice forever in training, and retry endlessly at inference.
• Architecture, design, “what to build” more often ride the imitation axis, because the verifier is weak, slow, or disputed. AI improves at them mainly by absorbing more human examples — the exact supply that’s running dry.

The two halves of your job don’t merely differ in how verifiable they are. They sit on two different scaling curves. Coding capability compounds. Conceptual capability creeps. This asymmetry — not a slogan about “creativity” — is why their automation trajectories diverge. AI advances along the contours of feedback — and design has almost none.

The law eats its own tail

Now the twist. What survives on the unverifiable side — choosing the architecture, deciding what to build, naming the trade-offs, defining what “correct” even means — is largely authoring the spec and the verifier.
Choosing the objective, not optimizing against it.

And there is no clean verifier for choosing the right objective. Metrics can help, but they’re partial, delayed, and often gameable. If you already had a perfect one, you’d have specified the problem — which was the hard part all along. Taste, judgment, and trade-offs under ambiguity have no answer key.

So Verifier’s Law eats its own tail: the one task it structurally cannot automate is authoring the objective itself. A verifier can encode an objective — it can’t choose one. And that is exactly where your leverage concentrates.
Not below the AI, grinding out implementations, but above it, setting the objectives and checks it optimizes against. Your job moves a level up: to the specs, the acceptance criteria, the definition of done. The skill isn’t writing code. It’s making work verifiable — and knowing what to leave to human judgment, outside the verifier.

The honest version

This is easy to turn into comfort food — “the interesting work is still yours.”

Two caveats keep it honest:

1. The frontier moves. Better LLM judges, formal specs, and simulators keep converting yesterday’s unverifiable tasks into tomorrow’s verifiable ones. The claim is about the order and speed of automation — not a permanent moat.
2. It’s a skill-mix shift, not job safety. Even where conceptual work resists, the volume of pure-implementation work shrinks and the premium moves to spec-authoring, verifier design, and judgment. “Architects are safe” is the wrong takeaway. “Leverage moves from producing implementations to authoring objectives and checks” is the right one.

AI eats the verifiable first. The durable human work is authoring the objective — the one thing the law can’t automate.

This is a guest article by Kamen Petroff,a Software Developer at cycos AG (an Atos company).

Sources:

• Jason Wei, “Asymmetry of Verification and Verifier’s Law” (2025)
• Rich Sutton, “The Bitter Lesson” (2019)
• Silver et al., “Mastering the Game of Go without Human Knowledge” — AlphaGo Zero, Nature (2017)
• DeepSeek-AI, “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning” (2025)