University SWE Expert

We are hiring software engineers to author original coding tasks based on real codebases, used to evaluate and train advanced AI coding agents. You’ll connect one of your own private codebases and turn a real engineering problem from it into a task: a clear, maintainer-style issue, a hidden test suite, and a reference solution. Every submission is validated automatically before it’s approved, so the ideal candidate is someone who can write a problem a strong engineer would recognize as real work, and a test suite that verifies a solution the way a careful reviewer would.

Responsibilities

• Connect a substantial private codebase of your own to author tasks against.
• Author an original coding task from that codebase: a clear, maintainer-style issue describing the problem to solve.
• Build a hidden test suite for each task, including both fail-to-pass tests (fail before the fix, pass after) and pass-to-pass tests (verify nothing else breaks).
• Write a reference solution that resolves the issue and passes every test in the hidden suite.
• Iterate on submissions as needed - you’ll have up to 5 attempts to get a task through automated validation, so take your time rather than rushing it.

• Proficiency in at least one modern programming language (e.g., JavaScript/TypeScript, Java, Go, Rust, C/C++, Python, etc.).
• Access to a substantial, complex, private codebase you own and can author tasks against.
• Strong technical writing, documentation, and testing skills.
• Ability to reason about how a bug or feature ripples across a multi-file codebase., * University students with software engineering internship experience.
• Students majoring in Computer Science, Data Science, or related fields.
• Open source contributors with a track record of real commits/PRs.
• Background in code review or test authoring.

Pay

• $300 per codebase you connect and use as a basis for tasks.
• $75 per task that is fully approved.