Experienced Software Engineer - AI Coding Tasks

We’re looking for experienced software engineers to build AI coding tasks using a codebase they know well. We pay US$1,000 per task, and you can complete as many tasks as you want.

You’ll turn a real change from a codebase you know well into a task that today’s best AI coding models struggle with. The task should be challenging for the models but fair for a skilled engineer.

Your tasks will be used to evaluate and train AI coding models.

To get started, complete a short screener interview and upload your resume. If selected, you’ll choose a change from a repository you know and submit a short proposal for review.

Once your proposal is approved, you’ll build the task on our platform by setting up the repository in its original state, writing the task brief and correct solution, and creating grading checks such as tests, build commands and pass/fail criteria.

You’ll then stress-test your task with incorrect and shortcut solutions to ensure your checks catch them, while confirming that alternative correct solutions still pass. Next, you’ll run the task against leading AI models on our platform to see whether strong models struggle with it.

A reviewer will assess your task and provide feedback. You’ll revise it as needed, with most tasks requiring more than one round of review. Once your task is accepted, you can move on to the next one.

• At least 5 years of professional software engineering experience.
• Code merged into a public open-source repository, ideally one you know well. Code from a private codebase you own may also qualify, but employer or client code does not.
• Strong skills in at least one of Python, TypeScript, Go, Rust, Java or C++.
• Solid experience with Docker, Linux, Git and automated testing.
• Regular use of an AI coding tool.
• Experience writing tests, rubrics or evaluation criteria, ideally for AI models, and an interest in finding gaps in specifications.
• The ability to explain technical changes clearly and concisely in your own words.
• Availability for at least 10 hours a week, starting now.
• A secure personal or work computer. Public or shared machines are not permitted.

It would also be helpful to have deep expertise in an area such as databases, compilers, distributed systems, security or infrastructure, or experience building agent tasks or reinforcement learning (RL) environments.

The role is fully remote and open to applicants from any location. Once your proposal qualifies, you’ll sign Askable’s contract and accept its terms.

You’ll receive US$1,000 for each task that passes our quality review, paid directly by Askable rather than through an agency. You can complete as many tasks as you want, with a minimum expectation of one accepted task per week and at least 10 hours of work each week.