software engineers
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
Job description
We’re hiring software engineers for an ongoing remote project focused on building robust evaluations for AI coding agents. Your expertise ensures these models can handle realistic, complex problems in a terminal environment. This work directly feeds into improving the accuracy and reliability of automated engineering tools. How It Works
You will design realistic, self-contained software-engineering problems from scratch. The work involves implementing features, debugging software, configuring Docker environments, and writing automated tests. You will also create reference solutions and verify that each task can be graded reliably by our systems. All work happens remotely via your own machine using standard command-line tools and GitHub. Who This Is For, This opportunity is best suited for experienced software engineers who are comfortable navigating unfamiliar codebases. We welcome backend developers, full-stack engineers, DevOps specialists, and systems engineers. Candidates must be highly proficient with command-line tools and writing clean, deterministic tests. What You’ll Do
- Design self-contained software-engineering problems for AI agents to solve.
- Build and configure development environments using Docker.
- Write automated, deterministic tests to reliably grade AI performance.
- Create accurate reference solutions for complex coding challenges.
Requirements
- Professional experience working as a software engineer.
- Comfortable navigating unfamiliar codebases independently.
- Strong proficiency with GitHub and command-line tools.
- Willingness to complete a one-time skills assessment before starting.
Benefits & conditions
Posted 11 Hours Ago Be an Early Applicant Remote Hiring Remotely in United States 175-175 Hourly Mid level Remote Hiring Remotely in United States 175-175 Hourly Mid level Design and implement self-contained software-engineering problems for AI coding agents. Build and configure Docker development environments, write deterministic automated tests, create reference solutions, and verify reliable automated grading. Work remotely using command-line tools and GitHub; participate in a one-time paid skills assessment. The summary above was generated by AI What We’re Researching, $175 per task
Ready to participate?
Start your paid interview now
About Terac
Terac is building the world’s largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com or on YouTube at @jointerac., Remote or Hybrid United States 117K-158K Annually Senior level 117K-158K Annually Senior level Cloud * Fintech * Software * Business Intelligence * Consulting * Financial Services Lead transaction advisory engagements by performing financial analysis and modeling, evaluating deal structures and valuations, managing multiple projects, liaising with clients and cross-functional teams, preparing clear reports and presentations, and reviewing junior team members’ work to ensure accuracy and high-quality deliverables. Top Skills: ExcelFinancial Databases Wipfli
Senior, Transaction Advisory Services
An Hour Ago Remote or Hybrid United States 88K-118K Annually Senior level 88K-118K Annually Senior level Cloud * Fintech * Software * Business Intelligence * Consulting * Financial Services Lead buy- and sell-side Quality of Earnings engagements: analyze financials, assess earnings quality, working capital, and net debt; prepare diligence reports and presentations; manage VDRs; mentor junior staff; support valuation, deal structuring, integration planning, and business development. Top Skills: ExcelFinancial DatabasesVirtual Data Rooms, ServiceNow, 5 Hours Ago Remote or Hybrid 131K-216K Annually Senior level 131K-216K Annually Senior level Artificial Intelligence * Cloud * HR Tech * Information Technology * Productivity * Software * Automation Drive new SaaS license revenue by prospecting and closing enterprise education customers. Build C-suite relationships, lead account and territory planning, orchestrate cross-functional teams, act as trusted advisor on IT and security roadmaps, and bring specialists into deals. Travel up to 50%. Top Skills: AIArmisSaaSServicenowVeza
What you need to know about the Colorado Tech Scene
With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.
Key Facts About Colorado Tech
- Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
- Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
- Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
- Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
- Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
Dev Digest 121 - AI goes offline
Fully Remote Software Engineer Jobs
Is Software Engineering Over-Saturated?
Find a Developer Job: 12 Best Job Sites For Developers