> Markdown version of [/jobs/ext/1478119-agent-regression-test-evaluation-specialist](https://www.wearedevelopers.com/jobs/ext/1478119-agent-regression-test-evaluation-specialist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Agent Regression Test Evaluation Specialist

- **Company:** Human Union Data, Inc.
- **Location:** United States (Remote available)
- **Contract:** Permanent contract
- **Skills:** Testing (Software), Artificial Intelligence, Code Review, Regression Testing
- **Published:** July 29, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=e9204b4c40e9dd5e

## About the Role

* Direct working experience in agent regression test evaluation specialist operations on real teams for Agent Regression Test Evaluation Specialist work.
 * Comfort applying multi-page rubrics consistently across long batches.
 * Clear written reasoning that names the policy or workflow being applied.
 * Strong attention to detail and the ability to flag when a prompt itself is the problem.
 * Reliable async availability for at least 10 hours per week., * Prior experience training, calibrating, or QA-ing operations teams.
 * Familiarity with AI-assisted workflow tooling and its failure modes.
 * Bilingual experience for cross-region operations.

Skills

 * Operational review
 * Policy adherence
 * Workflow judgment
 * Stakeholder communication
 * Agent Regression Test Evaluation specialist operations
 * Code review
 * Software testing
 * Agent evaluation
 * Agent
 * Regression
 * Test

Work model

Remote - US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

## Description

Agent Regression Test Evaluation Specialist is a remote review track for evaluating AI outputs across agent regression test evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it., * Review AI outputs against current agent regression test evaluation specialist operations workflows, playbooks, and firm policy for Agent Regression Test Evaluation Specialist assignments.

 * Grade tone, escalation logic, and stakeholder fit on a structured rubric.
 * Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
 * Capture the right next step so the modeling team can train on it.
 * Adjudicate disputed treatments against published playbooks or firm guidance.
 * Maintain reviewer-quality scores in inter-rater calibration cycles., * Grade a model's response to a real agent regression test evaluation specialist operations ticket and rate workflow, tone, and escalation.
 * Flag a missed escalation with the right severity tag and corrected next step.
 * Adjudicate a disputed playbook call between two reviewers using firm guidance.
 * Audit a 25-row batch for rubric consistency and report drift to the program lead.


## Related Videos

- [Excellent Software Testing](https://www.wearedevelopers.com/videos/87-excellent-software-testing)
- [HR ROBO SAPIENS: Decoding AI Agents and Workflow Automation for Modern Recruitment](https://www.wearedevelopers.com/videos/1470-hr-robo-sapiens-decoding-ai-agents-and-workflow-automation-for-modern-recruitment)
- [Are Code Reviews Worth It? Insights from 16 Years of Review Data](https://www.wearedevelopers.com/videos/1135-are-code-reviews-worth-it-insights-from-16-years-of-review-data)
- [Trunk-Based Development at Scale: Real-World Insights from a High-Traffic Luxury E-Commerce Platform](https://www.wearedevelopers.com/videos/1435-trunk-based-development-at-scale-real-world-insights-from-a-high-traffic-luxury-e-commerce-platform)
- [Rethinking Workflows in the Agentic Era](https://www.wearedevelopers.com/videos/1540-rethinking-workflows-in-the-agentic-era)
- [Dirty Tests And How To Clean Them](https://www.wearedevelopers.com/videos/515-dirty-tests-and-how-to-clean-them)

## Related Articles

- [The Best Job Search Websites of 2025](https://www.wearedevelopers.com/magazine/368-the-best-job-search-websites-of-2025)
- [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)
- [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care)
- [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated)
- [The 8 Best Code Testing Tools](https://www.wearedevelopers.com/magazine/402-the-8-best-code-testing-tools)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)