Research Engineer, QC Automation in San Francisco

Our client is seeking a Research Engineer, QC Automation to automate quality control for training data created by companies using its infrastructure. This role will build systems that scale quality as the company meets continued strong demand, combining technical execution with strong judgment about data quality and genuine curiosity about unfamiliar domains., * Create QC systems grounded in true understanding and human judgment without relying heavily on LLMs

• Define and enforce quality standards for training data
• Design experiments and metrics to grade agent outputs
• Partner with data vendors to diagnose agent failure modes, debug quality issues, and improve data- processes
• Build systems for auditing supplier datasets, including sampling strategies, rule-based and model-assisted validation pipelines, and feedback loops
• Integrate QC learnings into infrastructure tools and the data-vendor portal to reduce anomalies, inconsistencies, and edge cases, * Build quality-control systems for reinforcement-learning training data and evaluations for frontier AI agents
• Help scale data quality as the company meets continued strong demand
• Work on infrastructure used by frontier labs, Fortune 500 companies, and startups
• Relocation and visa support are available for strong candidates

Required:

• Proficiency in Python, Docker, and Linux environments
• Strong judgment about what good data means and how to measure it
• Genuine curiosity about unfamiliar domains and skill at asking the questions needed to understand them
• Experience building scalable data-validation pipelines or automated QA/QC systems without a prescribed roadmap
• Experience with benchmarks and evaluations
• Early-stage startup experience and independent execution

Technical aptitude and learning potential matter more than years of experience., * Knowledge of statistics

• Strong written and verbal communication
• Comfort designing metrics, experiments, and QA/QC processes
• Ability to construct tasks in new evaluations
• Comfort in unstructured problem spaces

Our client is a Y Combinator-backed company building infrastructure to create reinforcement-learning training data and evaluations for frontier AI agents, along with a marketplace connecting that work to frontier labs. Its platform is used by frontier labs, Fortune 500 companies, and startups.