Researcher, Evals
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
Job description
The New Horizons Evaluations team is reimagining how we measure progress in interactive machine intelligence. As Evaluations Lead, you will design evaluation frameworks that capture not just what models know - but how they reason, remember, and interact over time. Youāll work at the intersection of research, product, and infrastructure to develop metrics, systems, and studies that define what āintelligenceā means in the next generation of AI. This role is ideal for someone who combines scientific rigor with technical execution, and whoās deeply curious about how people use - and want to use - intelligent systems. Your work will shape how Cartesia builds and evaluates frontier models, ensuring that progress isnāt measured solely by static benchmarks, but by deeper qualities like understanding, naturalness, and adaptability in real-world interaction., * Identify and define key model capabilities and behaviors that matter for next-generation model evals
- Develop and implement new evaluation pipelines with robust statistical analysis and clear reporting
- Partner closely with model training and research teams to embed evaluation systems directly into model development loops
- Prototype new user studies and behavioral experiments to ground evaluations in real-world use.
Requirements
Research Skills, Artificial Intelligence (AI), Product Engineering, Systems Engineering, Product Design, Benchmarking, Metrics, Design Evaluation, Statistics, Prototyping, * Experience designing or implementing evaluation frameworks for generative models (audio, text, or multimodal)
- Strong technical and analytical skills, ability to take open-ended research ideas and translate them into production-ready systems
- Creativity in defining novel quantitative metrics for subjective or behavioral qualities
- Excitement for building evaluation systems that bridge research and real-world use
- Curiosity and rigor in equal measure; motivation driven by discovering how to measure meaningful progress in intelligent behavior
Benefits & conditions
Compensation Competitive base salary alongside attractive equity package.
Health Insurance Fully covered medical insurance along with dental and vision for you and your family.
Parental Leave 9 weeks paternity & 12 weeks maternity leave
401(k)
Commuter Allowance A monthly stipend to help you get to and from the office.
About the company
About Cartesia
Our mission is to architect AI that learns from and interacts with the world like humans do.
Weāre pioneering the model architectures that will make this possible. Our founding team met as PhDs at the Stanford AI Lab, where we invented State Space Models or SSMs, a new primitive for training efficient, large-scale foundation models. Our team combines deep expertise in model innovation and systems engineering paired with a design-minded product engineering team to build and ship cutting edge models and experiences.
Weāre funded by leading investors at Index Ventures and Lightspeed Venture Partners, along with Factory, Conviction, A Star, General Catalyst, SV Angel, Databricks and others. Weāre fortunate to have the support of many amazing advisors, and 90+ angels across many industries, including the worldās foremost experts in AI.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleā¦