> Markdown version of [/jobs/ext/2711603-data-scientist-agent-evaluations-quality](https://www.wearedevelopers.com/jobs/ext/2711603-data-scientist-agent-evaluations-quality). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Scientist - Agent Evaluations & Quality

- **Company:** HEY NOW, INC.
- **Location:** Atherton, CA, United States
- **Contract:** Permanent contract
- **Skills:** A/B Testing, Artificial Intelligence, Big Data, Data Visualization, Python (Programming Language), Machine Learning, NumPy, Chatbots, Pandas, Scikit Learn, Information Technology, Data Analytics, Virtual Agents
- **Published:** September 4, 2026
- **Apply:** https://www.disabledperson.com/jobs/74810578-data-scientist-agent-evaluations-quality

## About the Role

* Strong foundation in Data Science and Statistics, with the ability to design experiments and build predictive or evaluative models.
 * Applied experience in data analytics and analysis to interpret complex interaction logs and derive actionable insights.
 * Proficiency in data visualization to communicate findings clearly to technical and non-technical stakeholders.
 * Hands-on experience with Python or R and common data science libraries (e.g., pandas, NumPy, scikit-learn, or similar).
 * Experience working with large datasets, preferably involving conversational AI, NLP, or product usage data.
 * Ability to define metrics, design evaluation pipelines, and run A/B tests or other experimental frameworks.
 * Bachelor's or Master's degree in a quantitative field such as Computer Science, Statistics, Mathematics, Data Science, or a related discipline, or equivalent practical experience.
 * Strong communication skills, with the ability to present complex analyses in a clear, structured way to cross-functional teams.
 * Comfort working in a fast-paced, early-stage environment and collaborating closely with product and engineering on-site in Palo Alto.

## Description

The Data Scientist, Agent Evaluations & Quality role focuses on measuring and improving the performance, reliability, and quality of Noah's AI assistant across SMS and voice interactions. Day-to-day, you'll design evaluation frameworks, develop metrics and KPIs for agent behavior, and analyze large datasets of user-agent conversations to identify patterns and quality issues. You'll build statistical and machine learning models to assess agent responses, run A/B tests, and deliver data-driven recommendations to product and engineering teams.

This is a highly collaborative role. You'll work closely with AI researchers, engineers, and product managers to refine models, improve the user experience, and ensure that executives receive accurate, helpful, and timely assistance.


## Related Videos

- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Beyond the Benchmark: How to Evaluate AI Agents in the Real World](https://www.wearedevelopers.com/videos/100269-beyond-the-benchmark-how-to-evaluate-ai-agents-in-the-real-world)
- [Advanced Typing in TypeScript](https://www.wearedevelopers.com/videos/496-advanced-typing-in-typescript)
- [Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.](https://www.wearedevelopers.com/videos/562-vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems)

## Related Articles

- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)
- [Trustworthy AI Starts at Deployment: 5 Checks Before You Ship](https://www.wearedevelopers.com/magazine/753-trustworthy-ai-starts-at-deployment-5-checks-before-you-ship)
- [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care)