Data Scientist - Agent Evaluations & Quality

The Data Scientist, Agent Evaluations & Quality role focuses on measuring and improving the performance, reliability, and quality of Noah’s AI assistant across SMS and voice interactions. Day-to-day, you’ll design evaluation frameworks, develop metrics and KPIs for agent behavior, and analyze large datasets of user-agent conversations to identify patterns and quality issues. You’ll build statistical and machine learning models to assess agent responses, run A/B tests, and deliver data-driven recommendations to product and engineering teams.

This is a highly collaborative role. You’ll work closely with AI researchers, engineers, and product managers to refine models, improve the user experience, and ensure that executives receive accurate, helpful, and timely assistance.

• Strong foundation in Data Science and Statistics, with the ability to design experiments and build predictive or evaluative models.
• Applied experience in data analytics and analysis to interpret complex interaction logs and derive actionable insights.
• Proficiency in data visualization to communicate findings clearly to technical and non-technical stakeholders.
• Hands-on experience with Python or R and common data science libraries (e.g., pandas, NumPy, scikit-learn, or similar).
• Experience working with large datasets, preferably involving conversational AI, NLP, or product usage data.
• Ability to define metrics, design evaluation pipelines, and run A/B tests or other experimental frameworks.
• Bachelor’s or Master’s degree in a quantitative field such as Computer Science, Statistics, Mathematics, Data Science, or a related discipline, or equivalent practical experience.
• Strong communication skills, with the ability to present complex analyses in a clear, structured way to cross-functional teams.
• Comfort working in a fast-paced, early-stage environment and collaborating closely with product and engineering on-site in Palo Alto.

Most companies chasing the “AI assistant” built a copilot: a chatbot that sits there and waits for you to type. Noah is something else entirely. Noah, branded as Hey Noah, is the world’s first fully autonomous SMS and voice-based assistant built for executives, and it doesn’t just chat with you, it acts on your behalf and communicates directly with the people who matter to you. It reaches out to your clients, coordinates with your team, and keeps your most important relationships warm, all over natural text and voice. No new app to learn, no dashboard to babysit. Noah does the work and hands back your time for the decisions that matter. Plenty have tried to build this. Nobody got it right, until now.

The market noticed immediately. Noah swept Product Hunt with #1 Product of the Day, Week, and Month, a clean sweep few launches ever pull off.

We’re a small, senior team of 8 in Palo Alto, backed by Hustle Fund and Neon. Noah was founded by Ashish Toshniwal, who previously bootstrapped YML into a $100M company, and engineering is led by our CTO, Ryan Brandt. We’ve built and scaled products before, and we’re looking for people who want to build the next one with us.