> Markdown version of [/jobs/48537-principal-product-manager-agentic-evals](https://www.wearedevelopers.com/jobs/48537-principal-product-manager-agentic-evals). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Principal Product Manager, Agentic Evals - **Company:** Expedia - **Location:** San Jose, United States (Remote available) - **Experience:** Expert - **Skills:** Product Management - **Published:** September 16, 2026 - **Apply:** https://expedia.wd108.myworkdayjobs.com/en-US/search/job/Principal-Product-Manager--Agentic-Evals_R-107795?source=WeAreDevelopers ## About the Role **Minimum Requirements: ** - Bachelor's degree in a technical, quantitative, or related field, or equivalent practical experience. - 10+ years of product management experience, including significant work with AI, ML, data platforms, or evaluation systems - Strong technical depth and experience partnering with DS/ML teams - Proven ability to build frameworks, metrics, or tools that scale across large organizations - Experience with model evaluation, instrumentation, experimentation, or system reliability - Strong written and verbal communication that brings clarity to complex technical spaces - Track record of influencing cross-functional leaders and driving alignment **Preferred Qualifications** - Experience building evaluation platforms, ML tooling, or governance frameworks - Familiarity with LLM behavior, model evaluation methods, or prompt testing - Understanding of trustworthy AI principles, safety evaluation, and failure-mode analysis ## Description **Introduction to The Team:** The Product Team creates high-quality end-to-end experiences for travelers, partners, and Expedia Group. Our focus on customer-centric innovation enables us to develop products that build loyalty and repeat business. We partner closely with teams across Expedia Group to drive growth and achieve results for our customers and the company.  Expedia Group powers travel for everyone, everywhere. As AI becomes core to how travelers plan, book, and manage their trips, our ability to evaluate quality, safety, and performance becomes essential. You will build the frameworks and tools that ensure every AI experience meets a high standard of clarity, accuracy, and trust.   The Core AI Experiences team defines the foundational methods for evaluating AI across the company. We create the metrics, tooling, and processes that guide how teams measure accuracy, safety, latency, helpfulness, and long-term traveler outcomes. In this role, you will lead the strategy and build-out of Expedia Group’s AI evaluation framework. You will work closely with engineering, data science, and product teams to drive consistency and support decisions grounded in evidence.  Your work will influence every AI-powered experience across Expedia Group and set the bar for quality, safety, and trust company-wide.   **In this Role You Will:** - Define Expedia Group’s AI evaluation framework, principles, and adoption strategy - Build tools and workflows that support reliable, scalable measurement across brands - Partner with engineering and data science on metric design, instrumentation, and model evaluation - Establish clear, durable standards for accuracy, safety, transparency, and performance - Publish evaluation insights, scorecards, and recommendations that shape roadmaps - Influence strategic decisions on AI investments and risk management - Drive alignment across product, engineering, data science, and platform teams on evaluation strategy, standards, and prioritization - Create governance models that ensure consistency and responsible deployment at scale - Shape cross-team operating models and evaluation practices that guide how AI products are built and deployed across the company ## About Expedia At Expedia Group, we connect travelers, partners, and advertisers in a single marketplace, using technology to make travel more predictive, personalized, and seamless. Through our consumer brands and B2B businesses, we help millions of travelers in more than 70 countries discover, book, and explore the world — while creating new opportunities for partners to grow.  Together with our employees and partners, we’re helping travelers explore the world. One journey at a time. **Industries:** Travel, Computer Software [Company profile](https://www.wearedevelopers.com/companies/4296-expedia) ### More Jobs at Expedia - [Integration Engagement Manager](https://www.wearedevelopers.com/jobs/ext/2995115-integration-engagement-manager) - [Principal Data & AI Engineer, Reporting and Insights](https://www.wearedevelopers.com/jobs/48540-principal-data-ai-engineer-reporting-and-insights) - [Integration Engagement Manager](https://www.wearedevelopers.com/jobs/ext/2936836-integration-engagement-manager) - [Principal Software Development Engineer](https://www.wearedevelopers.com/jobs/48542-principal-software-development-engineer) - [Senior Product Manager](https://www.wearedevelopers.com/jobs/48538-senior-product-manager) ## Related Videos - [Product Managers' Eternal Battle with Refactoring](https://www.wearedevelopers.com/videos/787-product-managers-eternal-battle-with-refactoring) - [Building Products in the era of GenAI](https://www.wearedevelopers.com/videos/827-building-products-in-the-era-of-genai) - [Why pair programming is the best usability testing tool for developer focused products?](https://www.wearedevelopers.com/videos/503-why-pair-programming-is-the-best-usability-testing-tool-for-developer-focused-products) - [LLMs in the wild: Building an AI agent that survives production](https://www.wearedevelopers.com/videos/100319-llms-in-the-wild-building-an-ai-agent-that-survives-production) - [Evals vs. Evil - AI and Package Security - Laurie Voss](https://www.wearedevelopers.com/videos/2131-evals-vs-evil-ai-and-package-security-laurie-voss) - [The Art of Becoming a Mature Product Team ](https://www.wearedevelopers.com/videos/454-the-art-of-becoming-a-mature-product-team) ## Related Articles - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Trustworthy AI Starts at Deployment: 5 Checks Before You Ship](https://www.wearedevelopers.com/magazine/753-trustworthy-ai-starts-at-deployment-5-checks-before-you-ship) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Prompt Engineering is a Job of the Past](https://www.wearedevelopers.com/magazine/342-prompt-engineering-is-a-job-of-the-past) - [ I Gave a Video Editor More Autonomy Than a Trading Bot. On Purpose.](https://www.wearedevelopers.com/magazine/773-i-gave-a-video-editor-more-autonomy-than-a-trading-bot-on-purpose)