Applied Scientist II, Core Shopping Data Science

Some CX shopping defects might be straightforward to detect and track. The interesting ones are not because they depend on what a customer perceives. For example, a search page may return legitimately different results, yet a shopper has no way to tell apart. We turn these ambiguous perception questions into a measurable artifact using LLMs, and we build the frameworks to know exactly where model judgment can be trusted and where a human must decide and proving it, against ground truth, at Amazon scale., Design and improve LLM-based labeling for perception-driven defects: prompt design, sampling strategy, and the split between model judgment and human annotation.

• Validate labeling quality against human ground truth, and build and maintain the golden datasets that make that validation possible.
• Extend perceived-duplicate measurement to other parts of the shopping experience, designing the methodology where none exists and evaluating approaches already in use where one does.
• Build reusable, production-grade labeling and evaluation tooling, batch inference, quality sampling, prompt and model version control, that operates on Amazon-scale data.
• Define and publish the standards other teams adopt for using LLMs to measure customer experience.
• Partner with science, engineering, and product teams across Stores to make quality measurement usable in their decisions, and present metric results and methodology changes to stakeholders.

A day in the life We are a small team, which means your work is visible and your scope grows as fast as you do. There are existing partnerships with other teams and a paved way to cross org influence.

PhD, or Master’s degree and 4+ years of CS, CE, ML or related field experience

• Experience programming in Java, C++, Python or related language
• Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing

Preferred Qualifications

• Experience using Unix/Linux
• Experience in professional software development

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.

USA, WA, Seattle - 142,800.00 - 193,200.00 USD annually