UI/UX Designer for AI Evaluation

Define what excellent product design looks like by creating task-specific grading rubrics and evaluating completed design work. You will not produce final designs, instead you will set precise quality standards and score AI-generated and human work samples with rigorous, well-reasoned written justifications that can be reproduced and defended. Key Responsibilities

• Design precise, task-specific grading criteria for real-world digital product and UX deliverables, including design systems and component libraries, responsive UI layouts and dashboards, end-to-end product flows such as onboarding, checkout, and booking, logged-in customer portals, UX writing and product content, and design-to-engineering or production handoff.
• Score AI-generated and human work samples against those criteria, providing detailed written justifications for every score.
• Apply consistent, evidence-based judgment so that scores are reproducible and defensible.
• Incorporate structured feedback from senior reviewers and iterate quickly on grading rubrics and scoring outputs.

• 5+ years of professional experience in product design, UI/UX, or design systems for real digital products, not brand or marketing sites.
• Experience at leading digital-product and UX studios, in-house product-design teams at major product-led companies, or e-commerce and marketplace companies with sophisticated customer portals and dashboards.
• Relevant background across one or more tracks, for example Product / UI / Design Systems Designer, UX Lead, UX Writer / Content Designer, or digital Product / Business Analyst.
• Deep fluency in hands-on craft: interaction and visual design, design systems, responsive layouts, end-to-end product flows, prototyping and production handoff in Figma, and delivering critique the way a design or UX lead would in a review.
• Exceptionally strong written communication, able to explain exactly why a piece of work does or does not meet the bar for a real product and its users.
• Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
• Prior experience with AI training, evaluation, or human-data projects is a strong plus.

• Washington DC
• $8,500 per month A World-Changing Company Palantir builds the world’s leading software for data-driven decisions and operations. By bringing the right data to the people who need it, our platform…