ML Data Quality Analyst

About the role: We are open to teaching and growing an individual for this role. You’ll own the quality of the training data behind Calder’s ML and robotics work. You’ll audit annotated video, image and multimodal datasets, turn customer requirements into clear QA standards, and build the dashboards and analysis that show where quality breaks down and how to fix it., * Review and audit annotated datasets for quality, accuracy and compliance with customer specs

• Turn customer requirements into annotation guidelines, QA rubrics and automated checks
• Investigate data quality issues, find root causes and spot trends across vendors, operators and annotators
• Write SQL and Python to pull, clean and analyze annotation and operational data
• Build and maintain dashboards, trackers and reports on QA performance, throughput and delivery readiness
• Design audit and sampling processes, and give structured feedback to annotation teams
• Train and calibrate annotation teams, including remote and vendor teams, on guidelines and edge cases
• Maintain label taxonomies and metadata standards
• Test new annotation and QA tool features and give clear, prioritized feedback to product and engineering
• Run pre-delivery validation (duplicates, restricted content, metadata completeness, spec caps)
• Track key quality metrics and recommend process improvements

• 1 year in data annotation, data quality, QA or AI data operations
• Understanding of how labeled data is used to train and evaluate ML models
• Working knowledge of SQL and Python for data analysis
• Advanced Excel and Google Sheets skills
• Hands-on experience with annotation, labeling or QA platforms
• Experience writing or applying annotation guidelines and QA rubrics
• Strong attention to detail, analytical problem-solving and clear written communication
• 2-3+ years of relevant experience
• Strong Python and SQL skills (e.g., building pipelines or automated QA checks)
• Experience with BI or dashboard tools (e.g., Superset, Looker, Tableau)
• Experience using LLMs or scripting to automate QA and data workflows
• Experience with video, image or computer vision datasets, ideally egocentric or sensor data
• Exposure to robotics, spatial computing or 3D/SLAM data
• Experience managing or coordinating vendor or offshore annotation teams
• Experience in fast-paced, high-volume data operations