AI Data Quality Reviewer

Blueprint School Network, Inc.
United States
12 days ago
Apply on talenthackers.applytojob.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Starter
Experience required
1 year minimum
Working hours
Regular working hours

Tech stack

Data Integrity Fraud Prevention and Detection

Job description

  • Review contributor submissions across text and video against project-specific guidelines and acceptance criteria
  • Approve, reject, or return submissions for correction while providing clear and feedback
  • Maintain high accuracy and consistency while meeting project throughput and turnaround-time targets
  • Identify duplicate, low-effort, synthetic, manipulated, or otherwise suspicious submissions and escalate potential integrity issues
  • Apply detailed review rubrics consistently across large volumes of submissions
  • Track recurring contributor errors and flag unclear or ineffective task instructions
  • Accurately record review decisions, rejection reasons, and other required information

Requirements

We are seeking a detail-oriented AI Data Quality Reviewer to help ensure the quality, accuracy, and integrity of data used in AI training and evaluation projects. In this role, you will review contributor submissions across text and video formats, assess them against project-specific guidelines, and make consistent quality decisions at scale.

The ideal candidate has experience working in high-volume review environments, possesses exceptional attention to detail, and can apply detailed guidelines consistently while maintaining productivity targets. You will play a critical role in maintaining dataset quality, identifying suspicious or low-quality submissions, and providing clear feedback to contributors., * 1-3+ years of experience in data annotation, quality assurance, content moderation, trust & safety, document review, or another high-volume review environment

  • Experience reviewing the work or submissions of other people against defined guidelines, policies, rubrics, or SOPs
  • Exceptional attention to detail and ability to identify subtle errors, inconsistencies, and quality issues
  • Strong judgment and ability to make consistent decisions when reviewing large volumes of submissions
  • Ability to quickly learn and accurately apply new project-specific guidelines and acceptance criteria
  • Strong (C-level) written and verbal English communication skills
  • Ability to provide concise, clear, and constructive reviewer feedback
  • Comfortable working toward measurable accuracy, productivity, and turnaround targets
  • Strong organizational skills and the ability to accurately document review decisions
  • Ability to work full-time EST hours

Nice to Have:

  • Previous experience reviewing AI training data, data annotations, RLHF tasks, or human-generated datasets
  • Background in content moderation, trust & safety, fraud detection, data integrity, or marketplace quality control

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on talenthackers.applytojob.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

1:45 min

Ensuring training dataset quality through required team diversity

Cassie Kozyrkov · World Congress 2022

1:18 min

Detecting answering machines and preventing automated voice fraud

Marius Obert Marius Obert · Coffee With Developers

1:55 min

Maintaining software and data integrity during execution

Christian Wenz Christian Wenz · World Congress 2026 Europe

5:39 min

Two-step fraud detection with rules and machine learning

Doraly Chezeu Sukem Doraly Chezeu Sukem +1 · World Congress 2024

4:47 min

Navigating data integrity and unpredictable generative AI outputs

Maish Saidel-Keesing Maish Saidel-Keesing · World Congress 2025

1:50 min

Balancing automated candidate screening with essential human oversight

Rudi Bauer Rudi Bauer +1 · Cappuccino with HR

Videos

See all

Related articles

See all