> Markdown version of [/jobs/ext/1319819-data-annotation-reviewer-qa](https://www.wearedevelopers.com/jobs/ext/1319819-data-annotation-reviewer-qa). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Annotation reviewer (QA) - **Company:** Pro Integrate - **Location:** Louisville, KY, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Microsoft Excel, Artificial Intelligence, Microsoft Outlook, Data as a Services, Design of User Interfaces, Microsoft Office, Microsoft PowerPoint, Large Language Models, Generative AI - **Published:** July 17, 2026 - **Apply:** https://www.dice.com/job-detail/98b3d90f-79b8-40ec-a848-40e36c72c89f ## About the Role Language Proficiency: * Native level fluency in English. * Strong written and verbal communication skills in English., * Located in Lousiville, KY and ability to work at client- on-site. * Required: Diploma and at least 2-3 years of experience in quality management, particularly in data services for AI. * Desired: Bachelor's degree, Linguistics or other related field of study * Experience writing data-labeling project guidelines and materials (or similar work in another field). * Ability to create useful and impactful insights for annotators and clients. * Basic familiarity with Microsoft Office 365 including Outlook, Excel, and PowerPoint. * General knowledge of online communication * Excellent communication both written and spoken. * Collaborative and solution focused. * Confidence and the ability to give/take feedback. * Ability to follow project directions and perform time bound tasks accurately and efficiently. * Ability to maintain quality while performing repetitive tasks. * Detail-oriented and problem-solving mindset. * Organized and focused enough to work independently as a role player within a team environment. Availability: * Minimum of 30 hours per week. * Willingness to participate in training sessions and demos. Certifications: * Successful completion of required certifications is mandatory. ## Description As QA at Centific specializing in Generative AI Data Services, you will play a pivotal role in driving the success of our cutting-edge projects focused on building and enhancing Large Language Models (LLMs). Your primary responsibility will be to adhere to quality review instructions, pass, review or reject human-labeled data crucial for training AI models. In this dynamic and evolving field, you will be at the forefront of bridging the gap between human expertise and artificial intelligence, contributing significantly to the development of next-generation AI technologies. As a problem solver and highly organized lead, you will collaborate closely with internal teams and a global workforce, ensuring seamless communication, efficient execution, and successful delivery of projects., As a Data Annotation Reviewer, you will evaluate data annotation and bounding box responses from annotators. Your job is to ensure the correct annotation was applied within the input parameters match and the output is accurate and relevant according to the guidelines., * Review annotations to confirm they match the intended purpose of the guidelines. * Validate that the annotator answer is appropriate for the chosen function. * Check if the Annotation output includes all necessary information to filfull the guidelines. * Identify and assess individual data points in the AI response for accuracy and relevance. * Provide comments and feedback to clarify decisions or flag inconsistencies., * Assisting in the development and implementation of quality assurance policies. * Collaboration on project guidelines and supplemental learning materials. * Ensure compliance with client-specific requirements and performance standards in quality aspects of project execution. * Provide feedback and recommendations to PM team regarding UI design. * Review annotators and arbitrators weekly via the production monitoring process. * Review metrics and spot check individual tasks. * Record and flag quality issues. * Conduct end-of-day reporting to ensure smooth project handoffs and continuity across shifts. ## Related Videos - [Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More](https://www.wearedevelopers.com/videos/869-inside-the-ai-revolution-how-microsoft-is-empowering-the-world-to-achieve-more) - [In-depth .NET Azure Functions: Isolated mode, performance and durable AI agents](https://www.wearedevelopers.com/videos/100207-in-depth-net-azure-functions-isolated-mode-performance-and-durable-ai-agents) - [Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative](https://www.wearedevelopers.com/videos/741-your-imaginations-is-no-longer-the-limit-how-generative-ai-empowers-people-to-be-creative) - [From Syntax to Singularity: AI’s Impact on Developer Roles](https://www.wearedevelopers.com/videos/900-from-syntax-to-singularity-ai-s-impact-on-developer-roles) - [Coffee with Developers - Maria Apazoglou](https://www.wearedevelopers.com/videos/1209-coffee-with-developers-maria-apazoglou) - [The shadows that follow the AI generative models](https://www.wearedevelopers.com/videos/624-the-shadows-that-follow-the-ai-generative-models) ## Related Articles - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [13 AI Tools You Have to Try](https://www.wearedevelopers.com/magazine/219-13-ai-tools-you-have-to-try) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)