> Markdown version of [/jobs/ext/1222743-head-of-data-labeling](https://www.wearedevelopers.com/jobs/ext/1222743-head-of-data-labeling). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Head of Data Labeling

- **Company:** White Circle
- **Location:** Paris, France
- **Experience:** Expert
- **Salary:** €60,000.0 - €100,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Data Analysis, Cognitive Science, Python (Programming Language), Machine Learning, DataOps, Software Safety, SQL Databases, Large Language Models, Prompt Engineering, Machine Learning Operations
- **Published:** July 10, 2026
- **Apply:** https://fr.indeed.com/viewjob?jk=1f08f40f8ed837d1

## About the Role

* Has experience leading data annotation or AI evaluation teams
 * Has strong operational and people management skills
 * Understands AI model evaluation, LLM behavior, and modern annotation workflows
 * Can design scalable processes without sacrificing quality
 * Communicates clearly across technical and non-technical teams
 * Thrives in fast-moving startup environments

You might be a great fit if you

 * Have managed annotation programs for LLMs, generative AI, or machine learning
 * Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation
 * Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops
 * Have managed distributed or global annotation teams
 * Have experience with vendor management and outsourcing operations

Bonus points

 * Familiarity with prompt engineering and AI safety policies
 * SQL, Python, or data analysis experience
 * Experience building internal annotation platforms or workflow automation
 * Background in linguistics, cognitive science, machine learning, or data operations

## Description

* Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)
 * Define annotation guidelines, quality standards, and evaluation frameworks
 * Develop quality assurance processes, calibration sessions, and auditing systems
 * Partner with AI researchers and engineers to translate research objectives into labeling workflows
 * Prioritise labeling projects based on business and research needs
 * Monitor operational metrics including quality, consistency, throughput, and cost
 * Improve annotation tooling, automation, and workflow efficiency
 * Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation
 * Analyse disagreement patterns and edge cases to improve guidelines and model performance
 * Manage vendor relationships and ensure consistent quality across distributed teams
 * Build reporting dashboards and communicate operational insights to leadership
 * Foster a culture of continuous improvement, accountability, and operational excellence, This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content.
You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently.


## Related Videos

- [Coffee with Developers - Maria Apazoglou](https://www.wearedevelopers.com/videos/1209-coffee-with-developers-maria-apazoglou)
- [Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases](https://www.wearedevelopers.com/videos/1146-fault-tolerance-and-consistency-at-scale-harnessing-the-power-of-distributed-sql-databases)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Rethinking Recruiting: What you didn’t know about Responsible AI](https://www.wearedevelopers.com/videos/1090-rethinking-recruiting-what-you-didn-t-know-about-responsible-ai)
- [Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases](https://www.wearedevelopers.com/videos/1520-fault-tolerance-and-consistency-at-scale-harnessing-the-power-of-distributed-sql-databases)
- [Data Science on Software Data](https://www.wearedevelopers.com/videos/162-data-science-on-software-data)

## Related Articles

- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)