> Markdown version of [/jobs/ext/1103188-data-scientist-generative-ai](https://www.wearedevelopers.com/jobs/ext/1103188-data-scientist-generative-ai). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Scientist - Generative AI - **Company:** Roblox - **Location:** San Mateo, CA, United States - **Experience:** Expert - **Salary:** $221,380.0 - **Contract:** Permanent contract - **Skills:** A/B Testing, Artificial Intelligence, Data Analysis, Big Data, Code Review, Apache Hive, Python (Programming Language), SQL Databases, Scripting, Large Language Models, Apache Spark, Reliability of Systems, Generative AI, Information Technology, Optimization Algorithms, Roblox, Code Restructuring, Data Generation - **Published:** June 13, 2026 - **Apply:** https://careers.roblox.com/jobs/8000879?gh_jid=8000879 ## About the Role * Advanced Degree: PhD or Master’s in Statistics, Economics, Computer Science, Applied Math, Physics, Engineering, or a related quantitative field. * Experience: 5+ years of experience in data science, analytics, or a quantitative role. * Technical Proficiency: Strong proficiency in SQL (Hive/Spark) for manipulating large datasets and scripting languages (Python or R) for analysis and modeling. * Experimentation and Causal Inference: A solid grounding in experimentation, causal inference, and statistical analysis, including test design and metric design for feature impact. * Problem Solving: A demonstrated track record of framing ambiguous problems, designing analytical approaches, and solving open-ended data science problems that drive business impact. * Learning Agility: Ability to effectively and responsibly use AI tools to enhance productivity and a passion for continuously improving methods in a fast-evolving field. * GenAI Familiarity: Familiarity with GenAI models and safety/quality evaluation methods. Expertise in the model training lifecycle is a plus (e.g., fine-tuning, RLHF, or synthetic data generation). * Engineering Development Workflow: Experience with engineering development workflows and engineering efficiency data is a plus for the Engineering Efficiency and Code Intelligence role. * Applied Research Background: A track record of applied research or publications in relevant technical fields is highly valued. ## Description As a Senior Data Scientist, you will play a critical role in a key area within our Foundation AI team: * Engineering Efficiency and Code Intelligence: Building the metrics, analytics, experimentation foundation, and AI workflow that powers how Roblox engineers and creators build and ship with AI and intelligent code systems. Whether you are focused on the end-user experience or the developer ecosystem, you will define how we measure safety, responsibility, quality, and efficiency. You will combine annotation analysis, design of experiments, causal inference, model-based evaluation methods (such as LLM-as-a-judge), optimization algorithm, and AI models to drive product decisions and model improvements. You Will: * Develop Evaluation Frameworks: Design and operationalize rigorous evaluation systems for either GenAI features (text, image, video, 3D, 4D) or internal AI Agents (Code Review, Refactor, Test Gen). This includes eval experiment design, dataset design, label reliability analysis, and implementing and finetuning LLM-as-judge methods. * Run Rigorous Experiments: Conduct online experiments (A/B tests) and causal inference to quantify the impact of GenAI features or AI-assisted coding tools. You will identify opportunities, measure lift, and ensure statistical rigor. * Define Success Metrics: Partner with cross-functional teams to define leading/lagging indicators—whether for GenAI safety and user satisfaction, or for engineering productivity and code health. * Build Automated Systems: Research and apply state-of-the-art methodologies to build reproducible evaluation tooling and agentic workflows that lift rigor and efficiency across the company. * Drive Strategy & Visibility: Develop dashboards and reporting frameworks that reveal trends (e.g., model performance or developer friction) and translate complex data into clear, prioritized recommendations for leadership. ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [JavaScript? No. Java Scripts! - Scripting with Java](https://www.wearedevelopers.com/videos/2094-javascript-no-java-scripts-scripting-with-java) - [Every CEO needs a digital twin to understand the scope of generative AI](https://www.wearedevelopers.com/videos/1006-every-ceo-needs-a-digital-twin-to-understand-the-scope-of-generative-ai) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Intermediate Bitcoin Script](https://www.wearedevelopers.com/videos/25-intermediate-bitcoin-script) - [Building Products in the era of GenAI](https://www.wearedevelopers.com/videos/827-building-products-in-the-era-of-genai) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts) - [The State of WebDev AI 2025 Results: What Can We Learn?](https://www.wearedevelopers.com/magazine/581-the-state-of-webdev-ai-2025-results-what-can-we-learn) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)