AI Data Scientist

Chaos Labs builds infrastructure for organizations adopting AI at scale. Our products help organizations understand their AI work, capture the knowledge it generates, and turn everyday AI activity into intelligence they own., Reporting directly to the Chief Science Officer we are looking for a talented Senior AI Data Scientist to join our team and help shape the future of AI., * Design and build machine learning systems that power enterprise search, retrieval, recommendations, and AI agents.

• Develop evaluation frameworks and metrics to measure the quality, performance, and business impact of LLM-powered applications.
• Build scalable data and ML pipelines that process and analyze large-scale AI interaction data.
• Apply statistical modeling, experimentation, and causal inference to guide product development and strategic decision-making.
• Collaborate closely with Product, Engineering, and Design to translate research and insights into production features.
• Analyze product usage and customer behavior to identify opportunities for improving AI adoption, productivity, and user experience.
• Prototype, evaluate, and deploy new AI capabilities using foundation models, embeddings, and retrieval-augmented generation (RAG) techniques.
• Define key performance indicators and build analytical frameworks that inform product strategy and company-wide decisions.
• Contribute to the technical direction of our AI platform by identifying new opportunities to leverage machine learning and generative AI.
• Mentor junior team members and help establish best practices in data science, experimentation, and machine learning across the organization.

• 5+ years of experience in Data Science, Applied Machine Learning, or a quantitative research role (or 3+ years with a PhD).
• Degree in Computer Science, Statistics, Mathematics, Machine Learning, Economics, Physics, or another highly quantitative field.
• Strong proficiency in Python and SQL, with experience building scalable data and ML pipelines.
• Deep understanding of statistics, experimentation, causal inference, and predictive modeling.
• Experience developing and deploying machine learning models in production.
• Experience working with large-scale datasets and modern data infrastructure.
• Familiarity with LLMs, embeddings, retrieval systems, RAG, or AI agents.
• Strong analytical thinking with the ability to translate ambiguous problems into measurable solutions.
• Excellent communication skills and the ability to work cross-functionally with engineering, product, and leadership.
• Comfortable operating in a fast-paced, high-ownership startup environment.

Preferred Qualifications

• Based in NYC with the ability to work from our Brooklyn office, or the ability to work remotely, with travel up to 30% of the year to collaborate in person with the team.
• Experience building or evaluating LLM-powered products or AI applications.
• Background in search, recommendation systems, information retrieval, or knowledge graphs.
• Experience designing A/B tests, offline evaluations, and ML benchmarking frameworks.
• Familiarity with vector databases, embedding models, and agent orchestration frameworks.
• Contributions to open-source ML projects or publications in machine learning, NLP, or related fields.
• Experience in B2B SaaS, developer tools, or enterprise AI products.
• Based in NYC and able to work from our Brooklyn office, or remote within the U.S. with the ability to travel up to 30% for team collaboration.

• Compensation & Equity - competitive package aligned with growth, performance, and merit
• Career Growth Opportunities - be part of a rapidly expanding, global technology company with room to grow professionally