> Markdown version of [/jobs/ext/2976693-asst-dir-data-scientist](https://www.wearedevelopers.com/jobs/ext/2976693-asst-dir-data-scientist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Asst Dir-Data Scientist - **Company:** Moody's Corporation - **Location:** King of Prussia, PA, United States - **Experience:** Experienced - **Salary:** $116,500.0 - $169,000.0 - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Microsoft Azure, Computer Programming, Python (Programming Language), Machine Learning, Natural Language Processing, Large Language Models, Model Validation, Generative AI, Information Technology, Machine Learning Operations - **Published:** September 18, 2026 - **Apply:** https://find.jobs/jobs-near-me/apply/ats-redirect/?id=2976685522-2 ## About the Role * Ph.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master's degree with 2-3 years of experience in machine learning evaluation or a related area * Strong foundations in statistical methods, experimental design, and hypothesis testing * Experience evaluating machine learning or NLP models, including designing experiments and interpreting results * Familiarity with LLM evaluation benchmarks and methodologies * Strong programming skills in Python or R * Excellent communication skills in English (both written and verbal) Preferred: * Experience evaluating LLMs or generative AI systems * Experience with production machine learning systems * Exposure to cloud platforms such as AWS, GCP, or Azure * Publications or demonstrated work in model evaluation, benchmarking, or related areas, * Ph.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master's degree with 2-3 years of experience in machine learning evaluation or a related area ## Description * Evaluate and validate large language models for production-grade analytical and decision-support systems * Design and implement evaluation frameworks for assessing LLM performance in credit analytics and decision-support contexts * Develop metrics and benchmarks to measure model robustness, reliability, consistency, and output quality * Analyze model behavior across diverse inputs, identifying failure modes, edge cases, and areas for improvement * Collaborate with model development and deployment teams to integrate validation processes into the model lifecycle * Conduct systematic assessments of model stability over time and across updates * Evaluate model outputs for bias, fairness, and economic relevance to credit risk applications * Develop and maintain documentation for evaluation methodologies, findings, and recommendations * Contribute to the advancement of best practices for LLM evaluation within the Credit COE About the Team Our Credit Center of Excellence (COE) team is responsible for maintaining and enhancing our industry-leading credit analytics and predictive modelling capabilities. We work closely with various teams including product management, commercial strategy, and go-to-market leaders to ensure the delivery of high-quality credit risk assessments and solutions. By joining our team, you will be part of exciting work in credit analytics with a global team spread across all US time zones, GMT, and GMT+1. ## Related Videos - [Introduction to Azure Machine Learning](https://www.wearedevelopers.com/videos/368-introduction-to-azure-machine-learning) - [Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative](https://www.wearedevelopers.com/videos/741-your-imaginations-is-no-longer-the-limit-how-generative-ai-empowers-people-to-be-creative) - [Developer Tools for Microsoft Azure](https://www.wearedevelopers.com/videos/450-developer-tools-for-microsoft-azure) - [Coffee with Developers - Maria Apazoglou](https://www.wearedevelopers.com/videos/1209-coffee-with-developers-maria-apazoglou) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [What non-automotive Machine Learning projects can learn from automotive Machine Learning projects](https://www.wearedevelopers.com/videos/397-what-non-automotive-machine-learning-projects-can-learn-from-automotive-machine-learning-projects) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)