AI ML Developer
Role details
Job location
Tech stack
Job description
- Lead end-to-end quality engineering for enterprise AI applications, including LLM-powered products, RAG pipelines, and agentic workflows.
- Design and execute prompt validation strategies, evaluating LLM responses for accuracy, semantic relevance, hallucination risk, and safety compliance.
- Build automated evaluation pipelines for AI model outputs using metrics such as BLEU, ROUGE, embedding-based similarity, precision, recall, and F1-score.
- Validate agentic systems for correctness, determinism, and failure mode handling.
- Architect and maintain Python-based automation frameworks for AI/ML model evaluation, regression testing, and continuous model quality monitoring., * Opportunity to work on cutting-edge AI and machine learning projects.
- Collaborate with innovative teams and industry experts.
- Enhance your skills in AI validation, automation, and data analysis.
- Contribute to responsible AI practices and safety standards.
- Be part of a forward-thinking organization committed to diversity and inclusion.
Upon completion of waiting period consultants are eligible for:
- Medical and Prescription Drug Plans
- Dental Plan
- Vision Plan
- Health Savings Account
- Health Flexible Spending Account
- Dependent Care Flexible Spending Account
- Supplemental Life Insurance
- Short Term and Long Term Disability Insurance
- Business Travel Insurance
- 401(k), Plus Match
- Weekly Pay
Requirements
Our client, a leading organization in the technology industry, is seeking an AI ML Developer to join their team. As an AI ML Developer, you will be part of the Data Science and AI Innovation department supporting AI and machine learning initiatives. The ideal candidate will have strong analytical skills, a proactive mindset, and excellent collaboration abilities, which will align successfully in the organization., * 10+ years of professional experience in Quality Engineering and Test Automation, validating complex enterprise applications.
- Proficient in validating AI/ML systems, including Generative AI and LLM-based applications.
- Strong proficiency in Python and experience building automation frameworks from the ground up.
- Practical experience with prompt validation, agentic workflow testing, and AI model evaluation.
- Working knowledge of evaluation metrics: BLEU, ROUGE, embedding similarity, precision, recall, F1-score, and human-evaluation methodologies.