> Markdown version of [/jobs/ext/4029568-machine-learning-engineer-evaluation-and-quality-creativity-apps](https://www.wearedevelopers.com/jobs/ext/4029568-machine-learning-engineer-evaluation-and-quality-creativity-apps). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Machine Learning Engineer, Evaluation and Quality, Creativity Apps - **Company:** Apple Inc. - **Location:** San Diego, CA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Apple IOS, Apple Mac Systems, Python (Programming Language), Machine Learning, Software Engineering, Software Quality Assurance (SQA), Strategies of Testing, Large Language Models, Information Technology - **Published:** October 30, 2026 - **Apply:** https://www.dice.com/job-detail/5adc926c-1428-494d-a741-c2fc50bec823 ## About the Role 5+ years of industry experience in machine learning or software engineering, with a focus on ML/AI model evaluation and quality. Experience with evaluation methodology for ML models, LLMs, and AI products or features. Strong understanding of the ML development cycle, from data and training to evaluation, deployment, and monitoring. Solid machine learning fundamentals and LLM knowledge, including agentic workflows, and you keep up with the latest developments in the field. Track record of applying LLMs, or agentic workflows to ML evaluation and quality. Experience in data curation, dataset creation, and dataset quality, including iterative improvement workflows that deliver high-quality datasets tailored to specific products. Experience designing, implementing, maintaining, or optimizing ML evaluation infrastructure at scale. Experience designing evaluation metrics based on the model, the product, and its use cases. Experience driving end-to-end evaluation and quality for AI products. Fluency in quality reporting, presenting quality data to both technical and non-technical audiences. Strong communication skills and experience working with and driving cross-functional partners. Proficient in Python and/or Swift or comparable languages. Bachelor's degree in Computer Science, Data Science, Machine Learning, or a related field, or equivalent experience. Preferred Qualifications Knowledge of the software development life cycle and software quality assurance methodologies. Experience in objective image and video quality evaluation and metrics. Knowledge of color science and/or experience with subjective image or video quality review. Passion for creative applications; experience as a creator (photo, video, music, or design) is a plus. Excitement about learning new technology and applying it to practical problems. Familiarity with iOS and macOS. ## Description In this role, you will own the evaluation and quality of AI features across Creativity Apps. You will work closely with Machine Learning engineering, infrastructure, product, and design teams to understand each model and use case, and translate them into evaluation methodology, testing strategy, and quality metrics that reflect what creators actually experience. You will: * Define evaluation strategy and metrics for ML models, LLMs, and AI features, tailored to the model, the product, and its use cases. * Curate and create evaluation datasets, measure and improve dataset quality, and run iterative improvement loops so that datasets stay representative of each product. * Design, build, maintain, and optimize evaluation infrastructure at scale, with reproducible runs, tracked baselines, and comparisons between model versions. * Apply ML, LLMs, and agentic workflows to the evaluation itself, including automated judging, test generation, and analysis, to make evaluation more efficient, more accurate, and more scalable. * Drive end-to-end evaluation and quality of AI features, from model checkpoints to integrated, shipping features, including triaging issues and driving them to resolution. * Report quality clearly, turning evaluation results into ship/no-ship risk and recommendations for both technical and non-technical audiences. * Partner with and drive cross-functional teams, comfortable with ambiguity and making educated, critical judgments.