> Markdown version of [/jobs/ext/4774013-machine-learning-engineer-llm-evaluator-hybrid-opportunity](https://www.wearedevelopers.com/jobs/ext/4774013-machine-learning-engineer-llm-evaluator-hybrid-opportunity). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Machine Learning Engineer - LLM Evaluator - Hybrid Opportunity - **Company:** TEKSYSTEMS INC. - **Location:** Cupertino, CA, United States (Remote available) - **Salary:** $114,400 - $135,200 - **Contract:** Temporary contract - **Skills:** Artificial Intelligence, Big Data, Decision Support Systems, Python (Programming Language), Machine Learning, SQL Databases, Large Language Models, Prompt Engineering, Build Management, Evaluation of Large Language Models - **Published:** November 18, 2026 - **Apply:** https://dejobs.org/x/x/0FC8A27B5DE1424F95F71B94E667C5FF/job/ ## About the Role * Experience designing and maintaining LLM evaluation frameworks and measurement methodologies. * Hands-on experience with autograders, LLM-as-a-Judge systems, and automated quality validation. * Strong prompt engineering skills, including the development of evaluation prompts and grading rubrics. * Proficiency with Python, SQL, and large-scale data analysis. * Experience creating KPIs, dashboards, and executive-ready reporting on AI quality. * Ability to evaluate model performance through error analysis, human-vs-model calibration, and quality benchmarking. * Experience working with production AI/LLM systems and large evaluation datasets. What Will Make You Stand Out * You've personally owned or led an AI/LLM evaluation strategy. * You've built scalable systems to measure AI quality automatically. * You can clearly explain how AI quality is defined, measured, and improved. * You have experience influencing product and engineering decisions through evaluation insights and data. Experience Level: Expert Job Type & Location ## Description * Hybrid role requiring onsite attendance in Cupertino, CA three (3) days per week. * Contract opportunity. * This role focuses on AI/LLM quality evaluation and measurement, not traditional model development. * Candidates should have experience owning or leading evaluation strategies for production AI/LLM systems. We are seeking an experienced Machine Learning Engineer to help define, measure, and improve the quality of AI systems powered by Large Language Models (LLMs). In this role, you will design and scale evaluation frameworks that ensure AI-generated outputs are accurate, reliable, and consistent across a variety of use cases. You will partner closely with Product, Engineering, Research, and Data teams to establish evaluation methodologies, automate quality assessment workflows, and develop the metrics that drive model improvements. This is an ideal opportunity for someone who enjoys solving complex problems at the intersection of AI quality, evaluation strategy, automation, and data-driven decision making. Top Skills * AI/LLM Evaluation & Quality Measurement * Problem Solving & System Design * Communication & Ownership * Python, SQL & Automation Top Responsibilities * Design and build LLM evaluation frameworks, including automated scoring systems, rubric-based grading, and LLM-as-a-Judge solutions. * Define and track evaluation metrics, KPIs, and dashboards that measure model quality and performance at scale. * Perform error analysis, root-cause investigations, and quality deep dives to identify opportunities for model improvement. * Partner with cross-functional teams to integrate evaluation standards, workflows, and quality requirements into production environments.