AI User Experience Reliability Lead

• Define and execute the strategy for measuring and improving the accuracy, stability, and task success of Qira's AI actions.

• Build and evolve evaluation frameworks, behavioral scorecards, and quality validation for models, prompts, retrievers, and task orchestration.

• Develop systems to detect hallucinations, regressions, safety deviations, and other behavioral anomalies in real time.

Safety & Guardrail Reliability

• Ensure the reliability of runtime safety systems, including content moderation, jailbreak/misuse detection, safety classifiers, and policy enforcement.

• Partner with Safety, Legal, Ethics, and Product teams to convert requirements into robust technical safety solutions.

• Validate safety updates through testing, evaluation, and monitored deployment.

AI Observability & Telemetry

• Define the telemetry, metrics, traces, and data needed to understand AI behavior endtoend across device, edge, and cloud.

• Collaborate with observability and platform teams to integrate AIspecific signals (quality, drift, safety events) into a unified reliability platform.

• Lead the creation of dashboards and analytics that provide deep insight into AI behavior and experience reliability.

Reliability Engineering & Architecture Influence

• Partner with engineering, AI/ML, and product teams to embed reliability into the design of prompts, models, policies, and workflow orchestration.

• Influence architecture to ensure AI behavior is predictable, testable, explainable, and resilient.

• Establish standards for AI evaluation, rollout safety, service readiness, and runtime validation.

CrossFunctional Leadership

• Represent AI experience reliability in architectural reviews, product decisions, roadmap development, and launch readiness.

• Drive cross-team alignment on reliability metrics, evaluation methods, and monitoring strategies.

• Collaborate with ML researchers, applied AI teams, data scientists, and UX to ensure usercentric reliability goals.

Execution & Delivery

• Lead major engineering initiatives across AI quality, evaluation, safety assurance, and behavioral monitoring.

• Set priorities, ensure accountability, and drive timely delivery of reliability systems and tooling.

• Foster a culture of engineering excellence, learning, and continuous improvement.

• 8+ years in AI/ML engineering, evaluation engineering, applied ML, reliability engineering, or large-scale distributed systems, with depth in AI behavior, evaluation, or safety.

• Bachelor's Degree in Computer Science, Engineering, Machine Learning, or a related field.

• Strong programming skills in Python (Go, Java, or C++ a plus).

• Experience instrumenting, evaluating, or operating AI systems in production.

• Deep understanding of LLMs, model behavior, evaluation methods, retrievalaugmented systems, or content moderation logic.

• Strong ability to lead technical initiatives and influence crossfunctional engineering teams.

Preferred Qualifications

• Experience with OpenTelemetry, Grafana, Prometheus, Loki, Tempo, or similar observability systems.

• Hands-on experience with hallucination detection, behavioral anomaly detection, or evaluation frameworks at scale.

• Experience in AI safety engineering, runtime validation, or policy enforcement systems.

• Understanding of hybrid architectures (device + edge + cloud).

• Background guiding teams or owning cross-functional architectural decisions.

• A passion for building AI systems that are correct, safe, reliable, and deeply aligned with user expectations.

Why Work at Lenovo We are Lenovo. We do what we say. We own what we do. We WOW our customers. Lenovo is a US$69 billion revenue global technology powerhouse, ranked #196 in the Fortune Global 500, and serving millions of customers every day in 180 markets. Focused on a bold vision to deliver Smarter Technology for All, Lenovo has built on its success as the world's largest PC company with a full-stack portfolio of AI-enabled, AI-ready, and AI-optimized devices (PCs, workstations, smartphones, tablets), infrastructure (server, storage, edge, high performance computing and software defined infrastructure), software, solutions, and services. Lenovo's continued investment in world-changing innovation is building a more equitable, trustworthy, and smarter future for everyone, everywhere. Lenovo is listed on the Hong Kong stock exchange under Lenovo Group Limited (HKSE: 992) (ADR: LNVGY). This transformation together with Lenovo's world-changing innovation is building a more inclusive, trustworthy, and smarter future for everyone, everywhere. To find out more visit www.lenovo.com, and read about the latest news via our StoryHub. Description and Requirements About Our Team Lenovo is building Quantum, a nextgeneration hybrid AI platform spanning Windows, Android, and cloud. As part of this vision, we are expanding the engineering organization behind Qira, Lenovo's crossdevice Personal AI that delivers intelligent, safe, and reliable experiences across Lenovo and Motorola products. We are hiring an AI User Experience Reliability Lead to define and drive the technical strategy for reliability across Qira's AI behaviors, outputs, and runtime safety systems. This role leads the engineering direction for how Qira's intelligence is evaluated, monitored, and improved - ensuring users experience correct, consistent, stable, and trustworthy AI behavior at global scale. This is a critical leadership position shaping one of the most important engineering domains within Qira and Lenovo's broader AI strategy., Qira's value is defined by its experience reliability - the confidence that users can trust its responses, its behavior, and its safety. In this role, you will: * Define the standards for AI behavioral quality and correctness * Build the systems that measure, validate, and enforce those standards * Influence architecture, evaluation, safety, and production behavior * Shape the AI experience for millions of Lenovo and Motorola users This is a rare opportunity to lead one of the most strategically important engineering functions in Lenovo's AI roadmap. The base salary budgeted range for this position is $190K - $230K. Individuals may also be considered for bonus and/or commission. Lenovo's various benefits can be found on www.lenovobenefits.com. We are an Equal Opportunity Employer and do not discriminate against any employee or applicant for employment because of race, color, sex, age, religion, sexual orientation, gender identity, national origin, status as a veteran, and basis of disability or any federal, state, or local protected class. Additional Locations: * United States of America - Illinois - Chicago * United States of America * United States of America - Illinois