> Markdown version of [/jobs/ext/2298890-principal-software-engineer-for-copilot-evals](https://www.wearedevelopers.com/jobs/ext/2298890-principal-software-engineer-for-copilot-evals). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Principal Software Engineer for Copilot Evals - **Company:** Microsoft - **Location:** Redmond, WA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** C (Programming Language), Java (Programming Language), JavaScript (Programming Language), Artificial Intelligence, C Sharp (Programming Language), C++ (Programming Language), Data Governance, Distributed Systems, Python (Programming Language), Azure Machine Learning, Data Streaming, AI Infrastructure, Large Language Models, Information Technology, Data Management - **Published:** August 29, 2026 - **Apply:** https://www.careerjet.com/job/us088cdb1ebbe1f6c75edf94f37e1f6aaf/eaa ## About the Role Establish data contracts and extensible schemas for customer signals, workflows, evaluation cases, rubrics, graders, runs, results, attribution, and regression assets. Develop quality-attribution workflows that combine automated diagnosis with human review; route losses to the appropriate product, data, model, tool, orchestration, or evaluation owner; track remediation; add regression protection; and verify improvement in customer outcomes. Create observability and benchmarks for pipeline health, agent behavior, data quality, grader health, coverage, cost, latency, operational reliability, and regression detection across CADET workflows and tooling. Design safe human-in-the-loop gates for ambiguous data, sensitive customer evidence, taxonomy changes, evaluation publication, and consequential quality decisions. Mentor engineers across the team and raise engineering quality through clear interfaces, testing, security, operational discipline, and technical documentation. Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. Extensive experience designing and delivering large-scale distributed systems, data platforms, developer platforms, or AI infrastructure in production, with practical exposure to large language model technologies, agent systems, or AI/ML platforms. Demonstrated principal-level technical leadership across teams and organizational boundaries. Solid understanding of security, privacy, data governance, and responsible engineering for sensitive data. Excellent communication and collaboration skills with the ability to influence senior engineering, science, and product leaders. Experience building AI/ML platforms, LLM or agent systems, evaluation infrastructure, experimentation systems, or model pipelines. Experience with retrieval, tool-using agents, prompt and rubric management, human-in-the-loop systems, or automated diagnosis. Experience building data and workflow platforms that combine batch, streaming, unstructured, and human-generated inputs. Experience with RLEs, synthetic environments, post-training pipelines, or large-scale evaluation execution. Experience integrating customer feedback, product telemetry, and engineering work-management systems. ## Related Videos - [Python-Based Data Streaming Pipelines Within Minutes](https://www.wearedevelopers.com/videos/1233-python-based-data-streaming-pipelines-within-minutes) - [Data Governance in the Era of AI](https://www.wearedevelopers.com/videos/1622-data-governance-in-the-era-of-ai) - [It's all about the Data](https://www.wearedevelopers.com/videos/425-it-s-all-about-the-data) - [AI Pair Programming with GitHub Copilot at SAP: Looking Back, Looking Forward!](https://www.wearedevelopers.com/videos/1546-ai-pair-programming-with-github-copilot-at-sap-looking-back-looking-forward) - [Unlocking Value from Data: The Key to Smarter Business Decisions-](https://www.wearedevelopers.com/videos/1367-unlocking-value-from-data-the-key-to-smarter-business-decisions) - [The Data Mesh as the end of the Datalake as we know it](https://www.wearedevelopers.com/videos/156-the-data-mesh-as-the-end-of-the-datalake-as-we-know-it) ## Related Articles - [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care) - [Liuba Gonta and Yuliya Khadasevic - GitHub Copilot Beyond the Basics - 10 Ways to Elevate Your Coding](https://www.wearedevelopers.com/magazine/490-liuba-gonta-and-yuliya-khadasevic-github-copilot-beyond-the-basics-10-ways-to-elevate-your-coding) - [GitHub Copilot: Beyond the Basics – 10 Ways to Elevate Your Coding](https://www.wearedevelopers.com/magazine/524-github-copilot-beyond-the-basics-10-ways-to-elevate-your-coding) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this) - [How software is steering vehicle technology](https://www.wearedevelopers.com/magazine/515-how-software-is-steering-vehicle-technology)