Lean Intelligence: Lessons from GitHub Copilot Data Science Efforts

GitHub Copilot handles millions of developer interactions every day! Behind every suggestion, chat response, and agentic action are countless decisions about which model or subagent to invoke, how long to let it run, and when the answer is already good enough. This talk shares lessons from the Code|AI research team's work on making the Copilot harness leaner without sacrificing quality. We'll dig into the experimentation, evaluation, and data science that helped us find and validate meaningful efficiency gains at scale. If evals, experimentation, and data science are your jam, this one's for you.