Evaluating AI models for code comprehension

With so many AI models out there, which one is actually the best at understanding code?

As we’ve built Graphite Reviewer, our high-signal AI-powered code review companion, we’ve run thousands of tests, gotten tons of feedback from users, and have extensively examined how these various models perform in real world scenarios.

In this talk we’ll explore the nuances between these models, how they perform specifically in relation to code comprehension, and finally answer the question: which AI model should developers be using?

This talk will include a deep dive into the various experiments we’ve run on AI code comprehension, comparing all of the top AI models today, and an exploration of the results and data we’ve collected.