Test Intelligence for Developers

When a system has “grown historically,” this often applies to the tests as well: they run for too long but find too few bugs. The reason is that such test suites often test too much and too little at the same time. They test too much because they include tests that incur costs but provide little added value compared to similar tests. They test too little because important functionality remains untested.

In the talk, I will present analyses mine data from your repos (version control, issue tracker, test coverage etc) to uncover these issues: Pareto optimization of test suites and test impact analysis identify the tests that currently have the best cost-benefit ratio. Test gap analysis reveals which code changes are still untested, indicating missing tests.

By doing so, these analyses provide us as developers with effective tools to understand and improve our test architecture. For each analysis, I outline its research background and show practical examples from large systems.