High test coverage is a vanity metric because it only measures code execution, not the quality or value of the verification.
Before implementing mutation testing, ensure your existing unit and integration tests are stable and not flaky to get meaningful results.
Mutation testing is a language-agnostic philosophy with various tools available like StrykerJS for TypeScript, Mutmut for Python, and others.
The demonstration walks through a simple e-commerce application built with TypeScript to showcase how mutation testing works in a real project.
Running the initial test suite shows 100% coverage, but a closer look reveals weak assertions that don't validate business logic correctly.
Follow the steps to install StrykerJS dependencies and create a configuration file to specify the package manager, reporters, and files to mutate.
The first StrykerJS run reveals a low mutation score, indicating that many "mutants" survived because the existing tests were not strong enough to catch them.
By replacing a weak `toBeDefined` check with a specific assertion that validates business logic, the test can now "kill" more mutants.
After strengthening the test assertion, re-running StrykerJS on the specific file shows a significant increase in the mutation score.
Mutation testing reports can be integrated into CI/CD workflows like GitHub Actions to provide immediate feedback and block pull requests that don't meet a quality threshold.
The key takeaway is that test coverage is an illusion, and mutation testing helps ensure your tests provide real verification and value.
Jobs that call for the skills explored in this talk.