CodePark/ questions / Coverage, mutation testing & CI
START PRACTICINGPRACTICE
Backend question · Backend testing › Coverage, mutation testing & CI

What is mutation testing, and what does it tell you that coverage doesn't?

Mutation testing checks whether your tests would catch real bugs. Coverage only shows that code ran.

A tool such as Stryker makes many small changes to your code. Each changed copy is a mutant. Examples:

  • flipping > to >=
  • replacing && with ||
  • removing a call
  • returning an empty string

The tool runs your tests against each mutant:

  • If a test fails, the mutant is killed. Good: your tests noticed the bug.
  • If every test still passes, the mutant survived. No test would notice that bug.

The mutation score is the share of mutants killed.

What it adds over coverage: it shows whether your assertions depend on what the code does. Each surviving mutant points to a precise weak spot, like a missing boundary test.

The cost: it is slow, because the whole suite runs many times. So:

  • Focus it on core domain modules.
  • Use incremental runs in CI (only changed code).
  • Accept some equivalent mutants: changes that don't change behaviour, so no test can kill them.
  • Mutants are small, deliberate changes to the code.
  • Killed means a test caught the change. Survived means there is a gap in the tests.
  • The mutation score measures how strong the tests are, not just what ran.
  • Mutation testing is slow, so target core modules and run it incrementally.