Mutation Testing

Mutation testing is making changes to a program, either manually or automatically, to identify changes that can be made that don't get caught by tests.

Mutation testing is similar to measuring [code coverage]. It's goal is to identify code not covered by tests, or possibly that looks like it's covered by tests because the lines of code are executing during a test, but the outcomes themselves such as contract function return value, or events published, are not tested.

How to do Mutation Testing

The cargo-mutants tool can be used to automatically and iteratively modify the Rust code, and rerun the tests after each mutation, to identify code not tested.

1. Install cargo-mutants:

   cargo install --locked cargo-mutants

2. Run the cargo mutants command inside your contract's crate directory.

   $ cargo mutants
   Found 4 mutants to test
   ok       Unmutated baseline in 19.0s build + 0.6s test
    INFO Auto-set test timeout to 20s
   MISSED   src/lib.rs:14:9: replace IncrementContract::increment -> u32 with 1 in 0.4s build + 0.4s test
   4 mutants tested in 23s: 1 missed, 3 caught

   Code that is identified as not covered by a test will be outputted as a MISSED line in the output.

   Diffs of each change that was attempted can be found in the mutants.out/diff directory.