Mutation Testing

Mutation testing is making changes to a program, either manually or automatically, to identify changes that can be made that don't get caught by tests.

Mutation testing is similar to measuring code coverage. It's goal is to identify code not covered by tests, or possibly that looks like it's covered by tests because the lines of code are executing during a test, but the outcomes themselves such as contract function return value, or events published, are not tested.

How to do Mutation Testing

The cargo-mutants tool can be used to automatically and iteratively modify the Rust code, and rerun the tests after each mutation, to identify code not tested.

1. Install cargo-mutants:

   cargo install --locked cargo-mutants

2. Run the cargo mutants command inside your contract's crate directory.

   $ cargo mutants
   Found 4 mutants to test
   ok       Unmutated baseline in 19.0s build + 0.6s test
    INFO Auto-set test timeout to 20s
   MISSED   src/lib.rs:14:9: replace IncrementContract::increment -> u32 with 1 in 0.4s build + 0.4s test
   4 mutants tested in 23s: 1 missed, 3 caught

   Code that is identified as not covered by a test will be outputted as a MISSED line in the output.

   Diffs of each change that was attempted can be found in the mutants.out/diff directory.