Fuzzing

​

How to Write Fuzz Tests with cargo-fuzz

1. Install the nightly Rust toolchain. Nightly Rust is required to run cargo-fuzz.
   rustup install nightly
   
2. Install cargo-fuzz.
   cargo install --locked cargo-fuzz
   
3. Initialize a fuzz project by running the following command inside your contract directory.
   cargo fuzz init
   
4. Open the contract’s Cargo.toml file. Add lib as a crate-type.
    [lib]
   -crate-type = ["cdylib"]
   +crate-type = ["lib", "cdylib"]
   
5. Open the generated fuzz/Cargo.toml file. Add the soroban-sdk dependency.
    [dependencies]
    libfuzzer-sys = "0.4"
   +soroban-sdk = { version = "*", features = ["testutils"] }
   
6. Open the generated fuzz/src/fuzz_target_1.rs file. It will look like the below.
   #![no_main]
   use libfuzzer_sys::fuzz_target;
   
   fuzz_target!(|data: &[u8]| {
       // fuzzed code goes here
   });
   
7. Fill out the fuzz_target! call with test setup and assertions. For example, for the increment example:
   #![no_main]
   use libfuzzer_sys::fuzz_target;
   use soroban_increment_with_fuzz_contract::{IncrementContract, IncrementContractClient};
   use soroban_sdk::{
       testutils::arbitrary::{arbitrary, Arbitrary},
       Env,
   };
   
   #[derive(Debug, Arbitrary)]
   pub struct Input {
       pub by: u64,
   }
   
   fuzz_target!(|input: Input| {
       let env = Env::default();
       let id = env.register(IncrementContract, ());
       let client = IncrementContractClient::new(&env, &id);
   
       let mut last: Option<u32> = None;
       for _ in input.by.. {
           match client.try_increment() {
               Ok(Ok(current)) => assert!(Some(current) > last),
               Err(Ok(_)) => {} // Expected error
               Ok(Err(_)) => panic!("success with wrong type returned"),
               Err(Err(_)) => panic!("unrecognised error"),
           }
       }
   });
   
8. Execute the fuzz target.
   cargo +nightly fuzz run --sanitizer=thread fuzz_target_1
   
   If you’re developing on MacOS you need to add the --sanitizer=thread flag in order to work around a known issue.

1. Create an environment, the Env.
2. Register the contract to be tested.
3. Invoke functions using a client.
4. Assert expectations.

​

How to Write Fuzz Tests with cargo-afl

1. Install cargo-afl. Installing it builds AFL++ from source, so a C compiler needs to be available.
   cargo install cargo-afl --locked
   
2. Configure the machine for fuzzing.
   cargo afl system-config
   
3. Open the contract’s Cargo.toml file. Add lib as a crate-type. The fuzz target imports the contract as a Rust library.
    [lib]
   -crate-type = ["cdylib"]
   +crate-type = ["lib", "cdylib"]
   
4. Create a fuzz target crate, for example with cargo new --bin fuzz inside your contract’s directory. Unlike a cargo-fuzz target, an AFL++ target depends on the arbitrary crate directly, because the fuzz! macro expands to code that refers to it by an absolute path, which only resolves if arbitrary is a direct dependency. Put the following in the new crate’s Cargo.toml. It replaces the empty [dependencies] table that cargo new generated.
   [dependencies]
   afl = "0.18"
   arbitrary = { version = "~1.3.0", features = ["derive"] }
   soroban-sdk = { version = "*", features = ["testutils"] }
   # The contract to fuzz. Use the package name from its Cargo.toml.
   soroban-increment-contract = { path = ".." }
   
   [[bin]]
   name = "fuzz_target_1"
   path = "src/fuzz_target_1.rs"
   
   # Prevent this from interfering with the contract's workspace, if it has one.
   [workspace]
   members = ["."]
   
5. Write the fuzz target at src/fuzz_target_1.rs, and delete the default src/main.rs cargo new created. An AFL++ target is a regular binary crate with a main function that calls the afl::fuzz! macro. For example, for the increment example:
   use afl::fuzz;
   use arbitrary::Arbitrary;
   use soroban_increment_contract::{IncrementContract, IncrementContractClient};
   use soroban_sdk::Env;
   
   #[derive(Debug, Arbitrary)]
   pub struct Input {
       pub by: u8,
   }
   
   fn main() {
       fuzz!(|input: Input| {
           // Create the `Env` inside the closure, not outside: AFL++ reuses the
           // process for many inputs, and state created outside the closure
           // would leak from one input into the next.
           let env = Env::default();
           let id = env.register(IncrementContract, ());
           let client = IncrementContractClient::new(&env, &id);
   
           let mut last: Option<u32> = None;
           for _ in 0..input.by {
               match client.try_increment() {
                   Ok(Ok(current)) => {
                       assert!(Some(current) > last);
                       last = Some(current);
                   }
                   Err(Ok(_)) => {} // Expected error
                   Ok(Err(_)) => panic!("success with wrong type returned"),
                   Err(Err(_)) => panic!("unrecognised error"),
               }
           }
       });
   }
   
6. Build the target. The remaining steps also run from inside the fuzz crate.
   cd fuzz
   cargo afl build
   
7. Fuzz the target, pointing at an input directory containing at least one seed input and an output directory to write results to.
   mkdir in out
   echo -n '00000000' > in/seed
   cargo afl fuzz -i in -o out target/debug/fuzz_target_1
   

RUST_BACKTRACE=1 ./target/debug/fuzz_target_1 < out/default/crashes/id:000000*

​

How to Get Code Coverage of cargo-fuzz Tests

1. Run the cargo-fuzz target until it has produced a corpus.
   cargo +nightly fuzz run --sanitizer thread fuzz_target_1
   
2. Install the llvm-tools for the nightly compiler.
   rustup component add --toolchain nightly llvm-tools-preview
   
3. Run the fuzz coverage command that’ll execute the corpus and write coverage data to the coverage directory in the profdata format.
   cargo +nightly fuzz coverage --sanitizer thread fuzz_target_1
   
4. Run the llvm-cov command to convert the profdata file to an lcov file.
   $(find $(rustc --print sysroot) -name llvm-cov) export \
       -instr-profile=fuzz/coverage/fuzz_target_1/coverage.profdata \
       -object target/$(rustc -vV | sed -n 's|host: ||p')/coverage/$(rustc -vV | sed -n 's|host: ||p')/release/fuzz_target_1 \
       --ignore-filename-regex "rustc" \
       -format=lcov \
       > lcov.info
   
   Load the lcov.info file into your IDE using its coverage feature. In VSCode this can be done by installing the Coverage Gutters extension and executing the Coverage Gutters: Watch command.