1Fuzzing OpenSSL 2=============== 3 4OpenSSL can use either LibFuzzer or AFL to do fuzzing. 5 6LibFuzzer 7--------- 8 9How to fuzz OpenSSL with [libfuzzer](http://llvm.org/docs/LibFuzzer.html), 10starting from a vanilla+OpenSSH server Ubuntu install. 11 12With `clang` from a package manager 13----------------------------------- 14 15Install `clang`, which [ships with `libfuzzer`](http://llvm.org/docs/LibFuzzer.html#fuzzer-usage) 16since version 6.0: 17 18 sudo apt-get install clang 19 20Configure `openssl` for fuzzing. For now, you'll still need to pass in the path 21to the `libFuzzer` library file while configuring; this is represented as 22`$PATH_TO_LIBFUZZER` below. A typical value would be 23`/usr/lib/llvm-7/lib/clang/7.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a`. 24 25 CC=clang ./config enable-fuzz-libfuzzer \ 26 --with-fuzzer-lib=$PATH_TO_LIBFUZZER \ 27 -DPEDANTIC enable-asan enable-ubsan no-shared \ 28 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \ 29 -fsanitize=fuzzer-no-link \ 30 enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \ 31 enable-weak-ssl-ciphers enable-rc5 enable-md2 \ 32 enable-ssl3 enable-ssl3-method enable-nextprotoneg \ 33 --debug 34 35Clang uses the gcc libstdc++ library so this must also be installed. You can 36check which version of gcc clang is using like this: 37 38 $ clang --verbose 39 Ubuntu clang version 14.0.0-1ubuntu1.1 40 Target: x86_64-pc-linux-gnu 41 Thread model: posix 42 InstalledDir: /usr/bin 43 Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/12 44 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10 45 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11 46 Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12 47 Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12 48 Candidate multilib: .;@m64 49 Selected multilib: .;@m64 50 51So, in the above example clang is using gcc version 12. Ensure that the selected 52gcc version has the relevant libstdc++ files installed: 53 54 $ ls /usr/lib/gcc/x86_64-linux-gnu/12 | grep stdc++ 55 libstdc++.a 56 libstdc++fs.a 57 libstdc++.so 58 59On Ubuntu for gcc-12 this requires the libstdc++-12-dev package installed. 60 61 $ sudo apt-get install libstdc++-12-dev 62 63Compile: 64 65 sudo apt-get install make 66 make clean 67 LDCMD=clang++ make -j4 68 69Finally, perform the actual fuzzing: 70 71 fuzz/helper.py $FUZZER 72 73where $FUZZER is one of the executables in `fuzz/`. 74It will run until you stop it. 75 76If you get a crash, you should find a corresponding input file in 77`fuzz/corpora/$FUZZER-crash/`. 78 79With `clang` from source/pre-built binaries 80------------------------------------------- 81 82You may also wish to use a pre-built binary from the [LLVM Download 83site](http://releases.llvm.org/download.html), or to [build `clang` from 84source](https://clang.llvm.org/get_started.html). After adding `clang` to your 85path and locating the `libfuzzer` library file, the procedure for configuring 86fuzzing is the same, except that you also need to specify 87a `--with-fuzzer-include` option, which should be the parent directory of the 88prebuilt fuzzer library. This is represented as `$PATH_TO_LIBFUZZER_DIR` below. 89 90 CC=clang ./config enable-fuzz-libfuzzer \ 91 --with-fuzzer-include=$PATH_TO_LIBFUZZER_DIR \ 92 --with-fuzzer-lib=$PATH_TO_LIBFUZZER \ 93 -DPEDANTIC enable-asan enable-ubsan no-shared \ 94 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \ 95 -fsanitize=fuzzer-no-link \ 96 enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \ 97 enable-weak-ssl-ciphers enable-rc5 enable-md2 \ 98 enable-ssl3 enable-ssl3-method enable-nextprotoneg \ 99 --debug 100 101AFL 102--- 103 104This is an alternative to using LibFuzzer. 105 106Configure for fuzzing: 107 108 sudo apt-get install afl-clang 109 CC=afl-clang-fast ./config enable-fuzz-afl no-shared no-module \ 110 -DPEDANTIC enable-tls1_3 enable-weak-ssl-ciphers enable-rc5 \ 111 enable-md2 enable-ssl3 enable-ssl3-method enable-nextprotoneg \ 112 enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \ 113 --debug 114 make clean 115 make 116 117The following options can also be enabled: enable-asan, enable-ubsan, enable-msan 118 119Run one of the fuzzers: 120 121 afl-fuzz -i fuzz/corpora/$FUZZER -o fuzz/corpora/$FUZZER/out fuzz/$FUZZER 122 123Where $FUZZER is one of the executables in `fuzz/`. 124 125Reproducing issues 126------------------ 127 128If a fuzzer generates a reproducible error, you can reproduce the problem using 129the fuzz/*-test binaries and the file generated by the fuzzer. They binaries 130don't need to be built for fuzzing, there is no need to set CC or the call 131config with enable-fuzz-* or -fsanitize-coverage, but some of the other options 132above might be needed. For instance the enable-asan or enable-ubsan option might 133be useful to show you when the problem happens. For the client and server fuzzer 134it might be needed to use -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION to 135reproduce the generated random numbers. 136 137To reproduce the crash you can run: 138 139 fuzz/$FUZZER-test $file 140 141To do all the tests of a specific fuzzer such as asn1 you can run 142 143 fuzz/asn1-test fuzz/corpora/asn1 144or 145 make test TESTS=fuzz_test_asn1 146 147To run several fuzz tests you can use for instance: 148 149 make test TESTS='test_fuzz_cmp test_fuzz_cms' 150 151To run all fuzz tests you can use: 152 153 make test TESTS='test_fuzz_*' 154 155Random numbers 156-------------- 157 158The client and server fuzzer normally generate random numbers as part of the TLS 159connection setup. This results in the coverage of the fuzzing corpus changing 160depending on the random numbers. This also has an effect for coverage of the 161rest of the test suite and you see the coverage change for each commit even when 162no code has been modified. 163 164Since we want to maximize the coverage of the fuzzing corpus, the client and 165server fuzzer will use predictable numbers instead of the random numbers. This 166is controlled by the FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION define. 167 168The coverage depends on the way the numbers are generated. We don't disable any 169check of hashes, but the corpus has the correct hash in it for the random 170numbers that were generated. For instance the client fuzzer will always generate 171the same client hello with the same random number in it, and so the server, as 172emulated by the file, can be generated for that client hello. 173 174Coverage changes 175---------------- 176 177Since the corpus depends on the default behaviour of the client and the server, 178changes in what they send by default will have an impact on the coverage. The 179corpus will need to be updated in that case. 180 181Updating the corpus 182------------------- 183 184The client and server corpus is generated with multiple config options: 185 186- The options as documented above 187- Without enable-ec_nistp_64_gcc_128 and without --debug 188- With no-asm 189- Using 32 bit 190- A default config, plus options needed to generate the fuzzer. 191 192The libfuzzer merge option is used to add the additional coverage 193from each config to the minimal set. 194 195Minimizing the corpus 196--------------------- 197 198When you have gathered corpus data from more than one fuzzer run 199or for any other reason want to minimize the data 200in some corpus subdirectory `fuzz/corpora/DIR` this can be done as follows: 201 202 mkdir fuzz/corpora/NEWDIR 203 fuzz/$FUZZER -merge=1 fuzz/corpora/NEWDIR fuzz/corpora/DIR 204