Fuzz Testing Daala

A place where I can keep notes on fuzz testing daala (adapted from https://gist.github.com/iankronquist/a15ad39f7231454c2f61 )

16:50:05         radens | Do you guys run fuzz testers on daala? I was playing around with afl-fuzz
                        | today and was thinking of the recent android bug.
16:53:38      +TD-Linux | radens, no, and we should
16:54:03      +TD-Linux | tons of fuzzing was done on opus, though.
16:54:13         radens | if I wanted to play around with that how would you suggest I get started?
16:54:50          +derf | Step 1 is to get yourself a version of libogg that disables the CRC
                        | check.
16:54:57         radens | CRC?
16:55:19          +derf | https://en.wikipedia.org/wiki/Cyclic_redundancy_check
16:57:18         radens | derf: would that be in here? https://github.com/gcp/libogg/blob/ab78196fd
                        | 59ad7a329a2b19d2bcec5d840a9a21f/src/framing.c
16:57:39         radens | I'll also have to figure out how to link daala against a source libogg,
                        | but that may not be so hard.
16:57:41          +derf | Yes, you'll need to apply a patch like
                        | https://pastebin.mozilla.org/8842594

Patching libogg

The patch to apply to libogg: http://people.xiph.org/~tmatth/ogg-no-crc.patch

Apply the patch and install in ~/local, e.g.,:

$ ./configure --prefix=/home/tristan/local && make && make install

I tried using afl-fuzz.

$ make clean
$ # Replace clang with gcc if you intend to go down that road
$ export AFL_CC=`which clang`
$ export AFL_CXX=`which clang++` # probably not necessary, not compiling any C++
$ PKG_CONFIG_PATH=/home/tristan/local/lib/pkgconfig CC=afl-clang ./configure
$ make

Make a 2 frame video

Use https://people.xiph.org/~tmatth/smallsrc.y4m

$ ./examples/encoder_example -v 30 smallsrc.y4m -o smallsrc.ogv

Dump a video

17:35:12          +derf | radens: Instead of modifying the dump_video shell script, you want to do
                        | something like libtool --mode=execute afl-fuzz <...>
                        | ./examples/dump_video

Enable assertions

./configure --enable-assertions

Running in parallel

On Linux, afl-fuzz will complain about CPU frequency scaling, so either do as instructed:

echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

or tell afl-fuzz to ignore CPUFREQ settings:

AFL_SKIP_CPUFREQ=1 afl-fuzz -i in -o out [path_to_tool] @@

Source: https://fuzzing-project.org/tutorial3.html

Run master process

libtool --mode=execute afl-fuzz -i testcases -o sync_dir -M fuzzer01 examples/encoder_example @@ -o /dev/null

Run secondary processes

libtool --mode=execute afl-fuzz -i testcases -o sync_dir -M fuzzer02 examples/encoder_example @@ -o /dev/null libtool --mode=execute afl-fuzz -i testcases -o sync_dir -M fuzzer03 examples/encoder_example @@ -o /dev/null libtool --mode=execute afl-fuzz -i testcases -o sync_dir -M fuzzer04 examples/encoder_example @@ -o /dev/null

To resume an interrupted process, replace "testcases" with "-". Stop after the main process has completed at least one cycle.

Minimizing testcases

Source: https://foxglovesecurity.com/2016/03/15/fuzzing-workflows-a-fuzz-job-from-start-to-finish/

cd sync_dir
mkdir queue_all
cp fuzzer*/queue/* queue_all/
libtool --mode=execute afl-cmin -i queue_all/ -o queue_cmin -- ../examples/encoder_example @@ -o /dev/null

This stage actually hung for me and is not strictly necessary.

Existing testcases

https://people.xiph.org/~tmatth/sync_dir.tgz

Note: if a crash is not reproducible, you may have to force the memory limit logged by AFL (e.g. 50M). If not reproducible in GDB, you may need to run (gdb) set disable-randomization off

To pipe a file as input when running in gdb, do (gdb ) run < crasher_input_file

Misc. Links

https://medium.com/@kierank_/towards-crashless-multimedia-playback-61938e867c66#.vc5bvdrtd

FLIF-hub/FLIF#304

tmatth/Fuzz testing daala.md