A place where I can keep notes on fuzz testing daala.
All tests here were done against commit sha d8daca8e9aadb1f6ba53e089b89824f170d59703 from Fri May 1, 2015.
16:50:05 radens | Do you guys run fuzz testers on daala? I was playing around with afl-fuzz
| today and was thinking of the recent android bug.
16:53:38 +TD-Linux | radens, no, and we should
16:54:03 +TD-Linux | tons of fuzzing was done on opus, though.
16:54:13 radens | if I wanted to play around with that how would you suggest I get started?
16:54:50 +derf | Step 1 is to get yourself a version of libogg that disables the CRC
| check.
16:54:57 radens | CRC?
16:55:19 +derf | https://en.wikipedia.org/wiki/Cyclic_redundancy_check
16:57:18 radens | derf: would that be in here? https://github.com/gcp/libogg/blob/ab78196fd
| 59ad7a329a2b19d2bcec5d840a9a21f/src/framing.c
16:57:39 radens | I'll also have to figure out how to link daala against a source libogg,
| but that may not be so hard.
16:57:41 +derf | Yes, you'll need to apply a patch like
| https://pastebin.mozilla.org/8842594
The patch to apply to libogg: https://pastebin.mozilla.org/8842594
I tried using afl-fuzz.
$ make clean
$ # Replace clang with gcc if you intend to go down that road
$ export AFL_CC=`which clang`
$ export AFL_CC=`which clang++`
$ CC=afl-clang ./configure
$ make
Use the claire_qcif-frames.y4m video from the wiki.
$ ./examples/encoder_example -v 30 claire_qcif-5.994Hz-labels.y4m -o claire.ogv
17:35:12 +derf | radens: Instead of modifying the dump_video shell script, you want to do
| something like libtool --mode=execute afl-fuzz <...>
| ./examples/dump_video
./configure --enable-assertions
american fuzzy lop 1.83b (dump_video)
┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐
│ run time : 6 days, 5 hrs, 22 min, 52 sec │ cycles done : 1 │
│ last new path : 0 days, 2 hrs, 38 min, 51 sec │ total paths : 1351 │
│ last uniq crash : 2 days, 5 hrs, 29 min, 2 sec │ uniq crashes : 47 │
│ last uniq hang : 2 days, 13 hrs, 57 min, 26 sec │ uniq hangs : 51 │
├─ cycle progress ────────────────────┬─ map coverage ─┴───────────────────────┤
│ now processing : 107* (7.92%) │ map density : 2365 (3.61%) │
│ paths timed out : 0 (0.00%) │ count coverage : 6.05 bits/tuple │
├─ stage progress ────────────────────┼─ findings in depth ────────────────────┤
│ now trying : interest 16/8 │ favored paths : 63 (4.66%) │
│ stage execs : 12.1k/37.6k (32.23%) │ new edges on : 91 (6.74%) │
│ total execs : 11.3M │ total crashes : 82.6k (47 unique) │
│ exec speed : 9.57/sec (zzzz...) │ total hangs : 39.3k (51 unique) │
├─ fuzzing strategy yields ───────────┴───────────────┬─ path geometry ────────┤
│ bit flips : 601/684k, 62/684k, 62/683k │ levels : 8 │
│ byte flips : 5/85.5k, 5/53.7k, 3/54.9k │ pending : 1220 │
│ arithmetics : 243/2.96M, 16/1.10M, 5/472k │ pend fav : 0 │
│ known ints : 40/278k, 81/1.29M, 96/2.17M │ own finds : 1350 │
│ dictionary : 0/0, 0/0, 25/509k │ imported : n/a │
│ havoc : 92/190k, 0/0 │ variable : 0 │
│ trim : 28.61%/45.1k, 37.73% ├────────────────────────┘
└─────────────────────────────────────────────────────┘ [cpu: 26%]
Tarball: http://people.oregonstate.edu/~kronquii/daala-afl-no-assertions-round-1.tar.gz
iankronquist@puppettop:(daala)(master) → ./libtool --mode=execute lldb ./examples/dump_video
(lldb) target create "/Users/iankronquist/gg/daala/examples/.libs/dump_video"
Current executable set to '/Users/iankronquist/gg/daala/examples/.libs/dump_video' (x86_64).
(lldb) r fuzz_out/crashes/id:000000,sig:06,src:000000,op:flip1,pos:39
Process 84907 launched: '/Users/iankronquist/gg/daala/examples/.libs/dump_video' (x86_64)
Encoded by Xiph's experimental encoder library daala 0.0-1018-gd8daca8
dump_video(84907,0x7fff7eca9300) malloc: *** mach_vm_map(size=18446744073307938816) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
dump_video(84907,0x7fff7eca9300) malloc: *** error for object 0xfffffffffffffffc: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 84907 stopped
* thread #1: tid = 0x1091b83, 0x00007fff9a714286 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
frame #0: 0x00007fff9a714286 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
-> 0x7fff9a714286 <+10>: jae 0x7fff9a714290 ; <+20>
0x7fff9a714288 <+12>: movq %rax, %rdi
0x7fff9a71428b <+15>: jmp 0x7fff9a70fc53 ; cerror_nocancel
0x7fff9a714290 <+20>: retq
(lldb) bt
* thread #1: tid = 0x1091b83, 0x00007fff9a714286 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
* frame #0: 0x00007fff9a714286 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff9a6f942f libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fff9a134b53 libsystem_c.dylib`abort + 129
frame #3: 0x00007fff8d9a61cb libsystem_malloc.dylib`free + 428
frame #4: 0x000000010002e110 libdaalabase.0.dylib`od_state_init(state=<unavailable>, info=<unavailable>) + 3456 at state.c:354
frame #5: 0x0000000100068fa0 libdaaladec.0.dylib`daala_decode_alloc [inlined] od_dec_init + 11 at decode.c:52
frame #6: 0x0000000100068f95 libdaaladec.0.dylib`daala_decode_alloc(info=0x0000000100005448, setup=<unavailable>) + 149 at decode.c:71
frame #7: 0x00000001000029d1 dump_video`main(argc=<unavailable>, argv=<unavailable>) + 5441 at dump_video.c:327
frame #8: 0x00007fff98ef75c9 libdyld.dylib`start + 1
frame #9: 0x00007fff98ef75c9 libdyld.dylib`start + 1
(lldb)