Bugbench is a benchmark suite created by Shan Lu.
Bug signature example is in this separate file.
- I need to access bugbench many times
- The copy I got needs some modification (compiler flags) to run on Linux
- I need take some notes
- I want to create bug signature along with it
benchmark | deterministic | original benchmark | patched version | manual bug signature | patched version | Comment |
---|---|---|---|---|---|---|
gzip-1.2.4 | Y | Y | N (official patch) | Y | N | |
ncompress-4.2.4 | Y | Y | N (manual patch) | Y | N | |
polymorph-0.4.0 (bug 1) | Y | Y | N (manual patch) | Y | N | |
bc-1.06 (bug 3) | Y | Y | N (manual patch) | N/A | N/A | Too complicated. code is generated by flex and bison |
man-1.5h1 | Y | Y | N (manual patch) | Y | N |
benchmark | deterministic | original benchmark | patched version | manual bug signature | patched version | Comment |
---|---|---|---|---|---|---|
bc-1.06 (bug 1) | N/A | No bug triggering input | ||||
bc-1.06 (bug 2) | N/A | No bug triggering input | ||||
polymorph-0.4.0 (bug 2) | N/A | No bug triggering input | ||||
squid-2.3 | N/A | Complicate to run, don’t know how to start and connect squid server | ||||
cvs-1.11.4 | N/A | Require running cvs server, no exploit-cvs.c file found |
In each benchmark directory, the three folders are added by me:
./*slice.txt
slice performed on the property violation line (the criteria file is./src/slicing-criteria.txt
)./patch
the patch that can fix the bug./helium
the folder containing bug signature../helium/addition
the bug signature created manually./helium/slicing
the bug signature created based on slice
benchmark | LOC | full slice size | data slice | control slice | comment |
---|---|---|---|---|---|
gzip | 5225 | 1982 | 1 | 55 | data slice is actually the criteria itself |
ncompress | 1436 | 450 | 1 | 63 | |
polymorph | 404 | 20 | 1 | 19 | |
man | 3036 | 1992 | 1206 | 90 |
- all statements in manual created bug signature are in the slice
- full slice is much bigger than bug signature
- understand the reasons that full slice can not simply built (see next sub-section)
- It is possible to carefully remove statements not in slice to make slice built, and can trigger the bug.
- The reason for slice to be so big. The reasons can be 1) control slice 2) correct path 3) compute irrelevant results. The first and third reason seem to be primary reasons for these benchmarks.
- slice will not contain the syntax meaningless constructs, like
- parenthesis,
- else clause,
- multi-line statements.
do
while
These hinder building. The use of AST can help this.
- slicing may not include the declaration of a variable, results in compile error.
- typedef is not included in slice
- Some global variables are not in slice, but is used in many places, including some statements in slice.
- if branches contains only one statement, which is also not in slice. Cannot simply delete it
- …