Giter VIP home page Giter VIP logo

bugbench's Introduction

My Copy of Bugbench

Bugbench is a benchmark suite created by Shan Lu.

Bug signature example is in this separate file.

Why creating this repo?

  • I need to access bugbench many times
  • The copy I got needs some modification (compiler flags) to run on Linux
  • I need take some notes
  • I want to create bug signature along with it

Statistics

benchmarkdeterministicoriginal benchmarkpatched versionmanual bug signaturepatched versionComment
gzip-1.2.4YYN (official patch)YN
ncompress-4.2.4YYN (manual patch)YN
polymorph-0.4.0 (bug 1)YYN (manual patch)YN
bc-1.06 (bug 3)YYN (manual patch)N/AN/AToo complicated. code is generated by flex and bison
man-1.5h1YYN (manual patch)YN
benchmarkdeterministicoriginal benchmarkpatched versionmanual bug signaturepatched versionComment
bc-1.06 (bug 1)N/ANo bug triggering input
bc-1.06 (bug 2)N/ANo bug triggering input
polymorph-0.4.0 (bug 2)N/ANo bug triggering input
squid-2.3N/AComplicate to run, don’t know how to start and connect squid server
cvs-1.11.4N/ARequire running cvs server, no exploit-cvs.c file found

Directory layout

In each benchmark directory, the three folders are added by me:

  • ./*slice.txt slice performed on the property violation line (the criteria file is ./src/slicing-criteria.txt)
  • ./patch the patch that can fix the bug
  • ./helium the folder containing bug signature.
  • ./helium/addition the bug signature created manually
  • ./helium/slicing the bug signature created based on slice

Slicing experiment

Overview

benchmarkLOCfull slice sizedata slicecontrol slicecomment
gzip52251982155data slice is actually the criteria itself
ncompress1436450163
polymorph40420119
man30361992120690

Conclusion from experiment

  1. all statements in manual created bug signature are in the slice
  2. full slice is much bigger than bug signature
  3. understand the reasons that full slice can not simply built (see next sub-section)
  4. It is possible to carefully remove statements not in slice to make slice built, and can trigger the bug.
  5. The reason for slice to be so big. The reasons can be 1) control slice 2) correct path 3) compute irrelevant results. The first and third reason seem to be primary reasons for these benchmarks.

Reasons for slice not built

  1. slice will not contain the syntax meaningless constructs, like
    • parenthesis,
    • else clause,
    • multi-line statements.
    • do while

    These hinder building. The use of AST can help this.

  2. slicing may not include the declaration of a variable, results in compile error.
  3. typedef is not included in slice
  4. Some global variables are not in slice, but is used in many places, including some statements in slice.
  5. if branches contains only one statement, which is also not in slice. Cannot simply delete it

bugbench's People

Contributors

lihebi avatar

Stargazers

 avatar  avatar Beichen Liu avatar Shamiul Hasan avatar

Watchers

James Cloos avatar

Forkers

pushkarkishore

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.