Support benchmark-specific arguments for a fuzzer about fuzzbench HOT 2 OPEN

Hi,

I am reading this https://google.github.io/fuzzbench/getting-started/adding-a-new-fuzzer/ to add AFLSmart (https://github.com/aflsmart/aflsmart) so it can be benchmarked using FuzzBench. However, AFLSmart supports a few more arguments, especially the one taking an input model (-g) to enable its structure-aware capability. I have two questions regarding this

1. How to support this -g argument in FuzzBench?

If this argument were not target/benchmark specific do you think it's unclear from the docs how this can be done? (asking because I want to know if our docs need improvement)

2. Since the input model is target program-specific. For example, LibPNG takes PNG input model, LibJpeg-turbo takes JPEG input model. As far as I understand, FuzzBench uses projects integrated into OSS-Fuzz for benchmarking so where should I store AFLSmart's input models and how to map them to the corresponding OSS-Fuzz projects?

I am thinking about a simple solution which could answer the above questions but I am not sure it is desirable. The solution is that I fix the -g argument with an input model, say "-g model.xml" and when I build a OSS-Fuzz project, I copy a corresponding input model to the working folder and rename it to model.xml.

That should work.
The other possibility is to infer which model to use from the BENCHMARK environment variable, when executing fuzz. I wouldn't say it's official API but I think relying on it is OK.

Does this help?

from fuzzbench.

Hi @jonathanmetzman,

Thank you for your reply. Your suggestions work. I have just sent a pull request (#84) for AFLSmart. I check the value of the BENCHMARK env. variable to decide which input model will be used. However, it works pretty well when the number of benchmarks is small, but it would be an issue when the benchmarks become large, especially when you consider different versions of a subject program as I see in another discussion. So I would like to see a new feature that can handle this in a more proper way. Fortunately, it seems that this feature is on the working list since I see Abhishek has marked my request as a "new feature" -- which is awesome!

Best regards,

Thuan

from fuzzbench.