Hi @joxeankoret

There was a case the fuzzer stopped working with error: Error: [Errno 10054] An existing connection was forcibly closed by the remote host.

I believe that this was cased by fuzzer trying to get the buffer from the queue and something wrong happened.

OS: Windows 7 SP1 x86

[Tue May 24 13:34:17 2016 1960:3252] Recording a total of 10 value(s) of coverage...
Traceback (most recent call last):
  File "bcf.py", line 683, in <module>
    main(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5])
  File "bcf.py", line 662, in main
    buf = fuzzer.fuzz(input_file, output, max_iterations)
  File "bcf.py", line 613, in fuzz
    self.record_metrics(input_file)
  File "bcf.py", line 260, in record_metrics
    p.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 419, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\multiprocessing\forking.py", line 67, in dispatcher
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 401, in save_reduce
    save(args)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 548, in save_tuple
    save(element)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 725, in save_inst
    save(stuff)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "C:\Python27\lib\multiprocessing\managers.py", line 484, in __reduce__
    return type(self).from_address, \
AttributeError: type object 'SyncManager' has no attribute 'from_address'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\lib\pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "C:\Python27\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python27\lib\pickle.py", line 880, in load_eof
    raise EOFError
EOFError

I didn't find any documentation related to how to configure fuzzer in distributed systems. I want to use nightmare fuzzer in more than one system(distributed system).

But it's boring as hell doing it. Volunteers?

gdb_iface.py:

288         crash_data.add_data("disassembly", int(self.pc), self.disasm)
289         for dis in self.disasm_around:
290           if type(dis[0]) is int or dis[0].isdigit():
291             crash_data.add_data("disassembly", dis[0], dis[1])

"type(dis[0])" also can be 'long', so if is true, we get exception.

patch is:

type(dis[0]) in (int, long)

fixed using

$ beanstalkd -z 55000000

which is not really obvious and its not documented in install.txt

Hi, I want to fuzzing antivirus on windows 7, have any tutorials inthere?

Please change
crash = self.iface.main(" ".join(cmd))
to
crash = self.iface.main(cmd)
in generic_fuzzer.py line 176.

This is causing CreateProcess failed when not using 'asan' as debugging interface.

Thanks!

BCCF configuration allows to generate coverage metrics based on non-unique basic blocks. However, this setting only has effect when generating initial metrics, but not when fuzzing cycles are executed:

https://github.com/joxeankoret/nightmare/blob/master/fuzzers/bcf.py#L264
https://github.com/joxeankoret/nightmare/blob/master/fuzzers/bcf.py#L498

This means that practically no fuzzing round will ever be able to produce better coverage than the inital ones.

The following commit fixes this with a trivial patch, but my git-fu isn't strong enough today to create a proper pull request:

v-p-b@daea994

Hi @joxeankoret

I noticed a critical bug in nightmare Hide Duplicates functionality.

When you do Hide Duplicates, some of the unique crashes also does not appear.

This is obviously an issue with the SQL query. I'm busy with some other stuff so did not find time to find the root cause.

Thanks.

./bcf.py

Traceback (most recent call last):
File "./bcf.py", line 31, in
from nfp_log import log, debug
File "/opt/nightmare/fuzzers/../runtime/nfp_log.py", line 13, in
from config import DEBUG
ImportError: cannot import name DEBUG

When the line from config import DEBUG is commented out:
Traceback (most recent call last):
File "./bcf.py", line 684, in
main(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5], int(sys.argv[6]))
File "./bcf.py", line 660, in main
fuzzer = CBlindCoverageFuzzer(arch, cfg, section)
File "./bcf.py", line 50, in init
self.read_configuration()
File "./bcf.py", line 93, in read_configuration
self.read_bininst_configuration(parser)
File "./bcf.py", line 115, in read_bininst_configuration
raise Exception("Binary instrumentation toolkit parameter bininst-tool does not exists in the given configuration file")
Exception: Binary instrumentation toolkit parameter bininst-tool does not exists in the given configuration file

I correctly configured the bininst-tool parameter to point to dynamario folder but it still gives the above error.

Hi @joxeankoret

I kept the test VM running with generic_fuzzer.py and every morning I find that the application hangs and it's not killed by the Timer thread.

https://github.com/joxeankoret/nightmare/blob/master/lib/interfaces/pykd_iface.py#L226

What I feel that pykd.breakin() is not sufficient to kill the process. I have added pykd.killAllProcesses() and hoping that it fixes the issue.

Same thing with https://github.com/joxeankoret/nightmare/blob/master/lib/interfaces/pykd_iface.py#L239

I will update you with my observation.

Thanks.

Implement triggers for each bug and for each crash to do specific actions for each one.

looks like generic_fuzzer.py is leaking fds somehow

[Tue Nov  4 23:28:23 2014 25176:139983205721856] Exception: [Errno 24] Too many open files: '/proc/31950/task'
Error: [Errno 24] Too many open files: '/proc/31950/task'

Why have README README.txt README.md LICENSE LICENSE.txt TODO TODO.txt... choose a single extension and use it, but dont confuse your users please

With the latest pykd (0.3.1.2) i get the following error running generic_fuzzer.py on Windows

[Thu May 12 17:05:07 2016 1648:1752] Exception: Python argument types in
    pykd.pykd.startProcess(str)
did not match C++ signature:
    startProcess(class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > commandline)
    startProcess(class std::basic_string<wchar_t,struct std::char_traits<wchar_t>,class std::allocator<wchar_t> > commandline, unsigned long debugOptions)

I think something must have changed in pykd startProcess() function, as the problem seems to be at pykd_iface.py:248
self.id = pykd.startProcess(self.program, debugChildren=True)

removing debugChildren=True seems to fix this. I need debugChildren but I'm very lazy, so I just downloaded pykd 0.3.0.38 and I'm now happy again, but there's probably a different way to pass that argument now.

but doesn't specifies where.

I would find far more readable to have a shellscript to make all those steps with few config vars than the whole text file. it will make the process a bit less nightmarish, but i guess users would be happy with that

to avoid loading deprecated or corrupted databases from older sources

I will be changing the license of this project to the GNU Affero GPL 3.0. While it means no change for 99,99% of users, I would like to know if you have a strong opinion against the change.

It would be great to create a docker image pre-configured so users doesn't need to go through the "nightmare" (@pancake dixit) of installing it. Any volunteers?

it used to be far more effective than dumb fuzzers to find crashes in ELF bins

not all distros have this pkg, so better to install and run from sauce. as well as all the deps. the install script can install all this stuff automatically in the ./prefix directory and avoid trashing the system directories

I have investigated a bit on this and realized that those are from a something named "python". I guess that's a bug and you should fix that asap.

thanks

in the generation script for example.. otherwise if yuu get infinite backtraces because of a configuration error like (BIGJOB) you cant stop the mad log to flood your terminal and requires intervention of the ^Z and kill %% Lords to solve the conflict.

No var to configure this

On line https://github.com/joxeankoret/nightmare/blob/master/lib/interfaces/pykd_iface.py#L320

There is a call missing to kill the process after the exception has occurred.

This can be fixed by adding pykd.killAllProcesses() after L320

and only the python code fails tries to use a var named "maximum" as a string with value "" instead of a numeric value. which produces a backtrace crash

The directory lib/lib/interfaces/ seems to be a duplicate of lib/interfaces/, and can be deleted?

Also, now pydistorm.py can be removed?

Thanks.

distorm is ok, but kinda outdated. tries to use python2.5. i guess that switching to capstone would be the best way

tarakogayer is probably offensive

Hi,

I created small vulnerable application, project in NFM, choosen radamsa as fuzzer, started beanstalkd, nfp_engine and generic_fuzzer. Then - it started working as normal, but after catching first crash got this message:

[Tue Oct 28 03:56:44 2014 15042:47990707519296] Launching debugger with command /tmp/test /tmp/tmpoMqWYU.test
Parsing /tmp/tmpoMqWYU.test
Format string hehe
[17576] - /usr/bin/python2.7 0xb
generic_fuzzer.py generic.cfg test
Process 17576 crashed with signal 11 (SIGSEGV)
rax 0x2b1d897451b0L 'H=\x01\xf0\xff\xffs1\xc3H\x83\xec\x08\xe8\xfe\x89\x01\x00H\x89\x04$\xb8\x01\x00\x00\x00\x0f\x05H\x8b<'
rbx 0x2b1d899fa7a0L '\x84*\xad\xfb\x00\x00\x00\x00\x00PG\x89\x1d+\x00\x00\x00PG\x89\x1d+\x00\x00\x00PG\x89\x1d+\x00\x00'
rcx 0x6eL
rdx 0x20L
rsi 0x7fff271b1d68L "n\x00\x1b'\xff\x7f\x00\x00\xdf3\x1b'\xff\x7f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf33\x1b'\xff\x7f\x00\x00"
rdi 0x7fff271b1a28L " Eg\x89\x1d+\x00\x000\xc9l\x89\x1d+\x00\x00\xa0\xa7\x9f\x89\x1d+\x00\x00\xc8\x1b\x1b'\xff\x7f\x00\x00"
rbp 0x7fff271b1b60L "\x80\x1c\x1b'\xff\x7f\x00\x00z4l\x89\x1d+\x00\x00 \x00\x00\x000\x00\x00\x00P\x1c\x1b'\xff\x7f\x00\x00"
rsp 0x7fff271b14f0L '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
rip 0x2b1d896bb85dL 'D\x89 \xe9\xbd\xe7\xff\xffI\x8bE\x08H\x8dP\x08I\x89U\x08\xeb\xb1\x83\xbd8\xfa\xff\xff\x00u\A'

00002b1d896bb85d MOV [RAX], R12D <--------- CRASH
Yep, we got a crash! \o/

[Tue Oct 28 03:56:45 2014 15042:47990707519296] We have a crash, moving to test-crash queue...
[Tue Oct 28 03:56:45 2014 15042:47990707519296] $PC 0x2b1d896bb85d Signal SIGSEGV Exploitable Unknown
[Tue Oct 28 03:56:45 2014 15042:47990707519296] 2b1d896bb85d: MOV [RAX], R12D
[Tue Oct 28 03:56:45 2014 15042:47990707519296] Launching debugger with command /tmp/test /tmp/tmpkdjmvj.test
[Tue Oct 28 03:56:45 2014 15042:47990707519296] Exception: ERROR - Must be attached to a process
Error: ERROR - Must be attached to a process
root@c2227:/home/zoczus/nightmare/fuzzers#

The following commit intends to fix out-of bounds indexing bugs, but the solution is incomplete as it can cause infinite loops:

v-p-b@2e41db1