sslab-gatech / hydra Goto Github PK

7. After executing two concurrent tasks for ext4 and f2fs fuzz for about 36 hours on x86_64 server with 282GB memory , ext4-combined-consistency and f2fs-combined-consistency reported segfault,
   and I found there is not enough memory available, tmpfs-separate consumes almost all memory, is this a bug or is it caused by my misuse?

none tmpfs 142G 142G 0 100% /tmp/mosbench/tmpfs-separate/1
none tmpfs 142G 142G 0 100% /tmp/mosbench/tmpfs-separate/4

~/hydra/src$ free -m
total used free shared buff/cache available
Mem: 289419 1510 1006 279783 286903 5485
Swap: 95366 9741 85625

LOVE YOUR WORK. I read the paper and noticed that most of the bugs detected by HYDR were acknowledged or fixed. Did u report the bug to official email account of LINUX or report them on some communities?

Followed instructions in the README. I also tried compiling again with CC=afl-gcc but whenever I run the run.py command and the reset is "No instrumentation detected".

Also unrelated how to add other filesystems or update btrfs.

Tested ext4 36h+ and reported 398 uniq crashes. I doubt this. Is there anything wrong?

 american fuzzy lop 2.52b (fuzzer_ext4-cpu4log4grp4)

┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐
│        run time : 1 days, 12 hrs, 40 min, 3 sec      │  cycles done : 97     │
│   last new path : 0 days, 0 hrs, 24 min, 6 sec       │  total paths : 3658   │
│ last uniq crash : 0 days, 22 hrs, 12 min, 26 sec     │ uniq crashes : 398    │
│  last uniq hang : 1 days, 12 hrs, 10 min, 13 sec     │   uniq hangs : 5      │

I started fuzzing according to the readme, the version of lkl is 5.0.0, but I suspected that kasan was not turned on, so I debugged it with gdb and found that the program did not execute kasan_malloc.
Looking forward to your reply, thanks~

@squizz617 I have few specific question to ask,

1. does hydra able to run external image filesys or its not feasible for that?
2. What are the parameters or code need to modify to run on hydra-fuzzer?

Hi, i encounter a problem, i create a ext4 image file , then call test command , but it fail.

how to create ext4 image: use my script:

#! /bin/bash

# i keep the same file list with sample/oracle/ext4-10.image

set -x

umount /tmp/ext4
rm -rf /tmp/ext4
mkdir /tmp/ext4
rm -f ext4.img

dd if=/dev/zero of=ext4.img bs=4k count=4096
mke2fs -t ext4 -c ext4.img
tune2fs -c0 -i0 ext4.img

mount -t ext4 ./ext4.img /tmp/ext4

cd /tmp/ext4
mkdir foo
mkdir foo/bar
touch foo/bar/baz
ln foo/bar/baz foo/bar/hln
echo "hello world\n" > foo/bar/baz
touch foo/bar/xattr
touch foo/bar/acl
touch foo/bar/æøå
echo "xyz\n" > foo/bar/æøå
#mkfifo foo/bar/fifo
touch foo/bar/fifo
ln -s mnt/foo/bar/baz foo/bar/sln

tree /tmp/ext4

how to test: run below command

# below command is copy from the terminal when i call "run.py ......"
# and i replace the image name to my image

sudo AFL_SKIP_BIN_CHECK=1 ./combined/afl-image-syscall/afl-fuzz -S fuzzer_ext4-cpu1log1grp1 -b shm_ext4-1 -s fs/ext4/ext4_wrapper.so -e ./ext4.img -y seed -i in-ext4-1 -o out-ext4-1 -u 1 -- lkl/tools/lkl/ext4-combined-consistency -t ext4 -i ./ext4.img -e emulator/emulator.py -l /tmp/mosbench/tmpfs-separate/1/log -d "/tmp/mosbench/tmpfs-separate/1/" -r -p @@

the fail message:

terminate called after throwing an instance of 'std::bad_alloc' [cpu001:100%]
what(): std::bad_alloc
Aborted sudo AFL_SKIP_BIN_CHECK=1 ./combined/afl-image-syscall/afl-fuzz -S fuzzer_ext4-cpu1log1grp1 -b shm_ext4-1 -s fs/ext4/ext4_wrapper.so -e ./ext4.img -y seed -i in-ext4-1 -o out-ext4-1 -u 1 -- lkl/tools/lkl/ext4-combined-consistency -t ext4 -i ./ext4.img -e emulator/emulator.py -l /tmp/mosbench/tmpfs-separate/1/log -d "/tmp/mosbench/tmpfs-separate/1/" -r -p @@

my system info:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.6 LTS
Release:	18.04
Codename:	bionic

$ uname -a
Linux ub1804 5.0.0-050000-generic #201903032031 SMP Mon Mar 4 01:33:18 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ gcc -v
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix

my other try:

my ubuntu system have kernel version 4.15 at first, i then upgrade it to 5.0, but same error.
my gcc version have version 4.7 at first, i then replace it to clang, but same error.

i also do a test, i run the test command with default ext4-10.image, it work perfectly, then i mount ext4-10.image, and edit the file fool/bar/baz as follow: delete a charactor, then save the file; then add back the charactor, and save the file , then i rerun the command with ext4-10.image, then it rise Segmentation fault.
it seems that, once the image file edit by my os, even though file content not change, it will rise a error.

what i want:

1. i hope you will help me to fix the issue.
2. i guess the problem is because of my os(kernel, lib version, gcc version...) is different to yours.
   so could you paste your local machine info, include, os release version, gcc version, os kernel version.

Thanks!

while fuzzing ext4 i get this:

and i found some file in src/out-ext4-/fuzzer_ext4-/hangs, but no logs in mosbench/tmpfs-separate/, then how can i debug these hangs?

I tried to install hydra and make install by executing the command make build-btrfs-imgwrp
But it failed and throwed the error message " error: 'FALLOC_FL_COLLAPSE_RANGE' was not declared in this scope". What's the problem here?

Besides, I have installed clang already, with a soft link to the path ..../hydra-master/src/llvm-build/bin/

Is it necessary to adapt lkl to the mainline version, and what changes do I need to make at the same time?

Hello Author,
Any suggestions why the image compression failed occur in hydra, I created the ext4 image using -O casefold feature? Error Message:
[-] image samples/oracle/ext4.image compression failed
Location: compress() ext4_fuzzer.cc:222. @squizz617

make -C afl-syscall
g++ -std=c++11 -g -fPIC -c -o FSCQ-consistency-exec.o FSCQ-consistency-exec.cpp
g++ -std=c++11 -g -fPIC -c -o yxv6-consistency-exec.o yxv6-consistency-exec.cpp
make[2]: Entering directory '/home/m00292095/git/hydra/src/combined'
make[2]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
make[2]: *** afl-syscall: No such file or directory.  Stop.
make[2]: Leaving directory '/home/m00292095/git/hydra/src/combined'
Makefile:33: recipe for target 'afl' failed
make[1]: *** [afl] Error 2
make[1]: *** Waiting for unfinished jobs....

may caused by ed561e5 update Makefile?

git diff combined/Makefile
diff --git a/src/combined/Makefile b/src/combined/Makefile
index 0f4b411..08371f3 100644
--- a/src/combined/Makefile
+++ b/src/combined/Makefile
@@ -30,7 +30,7 @@ yxv6-cc: yxv6-consistency-exec.o Image.o Program.o Utils.o Constants.o
        $(CXX) $(CXXFLAGS) -o $@ $^

 afl:
-       make -C afl-syscall
+       make -C afl-image-syscall

 %.o: %.cpp
        $(CXX) $(CXXFLAGS) -fPIC -c -o $@ $<

When I try to build XFS, I get the following error. No, other FS build throws any error.

NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

which pkg-config
/usr/bin/pkg-config

find /usr -name "pkg.m4"
/usr/share/aclocal/pkg.m4

./configure: line 15293: PKG_PROG_PKG_CONFIG: command not found
./configure: line 15308: syntax error near unexpected token `systemd,'
./configure: line 15308: `			PKG_CHECK_MODULES(systemd, systemd,'
Makefile:115: recipe for target 'include/builddefs' failed
make[2]: *** [include/builddefs] Error 2
make[2]: Leaving directory '/home/ubuntu/hydra/src/fs/xfs/xfsprogs-dev'
Makefile:9: recipe for target 'lib' failed
make[1]: *** [lib] Error 2
make[1]: Leaving directory '/home/ubuntu/hydra/src/fs/xfs'
Makefile:87: recipe for target 'build-xfs-imgwrp' failed
make: *** [build-xfs-imgwrp] Error 2

Hi Seulbae Kim,
I was in the master branch (commit id: fd16457), following the README.md to run the fuzzer, and a day later, the AFL UI informed me that an assertion error had been found and that the test cases were stored in the /tmp/mosbench/tmpfs-separate/10/log directory.
However, I do not have these files in the /tmp/mosbench/tmpfs-separate/ directory. Why is this?
In addition, since I run it as the root user, I should have permission to operate the directory, but I did not see any log files.
Some running screenshots are as follows:
AFL UI results:

The contents of the /tmp/mosbench/tmpfs-separate/directory after the error was found:

The execution result of command sudo ./prepare_fuzzing.sh, according to the README.md:

Dear Author, I am having some error while running the make file, here;s the error command look like any suggestions how to fix it. As I fixed couple of errors that previously found but now this one taking my lot of time to make it work,

make build-xfs-imgwrp Makefile:594: arch/x86/auto.conf: No such file or directory make: *** No rule to make target 'arch/x86/auto.conf'. Stop.

Additionally $ git checkout v4.16-backport this one also does not work, did the branch checkout name correct or its changed??

@tsgates @setuid0x0 @meng-xu

Thank you.

So I was running fuzzer, but I am not getting the GUI interface of it, I did steps 1-4 correctly without an error. @squizz617 Thanks.

3. When testing xfs, AFL stuck during afl-fuzz.c: main-->load_seed_image-->wrapper_compress-->dlsym, refers to xfs_fuzzer :: compress function and added debugg printfs can not help me to resolve the problem.There are no similar problems in the other 3 filesystems.

~/hydra/src$ ./run_new.py -t xfs -c 5 -l 5 -g 5 -i xfs-00
afl-fuzz 2.52b by <[email protected]>
[+] [fs-fuzz] shm name to store image buffer: shm_xfs-5
[+] [fs-fuzz] target wrapper (.so) path: fs/xfs/xfs_wrapper.so
[+] [fs-fuzz] seed image path: samples/oracle/xfs-00.image
[+] [fs-fuzz] syscall input directory: seed_xfs-00
[+] You have 32 CPU cores and 11 runnable tasks (utilization: 34%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[+] Found a free CPU core, binding to #5.
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[+] [+] Open shm shm_xfs-5 success.
[+] [+] Map shm shm_xfs-5 at 0x7f93d09b7000 size: 0x1000000.

[!] WARNING: [D] start alloc_printf...
[!] WARNING: [D] start wrapper_compress start ...
[!] WARNING: [D] seed_file -- samples/oracle/xfs-00.image

In EXPERIMENTS.md it says to run setup_logicbug.sh.
I checked out the logicbug branch and

$ find . -name setup_logicbug.sh

returns nothing. Is the file under a different name?
how do i instrument this fuzzer for other filesystems?

Hi,

I tested f2fs and btrfs exactly as mentioned in https://github.com/sslab-gatech/hydra/blob/master/README.md .
All works good except:

 $ sudo ./prepare_fuzzing.sh
returns:
tee: 'cpu*/cpufreq/scaling_governor': No such file or directory
performance

I have found no bugs for btrfs and f2fs.

The command i used for testing btrfs:
$ ./run.py -t btrfs -c 4 -l 10 -g 1

The command i used for testing f2fs:
$ ./run.py -t f2fs -c 4 -l 10 -g 1

Could please help me on this issue, thank you.

Kind Regards,
Jiyang

fixed

6. How to integrate SibylFS detector for Specification violation into Hydra framework? I did not find the relevant instructions

The instructions seem geared towards traditional filesystems like ext4 and btrfs. Can you provide instructions for FUSE filesystems, e.g., s3fs? The paper suggests that this is possible.

4. Hydra has supported ext4,btrfs,f2fs and xfs, can it be extended to other filesystems, such as ubifs, nfs, cifs, etc?

5. How are hydra/src/samples/oracle/*. image generated, what are the difference between * -00.image and * -10.image, can images of other filesystems be made in the same way?

Hi Seulbae Kim,
I’m using hydra (based on commit id: e7f0c5f) for Linux-4.19 FS fuzzing. I encountered the following problems during the test, hope you can give pointers:

1. In EXPERIMENTS.md, you mentioned "Test cases that trigger crash consistency bugs are stored under the specified log directory.",
   Is the ‘log directory’ refers to /tmp/mosbench/tmpfs-separate/4/log as below, and if so, I got crashes reported in ALF UI, but there is no .c exists?
   Current, my approach is referring to Janus's utils/afl-parse tool for out-ext4-1/fuzzer_ext4-cpu1log1grp1/crashes/id: 000000, sig: 12, src: 000000, op: fs-havoc-generate, rep: 32,
   after parsing, I got three files, .c/.c.raw/.img. Am I doing this correctly?

~/hydra/src$ cat out-ext4-4/fuzzer_ext4-cpu4log4grp4/crashes/README.txt
Command line used to find this crash:

./combined/afl-image-syscall/afl-fuzz -S fuzzer_ext4-cpu4log4grp4 -b shm_ext4-4 -s fs/ext4/ext4_wrapper.so -e samples/oracle/ext4-10.image -y seed_ext4-10 -i in-ext4-4 -o out-ext4-4 -u 4 -- lkl/tools/lkl/ext4-combined-consistency -t ext4 -i samples/oracle/ext4-10.image -e emulator/emulator.py -l /tmp/mosbench/tmpfs-separate/4/log -d /tmp/mosbench/tmpfs-separate/4/ -r -p @@

~/hydra/src$ ./utils/afl-parse_janus -i samples/oracle/ext4-10.image -t ext4 -f out-ext4-4/fuzzer_ext4-cpu4log4grp4/crashes/id\:000000\,sig\:12\,src\:000002\,op\:fs-havoc-generate\,rep\:64 -o poc_id\:000000

output 3 files as below:
poc_id:000000.c
poc_id:000000.c.raw
poc_id:000000.img