On the latest master enabling the SYSCALLS plugin makes DRAKVUF stuck after displaying

It works on my machine (see <a href="http://ci.drakvuf.com:8080/job/DRAKVUF/21/console

Infinite loop in drakvuf_get_symbols_from_rekall about drakvuf HOT 25 CLOSED

tklengyel commented on August 15, 2024

Infinite loop in drakvuf_get_symbols_from_rekall

from drakvuf.

Comments (25)

tklengyel commented on August 15, 2024

It works on my machine (see http://ci.drakvuf.com:8080/job/DRAKVUF/21/console).. How much memory did you give the VM?

from drakvuf.

v-p-b commented on August 15, 2024

The VM has 3G of memory. Can you estimate how much time does it usually take for you to have all the traps set for SYSCALLS?

from drakvuf.

tklengyel commented on August 15, 2024

Then I'm not sure what could be the issue, I'm unable to reproduce it. It seems like libdrakvuf init finished successfully but then in the drakvuf_loop something goes wrong. Can you enable the debug output?

from drakvuf.

v-p-b commented on August 15, 2024

Yes, the above output is the full result I get in debug mode.

from drakvuf.

tklengyel commented on August 15, 2024

That's very odd because that means that you didn't even reach drakvuf_loop (that should be printing Started DRAKVUF loop in debug mode). So the only thing in between drakvuf_init and the loop is plugin init. Do you have any custom plugin changes? If not, try to clean the entire project before building (git clean -xdf; git reset --hard) and recompile.

from drakvuf.

v-p-b commented on August 15, 2024

That didn't help but this must be some residural code indeed: EXMON is not enabled but DRAKVUF still tries to resolve _KTRAP_FRAME.

I noticed that after upgrading from the C version to C++ some configs remained in the .deps/ directories referencing old code - maybe this problem is related. Do these dirs belong to autoreconf? How do I properly remove them (make clean didn't help)?

from drakvuf.

tklengyel commented on August 15, 2024

git clean -xdf should remove all files that are not part of the original source distribution, including .deps

from drakvuf.

v-p-b commented on August 15, 2024

Just realized that vmi.h also references _KTRAP_FRAME it's probably not the error of EXMON that I see...

from drakvuf.

tklengyel commented on August 15, 2024

You can always just clone a fresh copy and try it with that just to make sure.

from drakvuf.

v-p-b commented on August 15, 2024

Did that, same result :(

from drakvuf.

tklengyel commented on August 15, 2024

The only thing I could say is to try to add some printf's to see how far the system gets and maybe try to locate where it gets stuck in a loop. Or try reverting to an older revision and see if the issue is still present.

from drakvuf.

v-p-b commented on August 15, 2024

Thanks, I'm rebooting now, that can fix a lot of things :)

from drakvuf.

v-p-b commented on August 15, 2024

Looks like drakvuf_get_symbols_from_rekall is stuck in an infinite loop

from drakvuf.

tklengyel commented on August 15, 2024

Interesting, not sure why that would happen. Is the Rekall profile correct? I also made a branch called json_safety, can you give that a spin?

from drakvuf.

v-p-b commented on August 15, 2024

Testing with a freshliy fetched Rekall profile the json_safety branch results in this:

DRAKVUF v0.2
Init VMI on domID 12 -> testx86
Rekall profile: '_KTRAP_FRAME' has no 'Rip' member
Failed to find offset for _KTRAP_FRAME:Rip
Reservation increased? 0 with new gfn: 0x29d2d4
Xen altp2m view created with idx: 1 idr: 2
Inject traps in module list of [4]: System
Plugin 0 startup failed!
close_vmi_drakvuf finished
Failed to initialize DRAKVUF (-2)!

PDB GUID: 5b308b4ed6464159b87117c711e7340c2

from drakvuf.

tklengyel commented on August 15, 2024

Can you attach the rekall profile here?

from drakvuf.

v-p-b commented on August 15, 2024

testx86_2.rekall.txt

Edit: according to Python simplejson.tool it is correct JSON

from drakvuf.

v-p-b commented on August 15, 2024

For the record: Other plugins work just fine, SYSCALLS fails in the same way on a different 64-bit Win7 guest.

from drakvuf.

tklengyel commented on August 15, 2024

I don't see anything wrong with the Rekall profile so I'm really not sure why it would fail. It really is just looking for $FUNCTIONS that start with "Nt". Can you try to add some extra printf's to drakvuf_get_symbols_from_rekall to see what functions it finds?

from drakvuf.

v-p-b commented on August 15, 2024

I did this to drakvuf_get_symbols_from_rekall:

    ret->symbols = g_malloc0(sizeof(symbols_t) * ret->count);

    struct json_object_iterator it = json_object_iter_begin(functions);
    struct json_object_iterator itEnd = json_object_iter_end(functions);
    uint32_t i=0;
    printf("ret->count %d\n",ret->count); // <==

    while (!json_object_iter_equal(&it, &itEnd) && i < ret->count) {
        ret->symbols[i].name = g_strdup(json_object_iter_peek_name(&it));
        printf("name: %s\n",ret->symbols[i].name); // <==
        ret->symbols[i].rva = json_object_get_int64(json_object_iter_peek_value(&it));
        i++;
        json_object_iter_next(&it);
    }

Result is that the count is set to 11590 (seems correct), but the extracted name is always "ADD_MAP_REGISTERS" (the first element of $FUNCTIONS in the profile). It looks like json_object_iter_next() doesn't step the iterator.

from drakvuf.

v-p-b commented on August 15, 2024

I guess I've found it:

# ldd drakvuf
        linux-vdso.so.1 =>  (0x00007ffdd9721000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3c5d80e000)
        libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f3c5d602000)
        libxenstore.so.3.0 => /usr/local/lib/libxenstore.so.3.0 (0x00007f3c5d3f8000)
        libxenctrl.so.4.6 => /usr/local/lib/libxenctrl.so.4.6 (0x00007f3c5d1c9000)
        libxenlight.so.4.6 => /usr/local/lib/libxenlight.so.4.6 (0x00007f3c5cf38000)
        libjson-c.so.2 => /lib/x86_64-linux-gnu/libjson-c.so.2 (0x00007f3c5cd2d000)
        libvmi-0.11.so => /usr/local/lib/libvmi-0.11.so (0x00007f3c5cb09000)
        libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f3c5c801000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3c5c5e3000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3c5c2df000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3c5c0c9000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3c5bd04000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3c5bb00000)
        libxenguest.so.4.6 => /usr/local/lib/libxenguest.so.4.6 (0x00007f3c5b8d6000)
        libblktapctl.so.1.0 => /usr/local/lib/libblktapctl.so.1.0 (0x00007f3c5b6ce000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f3c5b4cb000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f3c5b2c6000)
        libyajl.so.2 => /usr/lib/x86_64-linux-gnu/libyajl.so.2 (0x00007f3c5b0bd000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f3c5ae7f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3c5dd25000)
        libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f3c5ac6f000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3c5aa56000)

Note that the drakvuf binary is to load the now obsolete jansson lib! I still had that installed so I removed it and gave it another try, now I get this on make:

libtool: link: gcc -std=gnu99 -g -O2 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -g -O2 -o dirwatch dirwatch.o  ../xen_helper/.libs/libxenhelper.a -ljansson -lxenstore -lm -lxenctrl -lxenlight -ljson-c /usr/local/lib/libvmi.so -lglib-2.0 -lpthread
/usr/bin/ld: cannot find -ljansson
collect2: error: ld returned 1 exit status

So it seems that some parts of the code somehow still link to jansson and I bet that there is a name conflict in the iterator functions!

from drakvuf.

tklengyel commented on August 15, 2024

Something is very strange on your system, the build system should not be looking for jansson at all. Did you run ./autogen.sh before building? This is what ldd shows on my system:

ldd /usr/local/bin/drakvuf 
    linux-vdso.so.1 (0x00007ffc3d9a6000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f19dbdc4000)
    libxenstore.so.3.0 => /usr/local/lib/libxenstore.so.3.0 (0x00007f19dbbbb000)
    libxentoollog.so.1 => /usr/local/lib/libxentoollog.so.1 (0x00007f19db9b8000)
    libxenctrl.so.4.6 => /usr/local/lib/libxenctrl.so.4.6 (0x00007f19db78b000)
    libxenlight.so.4.6 => /usr/local/lib/libxenlight.so.4.6 (0x00007f19db4fb000)
    libjson-c.so.2 => /lib/x86_64-linux-gnu/libjson-c.so.2 (0x00007f19db2ef000)
    libvmi-0.11.so => /usr/local/lib/libvmi-0.11.so (0x00007f19db0ca000)
    libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f19dadbb000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f19dab9d000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f19da822000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f19da60c000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f19da262000)
    libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007f19da1df000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f19d9fdb000)
    libxenguest.so.4.6 => /usr/local/lib/libxenguest.so.4.6 (0x00007f19d9db1000)
    libblktapctl.so.1.0 => /usr/local/lib/libblktapctl.so.1.0 (0x00007f19d9baa000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f19d99a7000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f19d97a1000)
    libyajl.so.2 => /usr/lib/x86_64-linux-gnu/libyajl.so.2 (0x00007f19d9596000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f19d9326000)
    /lib64/ld-linux-x86-64.so.2 (0x00005589ef54f000)
    libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f19d9103000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f19d8efb000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f19d8cd8000)
    libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f19d89f6000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f19d87e6000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f19d85cb000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f19d83b6000)```

from drakvuf.

v-p-b commented on August 15, 2024

Yes I did. I also looked for the "jansson" string in every file after every build step:

autogen.sh => nothing (except travis config)
configure => nothing
make => fail, and some binaries have references to jansson

Then I realized I only removed the libjansson-dev package, so I removed libjansson4 too. Now configure fails b/c:

configure: error: LibVMI doesn't have events support.

I mean, WTF?!

from drakvuf.

v-p-b commented on August 15, 2024

OK, I reinstalled LibVMI, now everything seems to work fine.

I still don't understand how this whole thing happened, but I suspect I missed one of the changed installation steps during an upgrade.

I was thinking about creating some script that automatically complies a TODO list every time drakvuf.com changes, but I couldn't figure out a proper way yet...

Anyway, thank you for your support!

from drakvuf.

tklengyel commented on August 15, 2024

LibVMI also switched to libjson-c recently but it should still have worked with an older version. But apparently the two libs conflict in some weird ways..

from drakvuf.

Infinite loop in drakvuf_get_symbols_from_rekall about drakvuf HOT 25 CLOSED

Comments (25)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent