Giter VIP home page Giter VIP logo

nitro's Introduction


KVM-VMI

KVM-based Virtual Machine Instrospection.

Slack

Table of Contents

Overview

This project adds virtual machine introspection to the KVM hypervisor.

Virtual Machine Introspection is a technology that aims to understand the guest's execution context, solely based on the VM's hardware state, for various purposes:

  • Debugging
  • Malware Analysis
  • Live-Memory Analysis
  • OS Hardening
  • Monitoring
  • Fuzzing

See the presentations section for more information.

This project is divided into 4 components:

  • kvm: linux kernel with vmi patches for KVM
  • qemu: patched to allow introspection
  • nitro (legacy): userland library which receives events, introspects the virtual machine state, and fills the semantic gap
  • libvmi: virtual machine instrospection library with unified API across Xen and KVM

At the moment, 2 versions of VMI patches are available for QEMU/KVM in this repository:

Installation

Follow the Setup guide

Presentations

References

The legacy VMI system contained in this repo (Nitro) is based on Jonas Pfoh's work:

Maintainers

@Wenzel

License

GNU General Public License v3.0

nitro's People

Contributors

pfohjo avatar soft avatar wenzel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nitro's Issues

Nitro get event ioctl is hanging if vm crashes

The procedure for running Nitro is the following:

  1. start VM
  2. wait for desktop
  3. run Nitro
  4. stop Nitro
  5. stop VM

At the moment there is not timeout on the get_event ioctl, so it will wait for the next event available, in the kernel, forever if the vm crashes.

Futhermore, the python threads are blocked until the process is killed because of that.

We need to change the way get_event works, and add a timeout on the kernel side if there is no event available, and return to the userland.

Configuration Format

Today, Nitro's back ends rely on libvmi configuration files being present. The configuration file paths are not specified explicitly but instead the config file is found by checking a list of predetermined locations dictated by libvmi's code base. I think there are certain problems with this approach.

First of, if you think Nitro as a library, it can lead to surprising results. Whether or not your Nitro-based application will work depends on some external config files being present in the file system. For a library, I think it makes more sense to be programmatically configurable. Maybe we could allow passing the libvmi configuration as a dictionary to the Nitro object or something. The libvmi API's allow this we just do not make use of them currently.

Second, for Nitro as an command line application, I think having the user to create a config file, not for Nitro but one of Nitro's dependencies is maybe not the most user friendly thing. Basically, we are extending Nitro's user interface to some of our (somewhat internal) dependencies. Maybe this is mostly a cosmetic thing since we would still require the same information. Additionally, I think it would be nice if the command line application allowed specifying the configuration file to be used explicitly (a --conf argument or something). I think having this around would create less confusion about which config file is used.

Third, we currently do not have nice way for configuring back ends in general. The back end factory automatically initializes back end objects. I think it would be nice if it could pass them some arbitrary back end dependent configuration data. For example, back ends could use these custom initialization parameters to disable some of the extractors.

So what I am proposing is:

  1. Make Nitro not use libvmi's global configuration files. Have back end factory provide programmatic interface for libvmi's initialization. This could be something as simple as having get_backend take another dict that contained the data required by libvmi.
  2. Make the command line interface support specifying a Nitro specific config file. Data-wise, this config file could be superset of libvmi's configuration. It would include all the data required by libvmi + some additional back end dependent configuration. I think having the ability to configure back ends for specific use cases would make them more flexible and lessen the need to find the right balance on what to extract.

I think this would not be huge change nor that much work. What do you think? Should I implement something like this?

Invalid Qemu command line argument definitions in domain template

Are you sure these are correct? At least my Qemu instance complained about unknown command line flags. I think the correct from is to split the -object, -chardev, and -accel flags into their own qemu:arg instances.

<qemu:arg value='-chardev socket,path=/tmp/introspector,id=chardev0'/>
<qemu:arg value='-object secret,id=key0,data=some'/>
<qemu:arg value='-object introspection,id=kvmi,chardev=chardev0,key=key0'/>
<qemu:arg value='-accel kvm,introspection=kvmi'/>

Finding process in kernel mode

Hi @Wenzel ,
First of all, I apologize for asking question(not submitting issue).
Since you are professional in Windows structs, do you know a way in windows to find all running process in kernel mode?
Now, I traverse EPROCESS then get KPROCESS and then go through the KTHREADs. However, I feel it is not the correct way to do so.

Unit Testing

Currently we have basic tests for backends in place. However, actually running these tests is arguably cumbersome as it requires a lot of setup, you have to have the patched kernel in place etc. It would be nice if we had some unit testing that maybe validated backend logic without actually requiring all the difficult-to-setup dependencies. This would make it easier to automatically run tests on the committed code. For this to work, we would likely have to mock some of the libraries.

I created this issue to track the progress of our efforts in testing Nitro.

Backend dispatch_hook exception handling

Currently, the code in Backend dispatch_hook is kind of messy. I guess it made sense when there was only a single back end. To be more specific, I would like for us to decide what should be done about exceptions. Currently we simply catch all the exceptions and log them but I wonder if this is the right approach to take. Previously, the method also caught some back end specific errors but that was lost with the move to multiple back ends.

What are your thoughts on the matter?

Questions about realization of nitro!

First,you said nitro start with handle_exception,i want to know when and where vm tricked into kvm.
Second,in the process of handling exception,nitro execute emulate_instruction,there have two definitions of emulate_instruction,one in arch/powerpc/kernel/traps.c and the second in arch/x86/include/asm/kvm_host.h,which one is correct?
Last,i want to know what's the connection between emulate_instruction in vmx.c and em_syscall in emulate.c,where the em_syscall be executed?I want to know more about emulate instruction,but there is very little information about it on the internet,could you give me some guide about it?Thanks.

Improving the test suite

Our test suite can be improved.
The current design has some limitations:

Problem 1: API

To run a test, you have to use the VMTest class interface provided and run your test like the following:

        def enter_NtOpenKey(syscall):
            pass

        def enter_NtCreateKey(syscall):
            pass

        hooks = {
            'NtOpenKey': enter_NtOpenKey,
            'NtCreateKey': enter_NtCreateKey,
        }
        events, exec_time = self.vm_test.run(cdrom_iso, hooks=hooks)

This forces you to use callbacks with all the problems that goes with it.
For example it's impossible to maintain a state accross multiple callbacks.
-> What if i want to get some information from one callback and use in another one ?

The solution to this would be to let the developer use the Nitro API,and provide VMTest class just as a convenient wrapper.

Nitro API:

with Backend(domain, analyze_enabled) as backend:
        backend.nitro.set_traps(True)
        for event in backend.nitro.listen():
              syscall = backend.process_event(event)
              if syscall.name == 'NtOpenFile':
                  # test

Problem 2 : Configurability

test_nitro.py has been written just to test Windows_7_x64 and nothing more.

As a result, the vm name is hardcoded in the setUp:

class TestNitro(unittest.TestCase):

    def setUp(self):
        con = libvirt.open('qemu:///system')
        domain = con.lookupByName('nitro_win7x64')
        self.vm_test = VMTest(domain)
        self.cdrom = CDROM()
        # clean old test directory
        test_dir_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), self._testMethodName)
        shutil.rmtree(test_dir_path, ignore_errors=True)
        os.makedirs(test_dir_path, exist_ok=True)
        self.script_dir = os.path.dirname(os.path.realpath(__file__))
        # chdir into this directory for the test
        self.origin_wd = os.getcwd()
        os.chdir(test_dir_path)
        # create logging file handler
        self.f_handler = logging.FileHandler('test.log', mode='w')
        logging.getLogger().addHandler(self.f_handler)
        logging.info('Starting test at {}'.format(datetime.datetime.now()))

As we want to make tests on Windows 7 32 bits, as well as Windows 8, Windows 10 and for Linux, we have to find a may to make it configurable

Problem 3 : Split the tests

The test have only been designed for windows_7_x64 and as such, some of them will not run on a Windows_7_x86.
On the other hand, some tests are generic enough to be run accross all Windows OS, and even on Linux

We have to group the tests that are generic enough and split the other in separate test modules, that will be called according to the configuration specified (either command line or a config file)

what's the exactly combination to use nitro?

To repeat nitro, I've installed an old version of libvmi, nitro-kmod_build, and nitro. Everything works, except that no events are captured as described.
This is what I get while running

 ./main.py ubuntu18.04 
Finding QEMU pid for domain ubuntu18.04
Detected 1 VCPUs
Start listening on VCPU 0

If using the nitro version of libvmi, following problems encounterd:

sudo  ./main.py ubuntu18.04
Finding QEMU pid for domain ubuntu18.04
Detected 1 VCPUs
VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
Traceback (most recent call last):
  File "./main.py", line 90, in <module>
    main()
  File "./main.py", line 86, in main
    runner.run()
  File "./main.py", line 48, in run
    self.nitro = Nitro(self.domain, self.analyze_enabled)
  File "/home/mininet/nfs/nitro/nitro/nitro.py", line 11, in __init__
    self.backend = get_backend(domain, self.listener, syscall_filtering)
  File "/home/mininet/nfs/nitro/nitro/backends/factory.py", line 22, in get_backend
    libvmi = Libvmi(domain.name())
  File "/home/mininet/nfs/nitro/nitro/libvmi.py", line 50, in __init__
    raise LibvmiError('VMI_FAILURE')
nitro.libvmi.LibvmiError: VMI_FAILURE

I can't even use native libvmi examples using the above version of libvmi, as it shows:

sudo vmi-process-list ubuntu18.04
VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
Failed to init LibVMI library.

I've tested it on ubuntu14.04 to meet the requirements of nitro-kmod, gcc, python, and so on. It's really unclear what's the exactly environment to repeat the results described in Readme.

So my question is:

  1. What's the exact version(branch/commit) of "the modified kvm modules" describled at https://github.com/KVM-VMI/nitro#usage, and where can I find it?
  2. What's the exact version(branch/commit) of libvmi described at https://github.com/KVM-VMI/nitro#libvmi?
  3. Which version of kernel , gcc, and python should I choose to configure all the components?

I need to repeat the result of nitro to compare it with a new VMI tool, so it will be greatly appreciated if anyone who successfully configured nitro can guide me through it.

Pipeline Oriented Architecture

I've been thinking that the core of Nitro API could be structured to be more pipeline-oriented. Previously, we were discussing what information backends should extract and what should their responsibilities be. My understanding is that we want Nitro to be a flexible framework for all kinds of different use cases. What ever we decide regarding what analysis backends should do is likely to be wrong for someones use case. In the current architecture, backends are kind of special as they are automatically created by the Nitro class (when analysis is enabled).

What if the core Nitro API was structured more like a pipeline where events would pass through various extraction steps where each extractor could alter/extend the events with more information. Nitro's patched KVM could act as a source for the pipeline, but the could be other sources like mock events for testing purpose or other mechanism for extracting system calls from VMs. The API could allow the user to define a processing pipeline where they could select just the functionality they require. If they did not need to know, lets say, the process associated with a particular event, they would simply leave that extractor out of the pipeline.

In this architecture, some of the processing steps could of course depend on other pipeline components having happened before them. For example, a step extracting detailed user information associated with a processes could depend on a step that extracted process descriptors. In a more statically-typed language you could probaly even represent these requirements in the type signatures for the pipeline processing steps, but even if we do not get that benefit I think this would still make the internal architecture cleaner.

This would not necessarily be a huge change, more like a cosmetic change to the user facing API. Currently, the user of the framework initializes a Nitro object which automatically brings in a backend if anlysis is enabled. What I envision, is an API where the user defines a pipeline (I want this event source and then I want to chain the events into this extractor and then the output from that to this other thing... etc) and starts the pipeline. Each step in the pipeline will process events from the previous source and emit new emits or produce some side effects. At the end of the pipeline we could have sinks that dump the produced events into a database or JSON or something. Basically, this would put other analysis steps in a more equal footing with the backends as the current backends would be "just another step in the analysis pipeline". If we wanted to, we could of course go even further and split up the current backend code into smaller pieces that extracted individual bits of information from events.

This change wouldn't necessary enable anything that the user cannot already do. However, it would enable us to not have to worry about question like what should the backend do as it wouldn't matter as much because the user of the framework could construct their pipeline however they like and choose whatever functionality they need. Additionally, I think it this architecture would be a bit cleaner as backends would be no different from other analysis steps.

I do not have a patch to demonstrate this idea right now and I opened this issue mostly to facilitate discussion about the idea. Do you think changing the user facing API this way would be sensible?

undefined reference in vagrant

Hi @Wenzel ,
I've ran main.py in vagrant. I've got this error:

vagrant@jessie:/vagrant/nitro$ ./main.py nitro_win7x64
Finding QEMU pid for domain nitro_win7x64
Detected 1 VCPUs
Dumping physical memory to /tmp/tmp2xqs280v/tmpbhzv8j73
Extracting symbols with Rekall
Loading symbols
Traceback (most recent call last):
  File "./main.py", line 76, in <module>
    main(docopt(__doc__))
  File "./main.py", line 51, in main
    with Backend(domain, analyze_enabled) as backend:
  File "/vagrant/nitro/nitro/backend.py", line 52, in __init__
    self.libvmi = Libvmi(domain.name())
  File "/vagrant/nitro/nitro/libvmi.py", line 62, in __init__
    self.libvmi = cdll.LoadLibrary('libvmi.so')
  File "/usr/lib/python3.4/ctypes/__init__.py", line 429, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.4/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/lib/libvmi.so: undefined symbol: xen_get_address_width

Also when I run process_list in libvmi, I get this error:

vagrant@jessie:/vagrant/libvmi/examples$ sudo ./vmi-process-list nitro_win7x64
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_address_width'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_set_vcpureg'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_init_vmi'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_write'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_vcpureg'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_vcpuregs'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_domainname'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_init'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_test'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_pause_vm'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_set_domainname'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_set_domainid'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_name_from_domainid'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_resume_vm'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_memsize'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_read_page'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_destroy'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_domainid_from_name'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_is_pv'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_set_vcpuregs'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_check_domainid'
/vagrant/libvmi/libvmi/.libs/libvmi.so: undefined reference to `xen_get_domainid'

Question about setup test vm.

Sorry about hijack other people's issue,i just want to know can i use virt-manager to create win7 test vm or use qemu-img and virt-install orders?

Improve nitro's performance.

I find that when use backend module to handle system call information,virtual machine performance is degraded,i think it's unreasonable to set vm paused when dealing with system call information,so is there any solution to improve nitro's performance?

Can't get symbols.

Hi,

When in nitro/symbols.py it is written autodetect_build_local='basic',
I get the error
root@cloud-security1:/home/shlomo/nitro/KVM-VMI/kvm-vmi/nitro# ./main.py --stdout nitro_win7x64
Dumping physical memory to /tmp/tmpom2kjng2/tmp_am5tp_i
Extracting symbols with Rekall
ERROR:root:Error: HTTP Error 404: Not Found
ERROR:root:Error: HTTP Error 404: Not Found
ERROR:root:Error: HTTP Error 404: Not Found

And when it is written autodetect_build_local='none',
I get the error
root@cloud-security1:/home/shlomo/nitro/KVM-VMI/kvm-vmi/nitro# ./main.py --stdout nitro_win7x64
Dumping physical memory to /tmp/tmprkrld2di/tmpf00bkm72
Extracting symbols with Rekall
ERROR:root:No profiles match this image. Try specifying manually.

What could be the reason for that.

Best regards,

S.P.

Improving Documentation

Nitro is currently lacking in documentation, this should be improved. I am creating this issue to track the progress of documentation project.

As of what the documentation should contain, as a starting point, I think we should have:

  • Basic API Documentation: Basic documentation of classes and functions.
  • Higher-Level Description of Project Structure: What are the different components and the relationships between them.
  • Example Usage: It could be nice if we had some simple examples of how the framework could be used. How to write new backends etc.

problem about the nitro/readme.md

you say " (Nitro only supports for now Windows XP x64 and Windows 7 x64, see the Note section below)" and " Start Nitro as root (go to the Notes section to see how to start it as a normal user)" ,
but i can't find the "Notes" section.
Could you tell me how to start the nitro as root ?
when i firstly use nitro , i input the "# ./main.py --nobackend nitro_win7x64", i get the result below:

kvm.py", line 140, in attach_vm
raise RuntimeError('Error: fail to attach to the VM ')
RuntimeError: Error: fail to attach to the VM

how can i solve the problem ?

vmwrite error when trying to set CPU registers in a callback

As the new syscall_parameters branch allows a read/write access to the syscall's arguments, i tried to read every arguments and write them back while in a callback.

It works fine for memory arguments.
However, for register arguments, at some point, the VM goes BSOD and dmesg looks like this

[17952.582712] vmwrite error: reg 6820 value 46 (err 70)
[17952.582717] CPU: 1 PID: 25724 Comm: nose2-3.4 Tainted: G            E   4.9.0-0.bpo.2-amd64 #1 Debian 4.9.18-1~bpo8+1.2
[17952.582717] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[17952.582719]  0000000000000000 ffffffff8c329dd5 ffff8b2c8c9d88c0 ffff8b2c8c9d88c0
[17952.582721]  ffffffffc0749540 ffffffffb33b8540 ffffffffc0731cd0 ffff8b2cf0e58700
[17952.582724]  ffff8b2cea86f140 00000000d958817f ffff8b2cd669a000 ffffa726c29c7f20
[17952.582726] Call Trace:
[17952.582730]  [<ffffffff8c329dd5>] ? dump_stack+0x5c/0x77
[17952.582749]  [<ffffffffc0749540>] ? kvm_arch_vcpu_ioctl_set_regs+0x1f0/0x210 [kvm]
[17952.582763]  [<ffffffffc0731cd0>] ? kvm_vcpu_ioctl+0x140/0x7c0 [kvm]
[17952.582765]  [<ffffffff8c2175cb>] ? do_vfs_ioctl+0x9b/0x600
[17952.582767]  [<ffffffff8c2035b3>] ? vfs_write+0x163/0x1a0
[17952.582769]  [<ffffffff8c217ba6>] ? SyS_ioctl+0x76/0x90
[17952.582772]  [<ffffffff8c5fc5bb>] ? system_call_fast_compare_end+0xc/0x9b

nitro can hook the function which from user level?

Hello,I know nitro use this code “self.sysvmi.backend.define_hook('NtCreateFile', enter_NtCreateFile)” to hook syscall,and NtCreateFile export from ntdll,but I want to hook the function like "RegSetValueA" export
from advapi32.dll.Can nitro be implemented under the current "UD" and "GP "mechanism? Thanks!

Backend Caching

Currently, Nitro backends simply clear all libvmi's caches to avoid possible inconsistencies. However, this has possible performance implications since we need to do more work at each incoming event. I created this issues to analyze the performance penalty that clearing all the caches has and to discuss various alternatives to the current approach. Could we be smarter about what caches to clear and when?

Use case planning

I created this ticket for brainstorming how Nitro could be used and what kind of APIs would be needed to support these use cases. The use cases could be low-level features or higher-level ideas. If there is need, I'll create more issues for individual items.

Off the top of my head, following things come to mind:

  • Collection of detailed information about individual applications. I guess this would include support for extracting system call arguments.
  • Altering of machine execution based on events that happened. So far our test have only included observing machine's activity.
  • Replacing, extending & altering OS system call handlers. I think this needs an API to control system call handler execution.

What do you think?

Why is the name of syscall unknown?

Hi @Wenzel , sorry for bothering you again.
But I have no idea what happened and how to solve the problem. So I have no choice but to ask you for help, I just want to know:

  • Did you ever meet this problem?
  • Would you please give me some suggestions?

Problem

All in all, in all the result of syscall, the filed "full_name"="Table0!Unknown" or "Table1!Unknown", and so "name"="Unknown".
This is the screenshot of nitro's output.
image

Something might help

I read the code in nitro/backends/windows/backend.py, and I think this problem could be caused by the incorrect result of nitro/backends/windows/get_symbols.py as follow:
image
As is in the code, the full names of syscall should have been loaded from the output of get_symbols.py, in the filed syscall_table. However in my result, there is not any syscall's name.
image

Also I have to emphasize that my rekall works not really well as I mentioned in #77 . When I analyze an image dumped manually, the process list is empty.


From the bottom of my heart, thank you again for help!

process_event() will not necessarly return the right syscall

Hi @Wenzel ,
Pushing syscall and poping it will not return the right one in all situations. Consider the situation where context switch occurs in the middle of handling one syscall. I think the previous version of process_event which calculates the name every time was better than this version.

RuntimeError:Process not found

Dear developers:

First of all, I want to apologize for bothering you even I knew you won't maintain this project anymore. I’ve tried my best to slove most of the problems by reading issues in https://github.com/google/rekall. However it’s archived last week, so I’m not able to view the issues anymore. So I’m sorry but I can only ask you for help.

It doesn’t matter if you have no idea either. I’d be quite grateful for your help (if possible).

I don’t know which information will be vital, so I want to describe the error information at first.
The problem now is “Process not found”. I noticed that:
1.It seems like nitro here can only backend process "svchost.exe".
2.I tried to print flink in find_eprocess(), there was actually a set of flink. This time the number of it was 35.
The detailed output is as below:

qp@qp:~/Desktop/sandbox/nitro$ sudo ./main.py win7x64
[sudo] password for qp:
Finding QEMU pid for domain win7x64
attach_vm PID = 2768
attach_vcpus
Detected 4 VCPUs
LibVMI Suggestion: set win_ntoskrnl=0x3c52000 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdbg=0x1e2120 in libvmi.conf for faster startup.
LibVMI Suggestion: set win_kdvb=0xfffff80003e34120 in libvmi.conf for faster startup.
Dumping physical memory to /tmp/tmpubwjmsl1/tmpo4tx3kq1
Extracting symbols with Rekall
WARNING:root:Unable to determine file size, assuming file is volatile.
Loading symbols
set_syscall_trap True
Start listening on VCPU 0
Start listening on VCPU 1
Start listening on VCPU 2
Start listening on VCPU 3
{'event': {'cr3': '0x742c9000',
'direction': 'enter',
'rax': '0x3',
'time': '2020-10-22T10:26:51.067791',
'type': 'syscall',
'vcpu': 0},
'full_name': 'Table0!Unknown',
'name': 'Unknown',
'process': {'command_line': 'C:\Windows\System32\svchost.exe '
'-k '
'secsvcs',
'create_time': '2020-10-22 '
'10:23:58',
'iswow64': False,
'name': 'svchost.exe',
'parent_pid': 428,
'path': '\Device\HarddiskVolume2\Windows\System32\svchost.exe',
'pid': 2560}}
{'event': {'cr3': '0x742c9000',
'direction': 'exit',
'rax': '0x0',
'time': '2020-10-22T10:26:51.068313',
'type': 'syscall',
'vcpu': 0},
'full_name': 'Table0!Unknown',
'name': 'Unknown',
'process': {'command_line': 'C:\Windows\System32\svchost.exe '
'-k '
'secsvcs',
'create_time': '2020-10-22 '
'10:23:58',
'iswow64': False,
'name': 'svchost.exe',
'parent_pid': 428,
'path': '\Device\HarddiskVolume2\Windows\System32\svchost.exe',
'pid': 2560}}

……
Traceback (most recent call last):
File "./main.py", line 91, in
main()
File "./main.py", line 87, in main
runner.run()
File "./main.py", line 55, in run
syscall = self.nitro.backend.process_event(event)
File "/home/qp/Desktop/sandbox/nitro/nitro/backends/windows/backend.py", line 116, in process_event
process = self.associate_process(cr3)
File "/home/qp/Desktop/sandbox/nitro/nitro/backends/windows/backend.py", line 159, in associate_process
p = self.find_eprocess(cr3)
File "/home/qp/Desktop/sandbox/nitro/nitro/backends/windows/backend.py", line 189, in find_eprocess
raise RuntimeError('Process not found')
RuntimeError: Process not found

Cause I can only generate the profile of my own virtual machine through rekall installed on python3.6, I also tried to modify /nitro/nitro/backends/windows/get_symbols.py&backend.py, which ended with the same error.

……
File "/home/qp/Desktop/sandbox/nitro/nitro/backends/windows/backend.py", line 189, in find_eprocess
raise RuntimeError('Process not found')

Also I have to mention that there is not any process when I use command line to analyze a memory image, in which a.json is the profile generated by rekall(py3). And command pslist return an empty list:

qp@qp:~/Desktop/sandbox/nitro/debug$ rekall -f win7x64.raw --profile ~/Desktop/pdbdump_5/a.json
2020-10-21 17:35:40,007:WARNING:rekall.1:Unable to determine file size, assuming file is volatile.

The Rekall Digital Forensic/Incident Response framework 1.7.2.rc1 (Hurricane Ridge).
"We can remember it for you wholesale!"
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License.
See http://www.rekall-forensic.com/docs/Manual/tutorial.html to get started.

[1] win7x64.raw 17:35:43> pslist
—> pslist()
_EPROCESS name pid ppid thread_count handle_count session_id wow64 process_create_time process_exit_time
‘’-------------- -------------------- ----- ------ ------------ ------------ ---------- ------ ------------------------ ------------------------‘’
Out<17:35:43> Plugin: pslist (WinPsList)
[1] win7x64.raw 17:35:43>

All in all, I think the problem is caused by rekall. But I couldn’t view the rekall/issues, so I have to ask you for help.
Once again, I’m quite grateful for you help if possible!

I have some questions want to know

The event in your project is syscall?
This syscall is only traditional system calls or include fast system calls?
Now you can catch the syscall which have been implemented are exit and enter? or all of syscall?
I want collection all syscall(traditional and fast) to do an anomaly detection. But I don't know much about this. I see you do many work from the libvmi project.
Look forward to your reply. Thank you!

Can Nitro extract the API in windows DLL?

Hi @Wenzel .
I want to know if nitro can extract the API in windows DLL?
Actually I programed a demo called the function GetSystemTime of kernel32.dll. But I don't know which syscall in nitro is related.
Because there is not a syscall named exactly GetSystemTime in profile (only nt and win32k table). So I guess the function actually called a few syscalls instead, which can be extracted by nitro.


If you didn't meet this kind of problem, just forget it. I'll try to debug the dll to find some information then.
Thank you for reading.
(And it couldn't be better if you have some suggestions.)

Designing a API to read/write syscall's arguments

Problem 1 : an API to access the syscall arguments

We need to design a API on top of the Sycall object that we are building from a Nitro eventl to provide a transparent way to get the syscall arguments for the user accross different operating systems and conventions.

Let's take for example NtOpenFile MSDN definition:

NTSTATUS NtOpenFile(
  _Out_ PHANDLE            FileHandle,
  _In_  ACCESS_MASK        DesiredAccess,
  _In_  POBJECT_ATTRIBUTES ObjectAttributes,
  _Out_ PIO_STATUS_BLOCK   IoStatusBlock,
  _In_  ULONG              ShareAccess,
  _In_  ULONG              OpenOptions
);

The arguments will be passed to the kernel in a specific way, which depends on

  • The operating system calling convention
  • The system call type (syscall or sysenter)

Windows/Linux syscall arguments passing

Windows sysenter Windows syscall Linux sysenter Linux syscall
arg 1 edx rcx ebx rdi
arg 2 edx + 4 rdx ecx rsi
arg 3 edx + 8 r8 edx rdx
arg 4 edx + 12 r9 esi r10
arg 5 edx + 16 rsp + ? edi r8
arg 6 edx + 20 rsp + ? + 8 ebp r9
arg 7 edx + 24 rsp + ? + 16 no no

References

Key findings

  • Linux syscalls have a maximum of 6 parameters, always
  • Linux always uses registers for the syscall argument passing
  • Windows uses registers and memory.

Problem 2 : Modifying the syscall arguments

An interesting feature of Nitro is the ability to modify the syscall arguments on the fly, and change the behavior of the guest.

Following what we discovered previously, if we want to provide such an API, it has to cover 2 cases:

  • argument is stored in a register
  • argument is stored in memory (Windows)

Respectively, we have 2 solutions :

  • make use of nitro_set_regs ioctl to replace the registers value
  • make use of libvmi's vmi_write_va API to write the memory

Current design

This is how a Nitro syscall callback currently looks like

        def enter_NtCreateFile(syscall):
            KeyHandle, DesiredAccess, object_attributes = syscall.collect_args(3)
            obj = ObjectAttributes(object_attributes, syscall.process)
            buffer = obj.ObjectName.Buffer
            access = FileAccessMask(DesiredAccess)
            syscall.hook = {
                'object_name': buffer,
                'access': access.rights
            }

Issues

  1. collect_args is limited to the Windows/syscall convention, which is hardcoded right now
  2. we are forcing the developer to unpack all values, even if he just wants the 3rd argument, he will have to write: *rest, object_attributes = syscall.collect_args(3)

Proposal

A new callback interface has been designed previously.
The backend is now sent as a parameter.

Apart from that, if we want the Syscall object to be able to modify the arguments, we need to pass the Nitro object which provides the API over nitro's ioctls, as well as libvmi.

Also, it might be interesting to make use of Python's __getitem__ and __setitem__ methods, to provide a list like API for the developer. He could access the arguments directly index them and this solves the previous issue raised about collect_args.

What a use case could look like:

        def enter_NtCreateFile(backend, syscall):
            object_attributes = syscall.args[2] # getting the 3rd argument
            # modifying a value
            syscall.args[2] = 0 # setting object_attributes pointer to NULL

-> Behing the scene, how could we design the Syscall class to make an interface accross the various calling conventions that we saw at the beginning ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.