Giter VIP home page Giter VIP logo

osquery-python's Introduction

osquery-python

osquery-python-logo

osquery exposes an operating system as a high-performance relational database. This allows you to write SQL-based queries to explore operating system data. With osquery, SQL tables represent abstract concepts such as running processes, loaded kernel modules, open network connections, browser plugins, hardware events or file hashes.

If you're interested in learning more about osquery, visit the GitHub project, the website, and the users guide.

What is osquery-python?

Build Status

In osquery, SQL tables, configuration retrieval, log handling, etc are implemented via a simple, robust plugin and extensions API. This project contains the official Python bindings for creating osquery extensions in Python. Consider the following example:

#!/usr/bin/env python

import osquery

@osquery.register_plugin
class MyTablePlugin(osquery.TablePlugin):
    def name(self):
        return "foobar"

    def columns(self):
        return [
            osquery.TableColumn(name="foo", type=osquery.STRING),
            osquery.TableColumn(name="baz", type=osquery.STRING),
        ]

    def generate(self, context):
        query_data = []

        for _ in range(2):
            row = {}
            row["foo"] = "bar"
            row["baz"] = "baz"
            query_data.append(row)

        return query_data

if __name__ == "__main__":
    osquery.start_extension(name="my_awesome_extension", version="1.0.0")

To test this code start an osquery shell:

osqueryi --nodisable_extensions
osquery> select value from osquery_flags where name = 'extensions_socket';
+-----------------------------------+
| value                             |
+-----------------------------------+
| /Users/USERNAME/.osquery/shell.em |
+-----------------------------------+

Then start the Python extension:

python ./my_table_plugin.py --socket /Users/USERNAME/.osquery/shell.em

Alternatively, you can also autoload your extension when starting an osquery shell:

osqueryi --extension path_to_my_table_plugin.py

This will register a table called "foobar". As you can see, the table will return two rows:

osquery> select * from foobar;
+-----+-----+
| foo | baz |
+-----+-----+
| bar | baz |
| bar | baz |
+-----+-----+
osquery>

This is obviously a contrived example, but it's easy to imagine the possibilities.

Using the instructions found on the wiki, you can easily deploy your extension with an existing osquery deployment.

Extensions are the core way that you can extend and customize osquery. At Facebook, we use extensions extensively to implement many plugins that take advantage of internal APIs and tools.

Execute queries in Python

The same Thrift bindings can be used to create a Python client for the osqueryd or osqueryi's extension socket. There are helper classes provided that spawn an ephemeral osquery process for consecutive or long running client instances.

import osquery

if __name__ == "__main__":
    # Spawn an osquery process using an ephemeral extension socket.
    instance = osquery.SpawnInstance()
    instance.open()  # This may raise an exception

    # Issues queries and call osquery Thrift APIs.
    instance.client.query("select timestamp from time")

Connect to an existing socket

In the example above the SpawnInstance() method is used to fork and configure an osquery instance. We can use similar APIs to connect to the Thrift socket of an existing osquery instance. Remember, normal UNIX permissions apply to the Thrift socket.

Imagine if you started osqueryd:

$ osqueryd --ephemeral --disable_logging --disable_database \
    --extensions_socket /home/you/.osquery/osqueryd.sock &

Then use the Python bindings:

import osquery

if __name__ == "__main__":
    # You must know the Thrift socket path
    # For an installed and running system osqueryd, this is:
    #   Linux and macOS: /var/osquery/osquery.em
    #   FreeBSD: /var/run/osquery.em
    #   Windows: \\.\pipe\osquery.em
    instance = osquery.ExtensionClient('/home/you/.osquery/osqueryd.sock')
    instance.open()  # This may raise an exception

    # Issue queries and call osquery Thrift APIs.
    client = instance.extension_client()
    client.query('select timestamp from time')

Install

To install from PyPi, run the following:

pip install osquery

Alternatively, to install from this repo, run the following:

python setup.py build
python setup.py install

Development

See CONTRIBUTING.md and the osquery wiki for development information.

Vulnerabilities

Facebook has a bug bounty program that includes osquery. If you find a security vulnerability in osquery, please submit it via the process outlined on that page and do not file a public issue. For more information on finding vulnerabilities in osquery, see a recent blog post about bug-hunting osquery.

osquery-python's People

Contributors

addvilz avatar bpmcdevitt avatar eoinmiller-r7 avatar facebook-github-bot avatar jarryshaw avatar malina-kirn avatar marpaia avatar mattf9 avatar muffins avatar nickovs avatar prasanthbazz avatar rubab-syed avatar thedrow avatar theopolis avatar waywardmonkeys avatar zwass avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

osquery-python's Issues

windows: osquery Python extensions do not appear to work on Windows

I haven't had too much time to triage this, but it seems that just running the generic python extension in Windows fails. osqueryi is creating the pipe correctly, as it shows up in sysinternals, however upon running python .\my_ext.py --socket \\.\pipe\shell.em, the code immediate returns. I haven't been able to trigger higher levels of verbosity, but again I haven't spent a large amount of time triaging this.

Not working in Python 3. ( Error: No Module named ttypes)

osquery-python scripts aren't running in python 3. I am getting following error :

Traceback (most recent call last): File "C:/Users/Mobeen/Documents/GitHub/osquery-python/examples/foobar_table.py", line 7, in <module> import osquery File "C:\Users\Mobeen\Documents\GitHub\osquery-python\osquery\__init__.py", line 33, in <module> from osquery.extension_client import DEFAULT_SOCKET_PATH, ExtensionClient File "C:\Users\Mobeen\Documents\GitHub\osquery-python\osquery\extension_client.py", line 17, in <module> from osquery.extensions.ExtensionManager import Client File "C:\Users\Mobeen\Documents\GitHub\osquery-python\osquery\extensions\ExtensionManager.py", line 10, in <module> import osquery.extensions.Extension File "C:\Users\Mobeen\Documents\GitHub\osquery-python\osquery\extensions\Extension.py", line 11, in <module> from ttypes import * ModuleNotFoundError: No module named 'ttypes'

I think this link, can provide more details on this issue.

Changing python version to 2.x does the job !

Replace TSimpleServer

./osquery/management.py:215:    server = TServer.TSimpleServer(processor, transport, tfactory, pfactory)

Should be replaced, this TServer implementation is just an example.

Table generation context is passed as double-encoded JSON

The context passed to table plugins' generate() methods is a JSON encoding of a JSON encoding of the data that the plugin actually cares about. If you want to access the actual context data structure then you need to use something like:

context = json.loads(json.loads(context))

While this issue has been in the code for at least six years, it's pretty clear from the source that this was not the original intention and is in fact a bug. Lines 41-43 of table_plugin.py read:

            ctx = {}
            if "context" in context:
                ctx = json.dumps(context["context"])

The underlying Thrift call context includes the table generation context as a string which contains the JSON-encoded details. The original author sets up a dictionary to pass to the generation function, then checks to see if a context was provided and, if present, replaces this with the JSON string encoding of the already-encoded context. It would make vastly more sense, and also be vastly more convenient for the user, to decode the context string value into another dictionary instead and thus for this to read:

                ctx = json.loads(context["context"])

This is almost certainly what the author intended and the current state is a typo. Hopefully this can still be fixed.

Docs on running queries within subprocess/calling osqueryi from python

Pardon if this is the wrong place to open this issue. So far I can't figure out how to call osqueryi from within a python script, which I'm not sure if this module would help with after seeing #11. example:

cmd = ['/usr/local/bin/osqueryi', '--csv', '"select name, path from kernel_extensions"']
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
list_with_pipes = proc.communicate()[0]
print list_with_pipes

Without setting shell=True I get a syntax error, but even with that, or using subprocess.check_output, I'm just getting an empty string. Halp?

osquery data lifetime in log file

Hello,

I have installed osquery in my lab... it is working fine..

when we run the conf file the data regarding the queries are stored in the log file..

and here can I know what is the life period of logs i.e., the data in the log file... do we have any such cases here?

No longer in beta mode?

I was able to install this library with pip. Should we remove the comment in the README file?

thrift.transport.TTransport.TTransportException: Called read on non-open pipe

I have a connection error. Any suggestions? Thank you

PS C:\WINDOWS\system32> python 'C:\Users\sampng\Desktop\python_script\superset.py'                                      Traceback (most recent call last):
  File "C:\Users\sampng\Desktop\python_script\superset.py", line 10, in <module>
    result=instance.client.query("select * from time")
  File "C:\Users\sampng\AppData\Local\Programs\Python\Python36\lib\site-packages\osquery\extensions\ExtensionManager.py", line 181, in query
    self.send_query(sql)
  File "C:\Users\sampng\AppData\Local\Programs\Python\Python36\lib\site-packages\osquery\extensions\ExtensionManager.py", line 190, in send_query
    self._oprot.trans.flush()
  File "C:\Users\sampng\AppData\Local\Programs\Python\Python36\lib\site-packages\thrift\transport\TTransport.py", line 179, in flush
    self.__trans.write(out)
  File "C:\Users\sampng\AppData\Local\Programs\Python\Python36\lib\site-packages\osquery\TPipe.py", line 128, in write
    message='Called read on non-open pipe')
thrift.transport.TTransport.TTransportException: Called read on non-open pipe

My Python script:

import os
import osquery

if __name__ == "__main__":
    # Spawn an osquery process using an ephemeral extension socket.
    instance = osquery.SpawnInstance(r"C:\Program Files\osquery\osqueryi.exe")
    instance.open()  # This may raise an exception

    # Issues queries and call osquery Thrift APIs.
    result=instance.client.query("select * from time")
    print (result)

osquery can't close win pipe

After any query on del of SpawnInstance I get an ignored exception
Exception ignored in <bound method SpawnInstance.__del__ of <osquery.management.SpawnInstance object at 0x021854F0>> pywintypes.error: (1, 'DicsonnectNamedPipe', 'Incorrect function.')
my script:
import osquery instance = osquery.SpawnInstance() instance.open() instance.client.query("<some query>")
Now i use some dirty "hack" as after
instance.client.query("<some query>")
i do:
instance.connection=None
My machine os:
Windows 7x64

BrokenPipeError when running deb_packages query

Bug report

What operating system and version are you using?

Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

What version of osquery are you using?

5.2.3

What steps did you take to reproduce the issue?

  1. Open a brand new EC2 Ubuntu instance

  2. Install osquery by performing the following

    sudo apt-get install -y gnupg software-properties-common python3-pip

    export debian_frontend=noninteractive
    export osquery_key=1484120ac4e9f8a1a577aeee97a80c63c9d8b80b

    sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $osquery_key
    sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
    sudo apt-get -y update
    sudo apt-get -y install osquery


   pip3 install osquery==3.0.7
  1. Run this small reproducible example:
    import osquery
    import time

    instance = osquery.SpawnInstance()
    instance.open()

    for i in range(5):
        # throws error in third iteration of `i`
        query = instance.client.query("select * from deb_packages")
        print(query.response)
        time.sleep(2)
  1. The following error during the third iteration of the loop:
        Traceback (most recent call last):
        File "/home/ubuntu/.local/lib/python3.10/site-packages/thrift/transport/TSocket.py", line 178, in write
            plus = self.handle.send(buff)
        BrokenPipeError: [Errno 32] Broken pipe

        During handling of the above exception, another exception occurred:

        Traceback (most recent call last):
        File "/home/ubuntu/osquery_demo.py", line 9, in <module>
            query = instance.client.query("select * from deb_packages")
        File "/home/ubuntu/.local/lib/python3.10/site-packages/osquery/extensions/ExtensionManager.py", line 181, in query
            self.send_query(sql)
        File "/home/ubuntu/.local/lib/python3.10/site-packages/osquery/extensions/ExtensionManager.py", line 190, in send_query
            self._oprot.trans.flush()
        File "/home/ubuntu/.local/lib/python3.10/site-packages/thrift/transport/TTransport.py", line 179, in flush
            self.__trans.write(out)
        File "/home/ubuntu/.local/lib/python3.10/site-packages/thrift/transport/TSocket.py", line 185, in write
            raise TTransportException(message="unexpected exception", inner=e)
        thrift.transport.TTransport.TTransportException: unexpected exception

What did you expect to see?

I expected osquery to return the query result for deb_packages 5 times(in the above demo example).

What did you see instead?

osquery successfully returned deb_packages two times in the loop but raised a BrokenPipeError exception on the third iteration of the loop.

UI to interact with OSQuery

I'm planning to create a UI, which provides an interactive way to query the OSQuery python in the backend and provide the result.

The idea is to create a Dynamic page that will allow the user to enter their query be it in SQL or in English syntax and execute the query in the back, this will provide much flexibility for the users.

What is your comment on the idea ?

Awaiting for your reply.

Could not connect to any of ['/tmp/pyosqsockndnbir6z']

Could not connect to any of ['/tmp/pyosqsockndnbir6z']
Encountering this error repeatedly .
Also it is raised after below subprocess command. I haven't executed the below command in my code.
/usr/bin/osqueryd --extensions_socket /tmp/pyosqsockndnbir6z --disable_database --disable_watchdog --disable_logging --ephemeral --config_path /dev/null

Make the tests reflect the new directory structure

Since breaking up the osquery module out of __init__.py into many smaller files, we should have the tests mirror the same directory structure, but in the tests top-level directory. We should also write some better tests while we at it, because they're not that complete right now.

autoload_extension not working

Environment:

  • Operating System: RHEL 7
  • osquery Version: 3.3.2
  • osquery-python Version: 3.0.5

The first way to load extension worked for me. However doing:
osqueryi --extension /root/test-osquery/create_table.py --verbose(I tried changing filename to end with .ext)

gives me:
I0312 21:29:56.564863 26086 process.cpp:172] Could not start extension process: /root/test-osquery/create_table.ext
terminating

Am I missing something obvious?

UnicodeDecodeError in Python3

This is an issue with thrift (dependency of this library), an open issue is already filed to that project.

Environment:

  • Operating System: Windows 10 Pro (Simplified Chinese)
  • Python Interpreter: Python 3.6.6
  • osquery Version: 3.3.0
  • osquery-python Version: 3.0.5

When querying, UnicodeDecodeError raised with error message: "'utf-8' codec can't decode byte 0xc3 in position 0: invalid continuation byte" from thrift.compat.binary_to_str, which is because the encoding of bin_val parameter should be "gbk".

Maybe try hacking the source code of thrift and include it as a vendor package when distribution? (just as pipenv and other projects do)

OSError: [Errno 24] Too many open files

Hi, i use osquery-python in simple REST service - https://gist.github.com/mgramin/0dd3872813047a27886d387a188781cd

But sometimes I get an error:

Traceback (most recent call last):
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 2328, in __call__
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 2314, in wsgi_app
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 269, in error_router
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 1760, in handle_exception
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 266, in error_router
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 2311, in wsgi_app
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 1834, in full_dispatch_request
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 269, in error_router
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 1737, in handle_user_exception
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 266, in error_router
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 1832, in full_dispatch_request
  File &quot;/usr/lib64/python2.7/site-packages/flask/app.py&quot;, line 1818, in dispatch_request
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 458, in wrapper
  File &quot;/usr/lib64/python2.7/site-packages/flask/views.py&quot;, line 88, in view
  File &quot;/usr/lib64/python2.7/site-packages/flask_restful/__init__.py&quot;, line 573, in dispatch_request
  File &quot;/tmp/osquery_rest.py&quot;, line 21, in post
  File &quot;/usr/lib/python2.7/site-packages/osquery/management.py&quot;, line 74, in __init__
  File &quot;/usr/lib64/python2.7/tempfile.py&quot;, line 304, in mkstemp
  File &quot;/usr/lib64/python2.7/tempfile.py&quot;, line 239, in _mkstemp_inner
OSError: [Errno 24] Too many open files: '/tmp/pyosqsockHE35yD'

Python 3.5: Resource Warning

I am using a SpawnInstance as an object within a class. Unfortunately

self._instance = osquery.SpawnInstance()
...
self._instance.open()
r = self._instance.client.query(strSQL)

Apparently the SpawnInstance object leaves three resources open on exit and this triggers Python 3 warnings:

sys:1: ResourceWarning: unclosed file <_io.BufferedWriter name=7>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=8>
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=10>

I tried to force a call to the __del__ function of the instance, but this does not prevent the warning. Unfortunately this warning is difficult to mask, because the source is somewhere likely in a thread.

I am puzzled because the fowlling code does not trigger the ResourceWarning on exit:

import osquery
instance = osquery.SpawnInstance()
instance.open()  # This may raise an exception
# Issues queries and call osquery Thrift APIs.
r = instance.client.query("select timestamp from time")
print(r)

Any idea what's happening?

How to get a python table to autoload?

I've managed to create a table in python and manually load it so that I can query it via osqueryi, but I can't figure out how to make that table autoload. I feel like I must be missing something obvious, but I'd appreciate any help.

osquery socket file leak

In management.py socket file is created on line 85 using mkstemp.

This file is NOT automatically removed, and remains forever in /tmp, so spawning osquery instances regularly results in many garbage tempfiles in /tmp. File deletion should be handled in __del__.

No required column support for table plugin

It's seems that the table plugin doesn't support complex stuff like required columns.
After some investigation, it looks like the thrift api doesn't support this - although I was able to set a column as required (changed the column's "op" to 2), the context always stays the same so I can't use the where clause to return only a subset of rows.
Am I missing something?

Example table not working, osqueryi 1.8.2

I've copy-pasted the exact MyTablePlugin example, renamed it with .ext, and get the following traceback when loading it as instructed in the ReadMe:

Traceback (most recent call last):
  File "/Users/abanks/Desktop/MyTablePlugin.ext", line 26, in <module>
    osquery.start_extension(name="my_awesome_extension", version="1.0.0")
  File "/Library/Python/2.7/site-packages/osquery/management.py", line 184, in start_extension
    message=status.message,
osquery.extensions.ttypes.ExtensionException: ExtensionException(code=1, _message='Failed adding registry broadcast', uuid=None)

TypeError when sending query using python3

running this code:

import osquery
CLIENT = osquery.ExtensionClient()
CLIENT.open()
RESULTS = CLIENT.extension_client().query(sys.argv[1])

results in the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/osquery-1.5.3-py3.4.egg/osquery/extensions/ExtensionManager.py", line 183, in query
  File "/usr/local/lib/python3.4/dist-packages/osquery-1.5.3-py3.4.egg/osquery/extensions/ExtensionManager.py", line 187, in send_query
  File "/usr/local/lib/python3.4/dist-packages/thrift/protocol/TBinaryProtocol.py", line 46, in writeMessageBegin
    self.writeI32(TBinaryProtocol.VERSION_1 | type)
  File "/usr/local/lib/python3.4/dist-packages/thrift/protocol/TBinaryProtocol.py", line 111, in writeI32
    self.trans.write(buff)
  File "/usr/local/lib/python3.4/dist-packages/thrift/transport/TTransport.py", line 168, in write
    raise e
  File "/usr/local/lib/python3.4/dist-packages/thrift/transport/TTransport.py", line 164, in write
    self.__wbuf.write(buf)
TypeError: string argument expected, got 'bytes' 

when ran using python3.

Should I run this code differently, when using python3, or is this a bug?

Thanks in advance

Fix typo error

I ran this example code and I got this exception : called read on non-open pipe.
I also tried alternate approach of SpawnInstance(). Still I get the same error.

import osquery
if __name__ == "__main__":
    instance = osquery.ExtensionClient('\\.pipe\osquery.em')
    print(instance.open())  # This may raise an exception
    client = instance.extension_client()
    client.query('select timestamp from time')

Screenshot (5)

No handlers could be found for logger "thrift.transport.TSocket"

Have osquery service running - sud./build/debug_centos7/osquery/osqueryd --allow_unsafe --pidfile /tmp/osquery.pid --database_path /tmp/osquery.db --extensions_socket=/var/osquery/osquery.em
and would like execute queries on the running instance. Tried out the python example and running in to the following issue. Any thoughts ?

[vagrant@pdhillon-vm2 examples]$ python run.py 'select * from time'
No handlers could be found for logger "thrift.transport.TSocket"
Traceback (most recent call last):
File "run.py", line 15, in
RESULTS = CLIENT.extension_client().query(sys.argv[1])
File "build/bdist.linux-x86_64/egg/osquery/extensions/ExtensionManager.py", line 184, in query
File "build/bdist.linux-x86_64/egg/osquery/extensions/ExtensionManager.py", line 193, in send_query
File "build/bdist.linux-x86_64/egg/thrift/transport/TTransport.py", line 177, in flush
File "build/bdist.linux-x86_64/egg/thrift/transport/TSocket.py", line 138, in write
thrift.transport.TTransport.TTransportException: Transport not open
[vagrant@pdhillon-vm2 examples]$

Can't use extensions in windows.

I started working on my custom table on windows but had to switch to Ubuntu due to pip issues. The extension works as intended on Ubuntu but when I tried on windows again it didn't work. I keep getting the following error: "Thrift message: TPipe::open ::CreateFile errored GLE=errono = 2". Nick Anderson on osquery.slack.com assured me this was a harmless info message. But when I go look in osquery.results.log it's empty whereas I use a built-in table like uptime, everything is a-ok. I even used the foobar_extension.ext on this git but no luck. I downgraded osquery to 2.11.2 as I started in that version but had since updated to 3.2.4, still no luck. Is this a known issue? Is it even an osquery-python issue (I'm only 70% sure)? Feel free to ask more information you need.

Can two processes of osquery use one extension socket?

Hi ,

I face one problem, when I set up this new table, and start osqueryi by "osqueryi --extensions_socket=/var/osquery/shell.em" , and then I open a screen and use "python test.py --socket /var/osquery/osquery.em", it works. but when I open another screen and also run "python test.py --socket /var/osquery/osquery.em", I can not find any information about new table.

Is there anything I am wrong ?

Add "osqpipe" concept with an open/pipe method

We can enable a Python interpreter or integration tool to run an osqueryd instance in the background using an "osqpipe" concept. The python process manages the subprocess and uses subsequent thrift calls into the transient socket created.

This would mimic the functionality/concept radare2's "r2pipe"

Release spawnInstance raise error.

  • platform : win11
  • python version: 3.9.7
  • osquery version: 5.2.3
  • osquery-python version: 3.0.7

when release a SpawnInstance, it always raise a error pywintypes.error: (1, 'DisconnectNamedPipe', 'Incorrect function.')

test code:

import osquery

if __name__ == "__main__":
    instance = osquery.SpawnInstance()
    instance.open()
    del instance
    print('end test')

And it will raise a error pywintypes.error: (1, 'DisconnectNamedPipe', 'Incorrect function.')

osquery.SpawnInstance() fails on windows

  1. pip install osquery in a windows Python virtualenv
  2. Run python
  3. import osquery
  4. osquery.SpawnInstance()

Expected:

<osquery.management.SpawnInstance object at 0xBLAHBLAHBBLAH>

Observed:

>>> core.mk_instance()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\path\to\core.py", line 229, in mk_instance
    instance = osquery.SpawnInstance()
  File "C:\path\to\env\lib\site-packages\osquery\management.py", line 71, in __init__
    self.path = WINDOWS_BINARY_PATH
NameError: name 'WINDOWS_BINARY_PATH' is not defined. Did you mean: 'LINUX_BINARY_PATH'?

Windows version (once working):

>>> inst.client.query("select * from system_info")
ExtensionResponse(status=ExtensionStatus(code=0, message='OK', uuid=0), response=[{'board_model': '', 'board_serial': '', 'board_vendor': '', 'board_version': '', 'computer_name': 'DESKTOP-U465U82', 'cpu_brand': 'Intel(R) Core(TM) m3-7Y30 CPU @ 1.00GHz', 'cpu_logical_cores': '4', 'cpu_microcode': '180', 'cpu_physical_cores': '2', 'cpu_sockets': '1', 'cpu_subtype': '-1', 'cpu_type': 'x86_64', 'hardware_model': 'UX330CAK', 'hardware_serial': 'H6N0WU011420239', 'hardware_vendor': 'ASUSTeK COMPUTER INC.', 'hardware_version': '-1', 'hostname': 'DESKTOP-U465U82', 'local_hostname': 'DESKTOP-U465U82', 'physical_memory': '8589934592', 'uuid': '304E3648-5557-3130-3134-323032333936'}])
>>> inst.client.query("select * from os_version")
ExtensionResponse(status=ExtensionStatus(code=0, message='OK', uuid=0), response=[{'arch': '64-bit', 'build': '19045', 'codename': 'Microsoft Windows 10 Home', 'install_date': '1708910913', 'major': '10', 'minor': '0', 'name': 'Microsoft Windows 10 Home', 'patch': '', 'platform': 'windows', 'platform_like': 'windows', 'revision': '4170', 'version': '10.0.19045'}])
>>>

Cannot start process from path: /usr/local/bin/osqueryd

I installed osquery using:

$ brew install osquery

Verified it works:

$ osqueryi
Using a virtual database. Need help, type '.help'
osquery> select timestamp from time;
+------------------------------+
| timestamp                    |
+------------------------------+
| Mon Apr 17 15:36:50 2023 UTC |
+------------------------------+

Copied the code from README:

import osquery

if __name__ == "__main__":
    instance = osquery.SpawnInstance()
    instance.open()
    instance.client.query("select timestamp from time;")

When I ran above code, I get this error:

Traceback (most recent call last):
  File "main.py", line 5, in <module>
    instance.open()
  File "./venv/lib/python3.8/site-packages/osquery/management.py", line 137, in open
    raise Exception("Cannot start process from path: %s" % (self.path))
Exception: Cannot start process from path: /usr/local/bin/osqueryd

I then noticed that osqueryi is simply a symlink to osqueryd which the above is looking for:

$ ls -a /usr/local/bin/osqueryi
lrwxr-xr-x 52 root 17 Apr 11:20 /usr/local/bin/osqueryi -> /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd

So I tried this:

import osquery

if __name__ == "__main__":
    instance = osquery.SpawnInstance("/usr/local/bin/osqueryi")
    instance.open()
    instance.client.query("select timestamp from time;")

But, now I get different error:

Traceback (most recent call last):
  File "./main.py", line 6, in <module>
    instance.client.query("select timestamp from time;")
  File "./venv/lib/python3.8/site-packages/osquery/extensions/ExtensionManager.py", line 181, in query
    self.send_query(sql)
  File "./venv/lib/python3.8/site-packages/osquery/extensions/ExtensionManager.py", line 190, in send_query
    self._oprot.trans.flush()
  File "./venv/lib/python3.8/site-packages/thrift/transport/TTransport.py", line 179, in flush
    self.__trans.write(out)
  File "./venv/lib/python3.8/site-packages/thrift/transport/TSocket.py", line 172, in write
    raise TTransportException(type=TTransportException.NOT_OPEN,
thrift.transport.TTransport.TTransportException: Transport not open

What am I doing wrong? Why is running code from the README doesn't work??

No module named 'win32event'

Traceback (most recent call last):
File "D:/Documenten/stage/OSIR/parser/csv_parser.py", line 3, in
import osquery
File "C:\Users\kenne.LAPTOP-66E28DMG\AppData\Local\Programs\Python\Python36-32\lib\site-packages\osquery_init_.py", line 34, in
from osquery.extension_client import DEFAULT_SOCKET_PATH, ExtensionClient,
File "C:\Users\kenne.LAPTOP-66E28DMG\AppData\Local\Programs\Python\Python36-32\lib\site-packages\osquery\extension_client.py", line 25, in
from osquery.TPipe import TPipe
File "C:\Users\kenne.LAPTOP-66E28DMG\AppData\Local\Programs\Python\Python36-32\lib\site-packages\osquery\TPipe.py", line 11, in
import win32event
ModuleNotFoundError: No module named 'win32event'

pip --version
pip 10.0.0

python --version
Python 3.6.5

SpawnInstance leaving pyosqsock and pyosqpid files in /tmp

I have been using the SpawnInstance class and have found that though the del method attempts to remove the temporary files created by the tempfiles Python module by calling os.unlink it does not seem to work on an AWS Ubuntu instance.

This came as a surprise to me when we deployed it since I developed the code on Mac OS and no such files were behind

Release?

Could you please bump the release? Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.