libyal / libfwsi Goto Github PK

Library to access the Windows Shell Item format

License: GNU Lesser General Public License v3.0

PowerShell 1.26% Shell 4.18% C 80.61% Makefile 1.66% C++ 0.18% Python 1.48% M4 9.91% Roff 0.72%

libfwsi's Introduction

libfwsi is a library to access the Windows Shell Item format.

Project information:

* Status: experimental
* Licence: LGPLv3+

For more information see:

* Project documentation: https://github.com/libyal/libfwsi/wiki/Home
* How to build from source: https://github.com/libyal/libfwsi/wiki/Building

libfwsi's People

Contributors

Stargazers

Watchers

Forkers

yang123vc ith4cker readercrap mak 5433d-r32433 woniuke neeraj7078 jcunews chennqqi wulinfa

libfwsi's Issues

Extend pyfwsi to provide support for more shell item types

"pyfwsi" it exposes 4 types of shell items: volume, file_entry, network_location and root_folder however it would be beneficial to have support for:

GUID: Control panel
Control Panel Category
Variable: Users property view
Users property view: Drive letter
Users Files Folder
Variable
Users property view

Is it possible to add support for those shell types as well?

Thanks!

line 23476 of configure has too many #'s

23466-ac_compiler_gnu=$ac_cv_c_compiler_gnu
23467-
23468- cat confdefs.h - <<_ACEOF >conftest.$ac_ext
23469-/* end confdefs.h. */
23470-#include <libclocale/features.h>
23471-int
23472-main ()
23473-{
23474-#if !defined( LIBCLOCALE_HAVE_WIDE_CHARACTER_TYPE ) || ( LIBCLOCALE_HAVE_WIDE_CHARACTER_TYPE != 1 )
23475-#error LIBCLOCALE_HAVE_WIDE_CHARACTER_TYPE not defined
23476:##endif

This causes the test program to fail every time.

Import Error: Undefined Symbol after compiling from source

I experience an error when starting log2timeline.py after compiling libfwsi from git into pyfwsi on an arch linux system.

> log2timeline.py --version
Traceback (most recent call last):
  File "/usr/bin/log2timeline.py", line 5, in <module>
    pkg_resources.run_script('plaso==1.2.1-20150206', 'log2timeline.py')
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 528, in run_script
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 1401, in run_script
  File "/usr/lib/python2.7/site-packages/plaso-1.2.1_20150206-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 31, in <module>

  File "build/bdist.linux-x86_64/egg/plaso/frontend/frontend.py", line 36, in <module>
  File "build/bdist.linux-x86_64/egg/plaso/parsers/__init__.py", line 27, in <module>
  File "build/bdist.linux-x86_64/egg/plaso/parsers/custom_destinations.py", line 31, in <module>
  File "build/bdist.linux-x86_64/egg/plaso/parsers/winlnk.py", line 27, in <module>
  File "build/bdist.linux-x86_64/egg/plaso/parsers/shared/shell_items.py", line 20, in <module>
ImportError: /usr/lib/python2.7/site-packages/pyfwsi.so: undefined symbol: libfwsi_file_entry_extension_get_access_time

The package was compiled via:

env PYTHON=python2.7 PYTHON_VERSION=2.7 ./configure --enable-python;
make;
sudo make install;
cd pyfwsi;
python2.7 setup.py build; 
sudo python2.7 setup.py install;

The file /usr/lib/python2.7/site-packages/pyfwsi.so has been created through the compiling steps.

Sorry if this may be a stupid question, but I compiled libsigscan just before that, and experienced no problems with the same command.

importing libfwsi for plaso

Traceback (most recent call last):
File "/usr/bin/log2timeline.py", line 31, in
from plaso.frontend import frontend
File "/usr/lib/python2.7/site-packages/plaso/frontend/frontend.py", line 36, in
from plaso import parsers # pylint: disable=unused-import
File "/usr/lib/python2.7/site-packages/plaso/parsers/init.py", line 26, in
from plaso.parsers import custom_destinations
File "/usr/lib/python2.7/site-packages/plaso/parsers/custom_destinations.py", line 31, in
from plaso.parsers import winlnk
File "/usr/lib/python2.7/site-packages/plaso/parsers/winlnk.py", line 27, in
from plaso.parsers.shared import shell_items
File "/usr/lib/python2.7/site-packages/plaso/parsers/shared/shell_items.py", line 20, in
import pyfwsi
ImportError: /usr/lib/python2.7/site-packages/pyfwsi.so: undefined symbol: libfwsi_notify_set_verbose

We have built the deps of libcsytem v20150101 and libbfio v20150102 before building libfwsi. Is this fixed in the git src and if so can you tag it so we can pull the tag?

Thanks

pyfwsi_items_new calls pyfwsi_items_init which raises NotImplementedError

libfwsi/pyfwsi/pyfwsi_items.c

Line 199 in b9df835

if( pyfwsi_items_init(

Also see: log2timeline/plaso#1987

File size if the size excess 0xFFFFFFFF (32 bits)

Dear,

Answer to a question of Windows Shell documentation, line 381

According to the Microsoft website, the if the filesize excess the 32 bits, then we should take the 32 LEAST SIGNIFICANT BITS

implement stand-alone usage of pyfwsi.item

Currently stand-alone usage of pyfwsi.item has issues e.g.:

IOError: pyfwsi_item_copy_from_byte_stream: unable to copy item from byte stream.
libfwsi_item_copy_from_byte_stream: invalid item.

Extension block 0xbeef0017 that does not match documentation?

It is said in the docs that this extension block has 74 bytes, but in the example shown it consists of 70 bytes.
There is one Unknown field more than in the example:

00000000: 1f  80  2e  81  43  93  37  1c  49  4a  a1  2e  4b  2d  81  0d   ....C.7. IJ..K-..
00000010: 95  6b [46  00][01  00][17  00  ef  be][00  00  00  00][01  00   .kF..... ........
00000020: 00  00][02  00  00  80][01  00  00  00][01  00  00  00][02  00   ........ ........
00000030: 00  00][00  00  00  00  00  00  00  00][02  00  00  00][00  00   ........ ........
00000040: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00   ........ ........
00000050: 00  00  00  00  00  00][14  00]                                  ........

Field 1: Size = [46 00] = 70 (table says this should be 74)
Field 2: Version = [01 00] = 1 (ok)
Field 3: Signature = [17 00 ef be] = 0xbeef0017 (ok)
Field 4: Unknown (zero) = [00 00 00 00] (ok)
Field 5: Unknown = [01 00 00 00] (ok)
Field 6: Unknown = [02 00 00 80] (ok)
Field 7: Unknown = [01 00 00 00] (ok)
Field 8: Unknown = [01 00 00 00] (ok)
Field 9: Unknown = [02 00 00 00] (ok)
Field 10: Unknwon = [00 00 00 00 00 00 00 00] (ok)
Field 11: Unknown = [02 00 00 00] (ok)
Field 12: Unknown = [00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00] (ok)
Field 13: First extension block version offset = [14 00] = 20 (ok)

Would it be possible to provide pre-built wheels?

It would be nice to have pre-built wheels for Linux, macOS and Windows for Python 3.7/3.8/3.9/3.10.
If you like I can send a patch to run this in the CI as GH action.
FWIW, I use this as an example for a project that I co-maintain https://github.com/WojciechMula/pyahocorasick/blob/master/.github/workflows/test-and-build.yml (Stolen with credits from Google's brotli repo)

New variant of a volume shell item?

I try to manually verify a windows shell item of type volume shell item.
I will use the second example in the Windows Shell Item format specification document for the explanation as the structure is identical to the one I need to parse.
Hex:
32 00 2e 80 3a cc bf b4 2c db 4c 42 b0 29 7f e9 9a 87 c6 41 1e 00 00 00 25 00 ef be 11 00 00 00 fa 66 a2 86 36 74 cf 01 2d 81 fe bc ba 9b cf 01 14 00 00 00
As described in the document I could identify the following bytes (GUID and Extension Block are assumptions from me):
32 00: Size
2e : Class Type Indicator -> Volume shell item
80: Unknown Flag
3a cc bf b4 2c db 4c 42 b0 29 7f e9 9a 87 c6 41: GUID not documented
1e 00 00 00 25 00 ef be 11 00 00 00 fa 66 a2 86 36 74 cf 01 2d 81 fe bc ba 9b cf 01 14 00: Extension Block not documented
00 00: Terminal Identifier

Am I missing something or are the GUID and Extension Block missing in the Table in Secion 3.3?

libfwsi_extension_block minimum size should be 8 not 6

Version: lnkinfo 20190922

==513==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6140000003f6 at pc 0x0000005204c3 bp 0x7ffeb5d945c0 sp 0x7ffeb5d945b8
READ of size 1 at 0x6140000003f6 thread T0
    #0 0x5204c2 in libfwsi_extension_block_copy_from_byte_stream /home/dhiraj/liblnk/libfwsi/libfwsi_extension_block.c:276:2
    libyal/liblnk#1 0x52a8f7 in libfwsi_item_copy_from_byte_stream /home/dhiraj/liblnk/libfwsi/libfwsi_item.c:1245:13
    libyal/liblnk#2 0x52e64f in libfwsi_item_list_copy_from_byte_stream /home/dhiraj/liblnk/libfwsi/libfwsi_item_list.c:334:7
    libyal/liblnk#3 0x517f94 in info_handle_link_target_identifier_fprint /home/dhiraj/liblnk/lnktools/info_handle.c:2207:7
    libyal/liblnk#4 0x518f5e in info_handle_file_fprint /home/dhiraj/liblnk/lnktools/info_handle.c:2667:6
    libyal/liblnk#5 0x519dd4 in main /home/dhiraj/liblnk/lnktools/lnkinfo.c:277:6
    libyal/liblnk#6 0x7f6705b65b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
    libyal/liblnk#7 0x41a319 in _start (/home/dhiraj/liblnk/lnktools/lnkinfo+0x41a319)

0x6140000003f6 is located 0 bytes to the right of 438-byte region [0x614000000240,0x6140000003f6)
allocated by thread T0 here:
    #0 0x4da1d0 in malloc (/home/dhiraj/liblnk/lnktools/lnkinfo+0x4da1d0)
    libyal/liblnk#1 0x517e37 in info_handle_link_target_identifier_fprint /home/dhiraj/liblnk/lnktools/info_handle.c:2159:45
    libyal/liblnk#2 0x518f5e in info_handle_file_fprint /home/dhiraj/liblnk/lnktools/info_handle.c:2667:6
    libyal/liblnk#3 0x7f6705b65b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/dhiraj/liblnk/libfwsi/libfwsi_extension_block.c:276:2 in libfwsi_extension_block_copy_from_byte_stream
Shadow bytes around the buggy address:
  0x0c287fff8020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c287fff8030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 fa
  0x0c287fff8040: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c287fff8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c287fff8060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c287fff8070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00[06]fa
  0x0c287fff8080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c287fff8090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c287fff80a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c287fff80b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c287fff80c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==513==ABORTING

To reproduce: ./lnkinfo $POC

current libfwsi-python on pypi won't install correctly

The current package seems not to install correctly through pip.

Steps to reproduce:

$ virtualenv test
Running virtualenv with interpreter /usr/bin/python2
[...]

Installing setuptools, pkg_resources, pip, wheel...done.
$ . test/bin/activate
(test) $ pip install libfwsi-python
Collecting libfwsi-python
  Using cached libfwsi-python-20180330.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-oeAmFM/libfwsi-python/setup.py", line 285, in <module>
        project_information = ProjectInformation()
      File "/tmp/pip-build-oeAmFM/libfwsi-python/setup.py", line 182, in __init__
        self._ReadConfigureAc()
      File "/tmp/pip-build-oeAmFM/libfwsi-python/setup.py", line 207, in _ReadConfigureAc
        file_object = open("configure.ac", "rb")
    IOError: [Errno 2] No such file or directory: 'configure.ac'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-oeAmFM/libfwsi-python/

Additonal LIBFWSI_ITEM_TYPES parsing.

Can we expect functionality for parsing LIBFWSI_ITEM_TYPE_DELEGATE and LIBFWSI_ITEM_TYPE_UNKNOWN_0x74 to be added?

Request for libfwsi_item_get_data to be added

How soon could we expect this function to libfwsi_item_get_data to be added to he libfwsi.h.in file?

Build instructions incorrect/outdated

The build-from-source instructions for this (and many other associated) modules appears to be incorrect/outdated.

On a vanilla Linux system the documented instructions boil down to:

% wget X.tar.gz
% tar xzvf X.tar.gz
% cd X
% ./synclibs.sh && ./autogen.sh
% ./configure --prefix=/path
% make
# make install
# ldconfig

However this does not result in the package being detected by plaso's check_dependencies.py.

There is a setuptools setup.py present, but this is not mentioned in the build instructions. Invoking this ...

# python setup.py install --prefix=/path

... results in further compilation and installation of the package such that it is now detected by plaso.

It appears that the correct build steps are something like:

% wget X.tar.gz
% tar xzvf X.tar.gz
% cd X
% ./synclibs.sh && ./autogen.sh
% ./configure --prefix=/path
% python setup.py build
# python setup.py install --prefix=/path

replace copy_from_byte_stream tests

Remove: https://github.com/libyal/libfwsi/blob/877035d4c364103135baa36384e705abeeb3e05d/tests/fwsi_test_item_list_copy_from_byte_stream.c
Replace by generated test

FTP IDList format addition

Hi Joachim !

Many thanks for documentation.

I made some research during development of my project Check Browsers' LNK, as a result I have addition to your docs: https://github.com/libyal/libfwsi/blob/master/documentation/Windows%20Shell%20Item%20format.asciidoc

    '-------------
    '3.7. FTP IDList format (must follow after IDList[0] ):
    '
    'Great thanks to Joachim Metz.
    'This specification was appended by Dragokas.
    '
    '2 bytes - size of this ID List
    '1 bytes - type = 0x61 (URI)
    '1 bytes - Flag (0x01, 0x02, 0x80 set if URI in Unicode)
    '2 bytes - *size of URI data (not including this 2 bytes itself)
    '4 bytes - unknown
    '4 bytes - unknown
    '8 bytes - timestamp (FILETIME) - first access time to the server (access != successfull authentification)
    '4 bytes - unknown (seen 0x00000000 and 0xFFFFFFFF)
    '12 bytes - unknown
    '4 bytes - unknown
    '4 bytes - size of actual URL string1 (4-bytes aligned)
    '... bytes - String1 (FTP server)
    '4 bytes - size of string2
    '... bytes - String2 (Login)**
    '4 bytes - size of string3
    '... bytes - String3 (Password)**
    '... bytes - unknown data (possible, string which represents URI protocol).
    
    '* if size of URI is 0, all next bytes of this ID List represents URL string
    '** result format is: ftp://Login:Password@Server/SubItem
          
    
    '3.7.1. FTP IDList sub item
    '
    'offset | bytes count | description
    '
    '00 | 2 bytes - size of sub item (include 2 bytes itself)
    '02 | 1 bytes - class type indicator ? (seen 0x00)
    '03 | 1 bytes - unknown (seen 0x00, 0x06, 0x09)
    '04 | 2 bytes - unknown
    '06 | 2 bytes - unknown (seen 0x05, 0x09)
    '08 | 2 bytes - unknown (seen 0x00, 0x03)
    '0A | 4 bytes - unknown (seen 0xC80, 0xC90, 0x10)
    '0E | 4 bytes - unknown (seen 0x200)
    '12 | 4 bytes - unknown
    '16 | 8 bytes - timestamp (FILETIME) - last modified time of folder on server
    '1E | 4 bytes - unknown (seen 0x0755)
    '22 | 4 bytes - unknown
    '26 | ... bytes - ANSI String (Sub Item value). Ends with 4-bytes NUL terminator
    'xx | ... bytes - Unicode String (Sub Item value). Ends with 4-bytes NUL terminator
    'xx | 2 bytes - TerminalID of IDList (0x00)

Here is a live example: target_-_copy.lnk

WBR,
Alex.

missing declaration libfwsi_item_copy_from_byte_stream

When building libregf 20150701 with libfwsi 20150614, I have the following warning:

report_handle.c:1843:6: warning: implicit declaration of function 'libfwsi_item_copy_from_byte_stream' is invalid in C99 [-Wimplicit-function-declaration]
if( libfwsi_item_copy_from_byte_stream(
^
1 warning generated.

This may lead to the same kind of bug as libyal/liblnk#2

Segfault in libfwsi_compressed_folder_values_read

Backtrace:

#0  0x00007fb8fd604666 in libfwsi_compressed_folder_values_read () from /usr/lib/libfwsi.so.1
#1  0x00007fb8fd60bbde in libfwsi_item_copy_from_byte_stream () from /usr/lib/libfwsi.so.1
#2  0x00007fb8fd60d27a in libfwsi_item_list_copy_from_byte_stream () from /usr/lib/libfwsi.so.1
#3  0x00007fb8fd8facd3 in pyfwsi_item_list_copy_from_byte_stream () from /usr/lib/python2.7/dist-packages/pyfwsi.so