srinivas11789 / pcapxray Goto Github PK

:snowflake: PcapXray - A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram including device identification, highlight important communication and file extraction

License: GNU General Public License v2.0

Python 97.09% Dockerfile 1.21% Shell 1.70%

forensics pcap network python computer-forensics tor-traffic forensic-analysis cybersecurity network-diagram tor

pcapxray's Introduction

PcapXray

A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram including device identification, highlight important communication and file extraction

PcapXray Design Specification

Wiki has more help too.

Goal:

Given a Pcap File, plot a network diagram displaying hosts in the network, network traffic, highlight important traffic and Tor traffic as well as potential malicious traffic including data involved in the communication.

Problem:

Investigation of a Pcap file takes a long time given initial glitch to start the investigation
Faced by every forensics investigator and anyone who is analyzing the network
Location: https://github.com/Srinivas11789/PcapXray

Solution: Speed up the investigation process

Make a network diagram with the following features from a Pcap file Tool Highlights:
Network Diagram – Summary Network Diagram of full network
Information:
- Web Traffic with Server Details
- Tor Traffic
- Possible Malicious traffic
- Data Obtained from Packet in Report – Device/Traffic/Payloads
- Device Details

Tool Image:

Components:

Network Diagram
Device/Traffic Details and Analysis
Malicious Traffic Identification
Tor Traffic
GUI – a gui with options to upload pcap file and display the network diagram

Setup

Python 3

apt install python3-pip
apt install python3-tk
apt install graphviz
apt install python3-pil python3-pil.imagetk
pip3 install -r requirements.txt
python3 Source/main.py

( Make sure to escalate privilege to allow file creations - Run with sudo )

For MAC:

brew install graphviz

Python 2

apt install python-tk
apt install graphviz
pip install -r requirements.txt
python Source/main.py

( Make sure to escalate privilege to allow file creations - Run with sudo )

Python Libraries Used: - All these libraries are required for functionality

Tkinter and TTK – Install from pip or apt-get – Ensure Tkinter and graphviz is installed (Most Linux contain by default)
- apt install python-tk
- apt install graphviz
- apt install python3-tk (for python3 support)
- Sometimes ImageTk errors are thrown in python3 env --> use apt install python3-pil python3-pil.imagetk
All these are included in the requirements.txt file
- Scapy – rdpcap to read the packets from the pcap file
- Ipwhois – to obtain whois information from ip
- Netaddr – to check ip information type
- Pillow – image processing library
- Stem – tor consensus data fetch library
- pyGraphviz – plot graph
- Networkx – plot graph
- Matplotlib – plot graph (not used as of now)

Demo

Getting started:

Clone the repository
pip install -r requirements.txt
python Source/main.py

Additional Information:

Tested on Linux
Options for Traffic include - Web (HTTP and HTTPS), Tor, Malicious, ICMP, DNS

Challenges:

Unstability of the TK GUI:
- Decision on the GUI between Django and TK, settled upon tk for a simple local interface, but the unstability of the tk gui caused a number of problems
Graph Plotting:
- Plotting a proper network graph which is readable from the data obtained was quite an effort, used different libraries to arrive at one.
Performance and Timing:
- The performance and timing of the total application was a big challenge with different data gathering and output generation

Known Bugs:

Memory Hogging
- Sometimes memory hogging occurs when lower RAM is present in the system as the data stored in the memory from the pcap file is huge
- Should be Fixed by moving data into a database than the memory itself
Race Condition
- Due to mainloop of the TK gui, other threads could undergo a race condition
- Should be fixed by moving to a better structured TK implementation or Web GUI
Tk GUI Unstability:
- Same reason as above
Code:
- clumsy and unstructured code flow
Current Fix in rare occasions: If any of the above issue occurs the progress bar keeps running and no output is generated, a restart of the app would be required.

Docker Containers of PcapXray

Dockerfile present in the root folder was used to build images
Already built docker images are found at dockerhub
- srinivas11789/pcapxray-1.0
- srinivas11789/pcapxray-2.2
Performing the steps in run.sh file manually would work to launch the tool via docker (I can help with errors)
Running run.sh scripts is an attempt to automate (would not work 100 percent)
- tested on mac and linux - will be better soon!...

Immediate Future Tasks: (Target: 3.0)

Clean up code (beautify code base from being a prototype)
Report generation on unique folders for all assets of a packet capture
Suspicious activity detection
Support more pcap reader engine
Traffic support: ICMP, DNS
Known file type detection and Extract
Python2 and Python3
Interactive map

Future:

Structured and clean code flow
Change the database from JSON to sqlite or prominent database, due to memory hogging
Change fronend to web based such as Django
Make the application more stable
More protocol support
Clean up code

Credits:

Thanks for making it better,
- Professor Marc Budofsky
- Kevin Gallagher
Thanks for all the dependent libraries used
Logo created with logomakr.com and www.inkscape.org

Just for Security Fun!

pcapxray's People

Contributors

Stargazers

Watchers

Forkers

securycore johnjohnsp1 techlord-rce s1gh chubbymaggie grammarknotsi fo0nikens memogarcia majid03 cephurs bikerdroid mmilleror puppycodes shalomz thegreenjedi expertasif kimkucheol shaunstanislauslau jdiazmx evil5hadow weeshlow goulahyane marclachapelle sukelluskello russhaun chunchila hybridious cclauss asorourx androidjaijai ayooshagarwal initpwn cneskey r0ug3 mlynchcogent r3p3r bigttys0 badexception alimp5 waldonhendricks szliuyujie maleus uldeepak heysadboy ksmaheshkumar neversatisfied m41doror hardware-forest-utopia hotelzululima kakuye akz747 isbang jilir heikipikker hack404-007 sunflowcoco methos2016 an2ni nanogithub yeahbytes shaglider noodled imagineagents preethi900 prodject anonymous369 mardoksp mayuan2018 ro9ueadmin rahulsunny51 ith4cker killvxk yanggongwang keyman9848 h3len xfaceme km269 songofhack ii0 5up3rc chingru naozibuhao aa42vecks1n shentao17792 indigos33k3r putaodoudou medresearcher marciopocebon jinnu92 eduriel the-l0st-technocrat fxcebx sherouxi omemishra kimba4u larrycameron80 c002 seabreg seanyu-git tim1512

pcapxray's Issues

no visualization pf pcap file

Problem with PyQt

Hello,

I am new to PyQt. I need some helps from anyone who is good at PyQt.
I really appreciate it.
I am doing a python GUI to view pcap file.
When I clicked on Open Button(which I add in GUI file), I want QTablewidget to display all row of PCAP file. Can anyone enlighten me on this point?

GUI interface

I only got this when i try to analyse a file.

Didn't understand the issue

Error on Start

Traceback (most recent call last):
  File "Source/main.py", line 12, in <module>
    import userInterface
ImportError: No module named userInterface

I get this error on macOS 10.13.5, running Python 2.7.15. This error occurs after running pip install -r requirements.txt.

Looks like a very cool project, hoping there's an easy package to install that makes it mac compatible, but I'm not seeing anything by googling around.

Flatpak/snap

Can this be packaged as a flatpak and/or snap?

Unit test for PcapXray

The project works with 8 modules < reference source/Module >

Creating issue to accelerate/track test frame building of each module
The test folder has a very basic example to start with.
Feel free to make a pull request for the test cases

libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory

works after apt-get install libgtk2.0-0

Large graph with more nodes makes graph clumsy

TypeError exception from pcap_reader

TypeError: __str__ returned non-string (type bytes)

does not affect function, but fails due to handling bytes with encrypted payload
related source: here

startup error

Hi.

This looks really cool but I get this error on a fresh install:

 (master *) Source $ python main.py 
Traceback (most recent call last):
  File "main.py", line 12, in <module>
    import userInterface
  File "Module/userInterface.py", line 11, in <module>
    from PIL import Image,ImageTk
ImportError: cannot import name ImageTk

I'm running on ubuntu (I also get the same error on Fedora 26). The PIL package doesn't have ImageTK. Is there a certain version I need?

 (master *) Source $ python --version
Python 2.7.14
(master *) Source $ python
Python 2.7.14 (default, Sep 23 2017, 22:06:14) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import PIL
>>> help(PIL)
Help on package PIL:

NAME
    PIL

FILE
    /usr/lib/python2.7/dist-packages/PIL/__init__.py

DESCRIPTION
    # The Python Imaging Library.
    # $Id$
    #
    # package placeholder
    #
    # Copyright (c) 1999 by Secret Labs AB.
    #
    # See the README file for information on usage and redistribution.
    #

PACKAGE CONTENTS
    BdfFontFile
    BmpImagePlugin
    BufrStubImagePlugin
    ContainerIO
    CurImagePlugin
    DcxImagePlugin
    DdsImagePlugin
    EpsImagePlugin
    ExifTags
    FitsStubImagePlugin
    FliImagePlugin
    FontFile
    FpxImagePlugin
    FtexImagePlugin
    GbrImagePlugin
    GdImageFile
    GifImagePlugin
    GimpGradientFile
    GimpPaletteFile
    GribStubImagePlugin
    Hdf5StubImagePlugin
    IcnsImagePlugin
    IcoImagePlugin
    ImImagePlugin
    Image
    ImageChops
    ImageCms
    ImageColor
    ImageDraw
    ImageDraw2
    ImageEnhance
    ImageFile
    ImageFilter
    ImageFont
    ImageGrab
    ImageMath
    ImageMode
    ImageMorph
    ImageOps
    ImagePalette
    ImagePath
    ImageQt
    ImageSequence
    ImageShow
    ImageStat
    ImageTransform
    ImageWin
    ImtImagePlugin
    IptcImagePlugin
    Jpeg2KImagePlugin
    JpegImagePlugin
    JpegPresets
    McIdasImagePlugin
    MicImagePlugin
    MpegImagePlugin
    MpoImagePlugin
    MspImagePlugin
    OleFileIO
    PSDraw
    PaletteFile
    PalmImagePlugin
    PcdImagePlugin
    PcfFontFile
    PcxImagePlugin
    PdfImagePlugin
    PixarImagePlugin
    PngImagePlugin
    PpmImagePlugin
    PsdImagePlugin
    PyAccess
    SgiImagePlugin
    SpiderImagePlugin
    SunImagePlugin
    TarIO
    TgaImagePlugin
    TiffImagePlugin
    TiffTags
    WalImageFile
    WebPImagePlugin
    WmfImagePlugin
    XVThumbImagePlugin
    XbmImagePlugin
    XpmImagePlugin
    _binary
    _imaging
    _imagingcms
    _imagingft
    _imagingmath
    _imagingmorph
    _tkinter_finder
    _util
    _webp
    features

DATA
    PILLOW_VERSION = '4.1.1'
    VERSION = '1.1.7'
    __version__ = '4.1.1'

VERSION
    4.1.1

(END)

Infinite progress bar problem

Issue: Sometimes the progress bar keeps loading even when the backend thread has finished its job.
Temperory Fix: Restarting the tool solves this error. ( bad fix )

imagetk lib not installing

tried this command but no use
apt install python3-pil python3-pil.imagetk

dont know what to do and how to resolve it
python version is 3.7.5

startup error

Hello,
What is this error:

┌──(root㉿kali)-[~/PcapXray]
└─# python3 Source/main.py
Interactive graph in app wont work as python version/platform is not supported (will launch in default browser)
Traceback (most recent call last):
File "/root/PcapXray/Source/main.py", line 52, in
main()
File "/root/PcapXray/Source/main.py", line 41, in main
base = Tk()
File "/usr/lib/python3.10/tkinter/init.py", line 2299, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

Numerous traffic reduces the graph readability

Problem:

Numerous traffic reduces the graph readability

Possible Solutions:

Provide Zoom to explore the graph
Work on resolution and image resizing to make graph clear and super readable

Feasibility or Challenges:
Under work

Tool could be invoked only from 'PcapXray/Source' directory

Issue: Tool could be invoked only from 'PcapXray/Source' directory
Possible Solution: Set generic PATH for Modules in sys.path.insert such that it could be invoked from any directory

Custom path for report directory and some corner cases

Issue: Report should be output to any custom directory based on user choice

Present condition - Report is generated in the same folder as main.py

Corner Cases:

Handle case when Report directory already exists in the given path. Sometimes the graph is not loaded when the stale Report directory from the previous run for the same pcap file exists.
WorkAround - Manually cleanup report directory before every pcap run
Fix - Load graph is it already exists

Scapy Issue

WARNING:scapy.runtime:more Calling str(pkt) on Python 3 makes no sense!

Crash on Start

Running python main.py on macOS 10.13.5 with Python version 2.7.15 produces error after PcapXray crashes during startup:

→ python main.py
2018-06-14 15:29:39.095 Python[26399:2499358] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7f9d3963aad0
2018-06-14 15:29:39.097 Python[26399:2499358] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7f9d3963aad0'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff50c706bb __exceptionPreprocess + 171
	1   libobjc.A.dylib                     0x00007fff77e7f942 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff50d079e4 -[NSObject(NSObject) doesNotRecognizeSelector:] + 132
	3   CoreFoundation                      0x00007fff50be8183 ___forwarding___ + 1443
	4   CoreFoundation                      0x00007fff50be7b58 _CF_forwarding_prep_0 + 120
	5   Tk                                  0x00007fff5d35a318 TkpInit + 467
	6   Tk                                  0x00007fff5d2d9252 Tk_Init + 1710
	7   _tkinter.so                         0x00000001021b6bc0 Tcl_AppInit + 84
	8   _tkinter.so                         0x00000001021b658b Tkinter_Create + 1061
	9   Python                              0x0000000101e58367 PyEval_EvalFrameEx + 19822
	10  Python                              0x0000000101e533e2 PyEval_EvalCodeEx + 1562
	11  Python                              0x0000000101df9ce4 function_call + 327
	12  Python                              0x0000000101ddc160 PyObject_Call + 97
	13  Python                              0x0000000101de6a0d instancemethod_call + 163
	14  Python                              0x0000000101ddc160 PyObject_Call + 97
	15  Python                              0x0000000101e5be1c PyEval_CallObjectWithKeywords + 159
	16  Python                              0x0000000101de4cef PyInstance_New + 123
	17  Python                              0x0000000101ddc160 PyObject_Call + 97
	18  Python                              0x0000000101e58354 PyEval_EvalFrameEx + 19803
	19  Python                              0x0000000101e5c489 fast_function + 196
	20  Python                              0x0000000101e5826a PyEval_EvalFrameEx + 19569
	21  Python                              0x0000000101e533e2 PyEval_EvalCodeEx + 1562
	22  Python                              0x0000000101e52dc2 PyEval_EvalCode + 32
	23  Python                              0x0000000101e747ab run_mod + 49
	24  Python                              0x0000000101e74852 PyRun_FileExFlags + 130
	25  Python                              0x0000000101e743d4 PyRun_SimpleFileExFlags + 706
	26  Python                              0x0000000101e85f54 Py_Main + 3136
	27  libdyld.dylib                       0x00007fff78a96015 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
[1]    26399 abort      python main.py

Exception: Python version not supported: 3.8.2

Python 3.8.2

sudo python3 Source/main.py
Traceback (most recent call last):
File "Source/main.py", line 12, in
from cefpython3 import cefpython as cef
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cefpython3/init.py", line 64, in
raise Exception("Python version not supported: " + sys.version)
Exception: Python version not supported: 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
[Clang 6.0 (clang-600.0.57)]

NSOpenPanel: NSInternalInconsistencyException with tk ui in mac os

2022-02-15 09:17:23.967 Python[9628:502841] *** Assertion failure in -[NSOpenPanel beginServicePanel:asyncExHandler:], NSVBOpenAndSavePanels.m:1907
2022-02-15 09:17:24.004 Python[9628:502841] -[NSSavePanel beginWithCompletionHandler:]_block_invoke caught non-fatal NSInternalInconsistencyException '<NSOpenPanel: 0x7fcecb72f600> is attempting to advance this Open/Save panel to run phase while another self.advanceToRunPhaseCompletionHandler is in waiting for a previous attempt. An Open/Save panel cannot start to advance more than once.' with user dictionary {
...

Interactive graph does not work if the full file path name has more than one "."

pyvis has a check for assert len(name.split(".")) == 2 which fails when a filename with full path containing "."

Hacky Fix:

Have no "." when along the folders where PcapXray is located.

Improve Readme.md file

Can you improvize the Readme.md file with End to End steps on how to use this tool?

Report generation with error in python2 for some strings

Python2 is not good with some strings leading to errors with report