viper-framework / viper Goto Github PK
View Code? Open in Web Editor NEWBinary analysis and management framework
License: Other
Binary analysis and management framework
License: Other
Have read the install howto: https://nex.sx/blog/index.html on a fresh blank 3.2.0-4-686-pae #1 SMP Debian 3.2.57-3 i686 GNU/Linux system.
Failed with:
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c pydeep.c -o build/temp.linux-i686-2.7/pydeep.o
pydeep.c:1:20: fatal error: Python.h: No such file or directory
(of course, you need to have python-dev installed)
should be added to you
"apt-get install gcc python-socksipy" command --> perfect
With fully updated viper i get the following message when trying to do yara scan:
"PreprocessError: data/yara/index.yara:0:Invalid file extension '.yara'.Can only include .yar"
misc shell 9fc0fd8725f6cc85ac5ccca266d5130c > reports malwr
You need to specify a valid username/password, login now? [y/N] y
Username: krmaxwell
Password:
[!] The command reports raised an exception:
Traceback (most recent call last):
File "/mnt/viper/viper/core/ui/console.py", line 176, in start
module.run()
File "/mnt/viper/modules/reports.py", line 181, in run
self.malwr()
File "/mnt/viper/modules/reports.py", line 97, in malwr
verify=False
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 425, in post
return self.request('POST', url, data=data, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 387, in send
raise Timeout(e)
Timeout: HTTPSConnectionPool(host='malwr.com', port=443): Read timed out.
clamav
gives the error:
[!] Check permissions of the binary folder, Can't open file or directory
binary folder permissions seems to be ok, even changed to 777 and still same error.
This only happens if the file was submitted via the api.py
Env: cygwin-x86_64
touch zero-length.test
./viper.py
> open -f zero-length.test
sessions -l
| 1 | | | 2014-07-14 01:04:16 | Yes |
| 1 | zero-length.test | __HASH__ | 2014-07-14 01:04:16 | Yes |
store
[!] The command store raised an exception: Traceback (most recent call last): File "/cygdrive/c/Users/user/viper/viper/core/ui/console.py", line 170, in start self.cmd.commands[root]['obj'](*args) File "/cygdrive/c/Users/user/viper/viper/core/ui/commands.py", line 469, in cmd_store if add_file(__sessions__.current.file, arg_tags): File "/cygdrive/c/Users/user/viper/viper/core/ui/commands.py", line 401, in add_file if get_sample_path(obj.sha256): File "/cygdrive/c/Users/user/viper/viper/core/storage.py", line 33, in get_sample_path path = os.path.join(__project__.get_path(), 'binaries', sha256[0], sha256[1], sha256[2], sha256[3], sha256) IndexError: string index out of range
It is more a cosmetic change, but it would make the developments easier: what do you think of splitting viper/core/ui/commands.py in multiple files (one per command)?
I can do it and send a pull requests but I would prefer to know what you think about it before spending a couple of hours doing the changes.
Wondering if that is useful, but of course wine needs to be installed
If useful I will add other OfficeMalScanner options tmrw
Wouldn't it be great to have a compressed Up- & Download functionality?
Especially when your host has AV installed and you use viper as remote repository.
This would allow to save a tremendous amount of space.
Lets say md5 + crc32 + sha1 + sha256 + sha512 take 272 bytes to store as ASCII strings, but only 30 bytes as binary.
you can strengthen xor searching even
http://c0defreak.blogspot.in/2013/08/breaking-single-character-xor-cipher_9663.html
small blogpost i wrote about it
Command:
shell > find latest -h
[!] The command find raised an exception:
Traceback (most recent call last):
File "/home/jaegeral/viper_test/viper/viper/core/ui/console.py", line 170, in start
self.cmd.commands[root]'obj'
File "/home/jaegeral/viper_test/viper/viper/core/ui/commands.py", line 584, in cmd_find
for item in items:
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2223, in iter
context = self._compile_context()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2993, in _compile_context
*_self._select_args
File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/expression.py", line 306, in select
*_kwargs)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/expression.py", line 4770, in init
_SelectBase.init(self, **kwargs)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/expression.py", line 4299, in init
self._limit = util.asint(limit)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 648, in asint
return int(value)
ValueError: invalid literal for int() with base 10: '-h'
Expected:
help message or hint to only use unsigned int
even better:
increase off argument check in total
Tried to iplort a file with japanese / chinese chars as filename, the file is somewhat imported, but not storred completely, re-import is not possible.
_
(_)
| | | | | _ | ___ |/ )
\ V /| | || | __| |
/ || /|___)| v1.1-dev
||
You have 0 files in your default repository
viper > store -f ./to_upload/
[+] Stored file "何應对国.exe" to /home/xxx/viper2/viper/binaries/2/1/c/9/21c9a43d22cd02babb34b45a9defb881ae8228f0d034a0779b1321e851cad6a4
viper > find latest
+---+------+------+-----+------+------------+
| # | Name | Mime | MD5 | Tags | Created At |
+---+------+------+-----+------+------------+
+---+------+------+-----+------+------------+
viper > exit
xxx@zoo:~/viper2/viper$ ./viper.py
_
(_)
| | | | | _ | ___ |/ )
\ V /| | || | __| |
/ || /|___)| v1.1-dev
||
You have 0 files in your default repository
viper >
xxx@zoo:/viper2/viper$ md5sum to_upload/*/viper2/viper$ ls -hall to_upload/
76931daaa7c7f134531e7938b3a9052c to_upload/何應对国.exe
xxx@zoo:
total 12K
drwxr-xr-x 2 xxx xxx 4.0K Aug 11 16:25 .
drwxr-xr-x 9 xxx xxx 4.0K Aug 11 16:25 ..
-rw-r--r-- 1 xxx xxx 6 Aug 11 16:25 何應对国.exe
Introduce preliminary module for parsing PDF files, extracting streams, extracting embedded content and any other feature that might come useful.
File exists already --> $SENDER - - [14/May/2014 09:17:32] "POST /file/add HTTP/1.1" 500 751
is returning only a "unable to store file" which is not clear enough.
Will have a look at it later
The 'Switched to default MD5 and Mime' Update breaks an existing Database.
If you want to migrate your existing DB Just open up in a sqlite editor and add a new column to the malware table. Name = mime, Type = String(255), nullable=True.
https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager/
This firefox add on is pretty handy if you don't have a sqlite manager installed.
If I find out how to do it in sqlalchemy ill throw together a migration script that will add a new column and populate the mime type.
Command attempted: rat -f blackshades
[!] The command rat raised an exception:
Traceback (most recent call last):
File "/usr/local/viper/viper/core/ui/console.py", line 176, in start
module.run()
File "/usr/local/viper/modules/rat.py", line 97, in run
self.get_config(value)
File "/usr/local/viper/modules/rat.py", line 41, in get_config
config = module.config(sessions.current.file.data)
File "/usr/local/viper/modules/rats/blackshades.py", line 75, in config
raw_config = config_extract(data)
File "/usr/local/viper/modules/rats/blackshades.py", line 50, in config_extract
return s
UnboundLocalError: local variable 's' referenced before assignment
File Not found when attempting to open -f any filename that contains a space.
Tab completion works to find the file but viper doesn't open a session.
shell > open -f /home/thehermit/vipertest/split filename.doc
[!] File not found
shell > open -f ../../vipertest/split filename.doc
[!] File not found
shell > open -f ../../vipertest/split\ filename.doc
[!] File not found
There are two identical files: foo.bar
and bar.foo
.
> open -f foo.bar
> store
> open -f bar.foo
> store
[!] Skip, file "foo.bar" appears to be already stored
Feature request:
It would be great to move the current session into a specific project in order to keep things organized.
The documented behaviour of close
command seems strange, because if a session gets reopened, the list of opened sessions will get difficult to maintain over time. Could you explain more?
Getting "[!] Missing dependency, install yara" even tough yara is already installed and working properly.
Host:
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty
Yara version 3.1.
My suggestion is to separate db access to a thread, create multiple workers with multiprocessing to do the hard work (hash calculations mostly ?).
The idea would be to try to speed up import of big to large collections (100k to millions)
Yara scan on a dll with some custom rules.
[!] The command yara raised an exception:
Traceback (most recent call last):
File "/opt/viper/viper/core/ui/console.py", line 176, in start
module.run()
File "/opt/viper/modules/yarascan.py", line 141, in run
self.scan()
File "/opt/viper/modules/yarascan.py", line 110, in scan
print(table(header=header, rows=rows))
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 240, in __str__
return self.__unicode__().encode(self.encoding)
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 243, in __unicode__
return self.get_string()
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 987, in get_string
formatted_rows = self._format_rows(rows, options)
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 942, in _format_rows
return [self._format_row(row, options) for row in rows]
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 939, in _format_row
return [self._format_value(field, value) for (field, value) in zip(self._field_names, row)]
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 890, in _format_value
return self._unicode(value)
File "/usr/local/lib/python2.7/dist-packages/prettytable.py", line 181, in _unicode
value = unicode(value, self.encoding, "strict")
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte
I believe this is the offending output from rows.append([match.rule, string[1], string[0], string[2]])
['Peid2yar_Microsoft_Visual_Cpp_70_DLL_', '$0', 228782L, 'U\x8b\xecS\x8b]\x08V\x8bu\x0c\x85\xf6W\x8b}\x10u\t\x83=$\xa1Mu\x00\xeb&\x83\xfe\x01']
More than anything, we should first experiment how doable it is to use the existing code structure with other user interfaces, like a GUI, web GUI or CLI.
I have the following feature request.
Perhaps you can make some setting configurable in a ~/.viper file
Things I have modified are the database URL (I point mine to PostgreSQL)
a fixed location for the binaries
a fixed location for the data
perhaps you could add malwr and anubus username password or API key
[!] The command store raised an exception:
Traceback (most recent call last):
File "/home/a/malware/viper/viper/core/ui/console.py", line 170, in start
self.cmd.commands[root]'obj'
File "/home/a/malware/viper/viper/core/ui/commands.py", line 337, in cmd_store
add_file(file_obj, arg_tags)
File "/home/a/malware/viper/viper/core/ui/commands.py", line 274, in add_file
if get_sample_path(obj.sha256):
File "/home/a/malware/viper/viper/core/storage.py", line 33, in get_sample_path
path = os.path.join(project.get_path(), 'binaries', sha256[0], sha256[1], sha256[2], sha256[3], sha256)
IndexError: string index out of range
Hola,
Just a silly typo I noticed in modules/office.py around line 226:
# Put it all together.
rows = [
['Summery Information', has_summary],
['Word', is_word],
That should be "Summary Information"
Thanks for publishing this cool tool!
Introduce preliminary module for performing analysis of DOC/XLS/PPT documents.
In order to store and fetch data in relation to projects the API needs to be rewritten.
That might be challanging due to the fact that each project has own databases.
Opinions?
This applies more to modules like strings or imports where the output can range potentially be hundreds or thousands of lines
Could it be possible to introduce a pagination / more function or just have the ability to redirect > to file.
Im happy to work on it just wonder which option would be preferred.
Instead of having a colored terminal prompt, it looks like this:
[36mshell > [0m
However, the string "You have 0 files in your default repository" displays correctly (purple)
Environment:
OS: OSX 10.9.4
Python: 2.7.8
Terminal: iTerm2 & Terminal.app
TERM=xterm
Happy to help debug this if there's anymore data I can provide.
Peepdf generates errors that are written in to an errors.txt file in the root of viper. It looks like an issue with relative imports.
Example show here
Traceback (most recent call last):
File "/home/thehermit/GitHub/viper/modules/peepdf/JSAnalysis.py", line 75, in analyseJS
code = jsbeautifier.beautify(code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 93, in beautify
return b.beautify(string, opts)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 210, in beautify
self.input = self.unpack(s, opts.eval_code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 245, in unpack
import jsbeautifier.unpackers as unpackers
ImportError: No module named jsbeautifier.unpackers
Traceback (most recent call last):
File "/home/thehermit/GitHub/viper/modules/peepdf/JSAnalysis.py", line 75, in analyseJS
code = jsbeautifier.beautify(code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 93, in beautify
return b.beautify(string, opts)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 210, in beautify
self.input = self.unpack(s, opts.eval_code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 245, in unpack
import jsbeautifier.unpackers as unpackers
ImportError: No module named jsbeautifier.unpackers
Traceback (most recent call last):
File "/home/thehermit/GitHub/viper/modules/peepdf/JSAnalysis.py", line 75, in analyseJS
code = jsbeautifier.beautify(code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 93, in beautify
return b.beautify(string, opts)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 210, in beautify
self.input = self.unpack(s, opts.eval_code)
File "/home/thehermit/GitHub/viper/modules/peepdf/jsbeautifier/__init__.py", line 245, in unpack
import jsbeautifier.unpackers as unpackers
ImportError: No module named jsbeautifier.unpackers
Assume an empty database, but an binaries folder with files and you want to add the binaries folder with
store --folder binaries
That will cause the store method to skip all files because they are already in filesystem.
Fix proposal will follow in a minute
Feature request:
Searching should be possible globally through all databases
Enhancement request for shell command and arguments autocompletion. Since open
interacts with a file system, it would be nice if it could complete file/directory names too.
In case users are downloading the release package instead of cloning from GitHub, we might want to have an update utility to update the framework itself and download the new modules.
We should brainstorm on what would be the best approach for this and perhaps look up examples from other projects.
Try to either use the store command in a session, or store -f folder to load a batch of files end with the following error.
[!] The command store raised an exception:
Traceback (most recent call last):
File "/home/vektor/viper/viper/core/ui/console.py", line 195, in start
self.cmd.commands[root]'obj'
File "/home/vektor/viper/viper/core/ui/commands.py", line 489, in cmd_store
if add_file(sessions.current.file, arg_tags):
File "/home/vektor/viper/viper/core/ui/commands.py", line 423, in add_file
new_path = store_sable(obj)
NameError: global name 'store_sable' is not defined
Idea: Similar to the cuckoo module sending a sample to another viper instance providing the same tags.
Mockup:
def usage():
print("usage: viper_remote [-H=host] [-p=port] -tags=tag1,tag2,tag3")
def help():
usage()
print("")
print("Options:")
print("\t--help (-h)\tShow this help message")
print("\t--host (-H)\tSpecify an host (default: localhost)")
print("\t--port (-p)\tSpecify a port (default: 8080")
print("\t--tags (-t)\tSpecify tags (default: keep tags")
print("")
example:
viper_remote -H 1.2.3.4 -p 8080
What do you guys think about it?
It would be nice to have a command like:
http://$viperhiost/file/$md5/sendtosandbox/$sandboxurl
Does such a feature make sense?
Hello, I want to work with viper under archlinux. But I get the following stacktrace:
Traceback (most recent call last):
File "viper.py", line 17, in <module>
c = console.Console()
File "/opt/viper/viper/core/ui/console.py", line 45, in __init__
self.cmd = Commands()
File "/opt/viper/viper/core/ui/commands.py", line 25, in __init__
self.db = Database()
File "/opt/viper/viper/core/database.py", line 146, in __init__
Base.metadata.create_all(self.engine)
File "/usr/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 3352, in create_all
tables=tables)
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1609, in _optional_conn_ctx_manager
with self.contextual_connect() as conn:
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1799, in contextual_connect
self.pool.connect(),
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 338, in connect
return _ConnectionFairy._checkout(self)
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 641, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 440, in checkout
rec = pool._do_get()
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1055, in _do_get
return self._create_connection()
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 285, in _create_connection
return _ConnectionRecord(self)
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 411, in __init__
self.connection = self.__connect()
File "/usr/lib/python2.7/site-packages/sqlalchemy/pool.py", line 537, in __connect
connection = self.__pool._creator()
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
connection_invalidated=invalidated
File "/usr/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (OperationalError) unable to open database file None None
I checked the requirements. Every requirement is installed.
It would be nice to have better support for Cuckoo features, based on what the REST API can do:
I already implemented this in my fork (https://github.com/haellowyyn/viper). It works pretty well so far. However, the last two mentioned use cases need hardcoded paths for the browser binary and the PCAP viewer. This would greatly benefit from a config file (#63).
tried the following command:
shell $FILE > tags add foo,bar
[!] You need to specify an option, either add or delete
that should be a help text like the following:
[!] You need to specify an option, either add or delete [-a=tags] [-d=tag]
would have saved me 30 seconds ;-)
> notes -h
Act: ...
--help (-h) Show this help message (-h) --list (-h)
Exp: ...
--help (-h) Show this help message (-h) --list (-l)
Env: cygwin-x86_64
Have a project test
with no stored files. Starting from directory ~/viper
. There is a file ~/viper/foo.bar
> open -f foo.bar
> note -l
[!] The command notes raised an exception: Traceback (most recent call last): File "~/viper/viper/core/ui/console.py", line 170, in start self.cmd.commands[root]['obj'](*args) File "~/viper/viper/core/ui/commands.py", line 276, in cmd_notes notes = Database().find(key='sha256', value=__sessions__.current.file.sha256)[0].note IndexError: list index out of range
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.