This repository contains the project-1 of Data Science Practicum (CSCI 8360) course at the University of Georgia, Spring 2019.

This project uses data from the Microsoft Malware Classification Challenge, which consists of nearly half a terabyte of uncompressed data. There are 9 classes of malware, and each instance of malware has one, and only one, category.

We built a Random Forest classifier which achieves an accuracy of 99.0077%.

Please refer to the Wiki for more details on our approach.

The following instructions will assist you get this project running on your local machine for developing and testing purpose.

• Apache Spark 2.3.2
• Python 3.7.2
• Anaconda

Run the random forest classifier. The data is automatically pulled from the internet. Byte count or Header count features can be selected (or both).

$ python low_mem_rf.py [Dataset] [Features] [Number of trees] [Maximum depth]

OR

$ spark-submit low_mem_rf.py [Dataset] [Features] [Number of trees] [Maximum depth]

All parameters are optional.

Dataset - s for small dataset, l for large dataset. Default: l.

Features - 11 for byte and header counts, 10 for only byte counts, 01 for only header counts. Default: 11

Number of trees - Any integer larger than 1. Default: 40

Maximum Depth - Any integer between 1 and 30 (inclusive). Default: 23

The prediction will be saved to disk in the current directory and named submit<Number of trees><Maximum depth>.txt By default it will be saved as submit4023.txt

Run the random forest classifier with .dll and opcode count [Optional].

$ spark-submit dllandOpcodefeatutes.py [path-to-opcodeslist.txt]

(Ordered alphabetically)

• Anuj Panchmia
• Sumer Singh
• Vishakha Atole

See the CONTRIBUTORS.md file for details.

This project is licensed under the MIT License - see the LICENSE file for details