danielwatson3141 / coderid Goto Github PK

There are many design decisions that have been made throughout this project that I would love to see built in as configurable options:

-Whether to consider comments
-Whether to consider whitespace
-Whether to consider string literals as unique tokens
-Toggle various feature sets: Tokens, chars, unigrams

Feature extraction has efficiency problems:

Takes too much memory
Takes too long due to duplicate work

Would love to see such things reduced to speed up iteration.

Root dir of repo is getting cluttered. I would like source files moved into an src directory and the necessary changes made to ensure everything stays working.

Currently, our tool is built around a RFC (Random Forest Classifier). It would be of great value if we could run other algorithms over our dataset to see what we can come across. I would like to see our RFC code abstracted out of the main program and brought into a more genericized class so we can attempt other classification techniques.

I would like the ability to normalize the whitespace of all code in a GPS. That is:

Any "unnecessary" spaces, those which do not effect the syntax of the code should be removed.
Any instances of multiple, consecutive, identical, white-space characters should be reduced to single instances of the character (except for tabs).
Newlines should only occur:

1. After semi-colons

2. After opening brackets {

3. Before and after closing brackets }

4. After colons : as in switch cases and labels.

5. After ) in for, if, and while blocks with no brackets

6. I am open to other logical instances.

All other newlines should be stripped out.

For example,
<-------------------------------------------------------------------->
for( int i=0; i<10; i++) would reduce to for(int i=0;i<10;i++). Note the only remaining space comes after the "int" as it is syntactically necessary.
<-------------------------------------------------------------------->
int i=0 reduces to int i=0 (got rid of double space)
<-------------------------------------------------------------------->
for(int i=0;i<10;i++)
foo()
remains unchanged
<-------------------------------------------------------------------->
code

code

would be reduced to:
code
code
as newlines count.
<-------------------------------------------------------------------->
The goal here is to propose and validate one step in an obfuscation process. That is, a way to prevent stylometric attacks without obscuring the meaning of the code.

coderID.py is getting too cluttered with various classification problem code. I would like this refactored out to a new file. The difficulty is that due to laziness, there are a lot of self references in said functions which will need to be refactored out to parameters and so forth.

I would like a command added that plots (or summarizes) the following information for inclusion in the paper:

Summary:
Total Number of functions
Total LOC
AVG function length (LOC)

Plot:
Function length (LOC) - Histogram

Our current feature reduction method isn't giving very consistent results. The threshold seems to be highly variable depending on the size of the dataset.

I would like to see recursive feature elimination implemented as this would be a more flexible approach.

Idea:
set lambda = .1 (some small fraction)
while score is not decreasing:
score model by cross validation
remove lamda * featureSetSize least important features
return featureSet

Time will be the biggest obstacle here. Has to be done efficiently.