danielwatson3141 / coderid Goto Github PK
View Code? Open in Web Editor NEWPrototype source code authorship attribution tool.
Prototype source code authorship attribution tool.
There are many design decisions that have been made throughout this project that I would love to see built in as configurable options:
-Whether to consider comments
-Whether to consider whitespace
-Whether to consider string literals as unique tokens
-Toggle various feature sets: Tokens, chars, unigrams
Feature extraction has efficiency problems:
Takes too much memory
Takes too long due to duplicate work
Would love to see such things reduced to speed up iteration.
Root dir of repo is getting cluttered. I would like source files moved into an src directory and the necessary changes made to ensure everything stays working.
Currently, our tool is built around a RFC (Random Forest Classifier). It would be of great value if we could run other algorithms over our dataset to see what we can come across. I would like to see our RFC code abstracted out of the main program and brought into a more genericized class so we can attempt other classification techniques.
I would like the ability to normalize the whitespace of all code in a GPS. That is:
Any "unnecessary" spaces, those which do not effect the syntax of the code should be removed.
Any instances of multiple, consecutive, identical, white-space characters should be reduced to single instances of the character (except for tabs).
Newlines should only occur:
After semi-colons
After opening brackets {
Before and after closing brackets }
After colons : as in switch cases and labels.
After ) in for, if, and while blocks with no brackets
I am open to other logical instances.
All other newlines should be stripped out.
For example,
<-------------------------------------------------------------------->
for( int i=0; i<10; i++) would reduce to for(int i=0;i<10;i++). Note the only remaining space comes after the "int" as it is syntactically necessary.
<-------------------------------------------------------------------->
int i=0 reduces to int i=0 (got rid of double space)
<-------------------------------------------------------------------->
for(int i=0;i<10;i++)
foo()
remains unchanged
<-------------------------------------------------------------------->
code
code
would be reduced to:
code
code
as newlines count.
<-------------------------------------------------------------------->
The goal here is to propose and validate one step in an obfuscation process. That is, a way to prevent stylometric attacks without obscuring the meaning of the code.
coderID.py is getting too cluttered with various classification problem code. I would like this refactored out to a new file. The difficulty is that due to laziness, there are a lot of self references in said functions which will need to be refactored out to parameters and so forth.
I would like a command added that plots (or summarizes) the following information for inclusion in the paper:
Summary:
Total Number of functions
Total LOC
AVG function length (LOC)
Plot:
Function length (LOC) - Histogram
Our current feature reduction method isn't giving very consistent results. The threshold seems to be highly variable depending on the size of the dataset.
I would like to see recursive feature elimination implemented as this would be a more flexible approach.
Idea:
set lambda = .1 (some small fraction)
while score is not decreasing:
score model by cross validation
remove lamda * featureSetSize least important features
return featureSet
Time will be the biggest obstacle here. Has to be done efficiently.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.