node graph correlation about collective-intelligence-framework HOT 12 CLOSED

http://horicky.blogspot.com/2010/07/google-pregel-graph-processing.html

https://groups.google.com/d/topic/ci-framework/loLVy0JmlFE/discussion

http://inductionapp.com/

gephi.org

http://csis.gmu.edu/noel/pubs/2006_CompComm.pdf

http://people.cis.ksu.edu/~halmohri/files/An%20Attack%20Graph-Based%20Probabilis
tic%20Security%20Metric.pdf

http://csrc.nist.gov/staff/Singhal/qop2008_DBN_paper.pdf

Correlating Intrusion Events and Building Attack Scenarios through Attack Graph 
Distances
http://users.encs.concordia.ca/~wang/idsj.pdf

Using attack graphs for correlating, hypothesizing, and predicting intrusion 
alerts
http://acsac.org/2004/papers/49.pdf

MulVAL: A logic-based network security analyzer
http://people.cis.ksu.edu/~xou/mulval/

The first step is probably determining the format and storage mechanism for the 
graph.  Rules such as what will be nodes, what will be edges, where attributes 
are recorded, etc will lead to fundamental constraints on how the graph can be 
used later on, (since some graph algorithms require graphs to meet certain 
constraints).

Also, the storage of the graph will be critical.  The method for storing the 
graph will have a direct effect on how quickly it can be searched.  Some 
notional use-cases will probably be necessary to understand how to store the 
data.

Once the structure is created, modules implementing various uses can then be 
created such as:
-Search for likely attack paths and expected sensor outputs for those paths
-Given a sensor state, likely malicious events and vulnerable conditions on the 
network.
-Given an event with arbitrary attributes, what are likely threats/consequences 
associated with that event.

To form a graph just to start messing around with, you can take the format for 
importing spreadsheets from gephi's site 
(https://gephi.org/users/supported-graph-formats/spreadsheet/) and just throw 
two tables together, (one for nodes and one for edges).

Then fill it with some data.  I created a bunch of ways to break into my home 
network.  (email scam, SMS scam, cold call tech support, direct attack on my 
servers, direct attack based on IP address from my email, etc).  You can have 
different threats (internet criminal, online enemy, hactivist, etc).   You can 
have multiple consequences (steals your banking information, steals your 
contact information, wipes your computers, turns on your webcams, etc, etc).  
You should get a nice little graph going.

From there, look up Breath First Searches (BFS, 
http://en.wikipedia.org/wiki/Breath_first_search) and Depth First Search (DFS, 
http://en.wikipedia.org/wiki/Depth-first_search).  Code up some quick examples 
and practice using them for looking for things in your graph (or updating 
values).  This is most analigious to the Intrusion Detection Problem.

The next thing I did was give distances to the edges to represent likelihood 
and then implemented Dijkstra's shortest algorithm 
(http://www.vogella.com/articles/JavaAlgorithmsDijkstra/article.html) but 
slightly modified to find the longest path (highest risk) and to multiple 
rather than add the distance.

Based on Joe Stewart's "Chasing APT" brief from Blackhat, it appears they 
already have a robust graph of threats and threat attributes.  If that could be 
married with a graph of attacks from Verizon's DBIR, you would have a clear 
connection between threats and attacks (and potential attacks).  If you then 
mapped events from sensors (including non-security events), you could then 
search through the DBIR portion of the graph back to the "Chasing APT" portion 
of the graph to find threat actors.

https://github.com/collectiveintel/cif-v2/issues/22