Implementation of various network properties using pySpark and GraphFrames. The various output Graphs are present in the Graphs folder and the various csv output files are present in the CSV folder. The stanford_graphs folder contains the various input graphs.

Run the following commands on any new terminal before running any code:

• export SPARK_HOME=/usr/local/spark-1.6.2-bin-hadoop2.6
• export PYSPARK_PYTHON=/usr/bin/python2
• export PYSPARK_DRIVER_PYTHON=/usr/bin/python2

The degree disstribution is the measure of the frequency of nodes that have a certain degree. The various graphs have been plotted using matplotlib.

Run the file using:

• $SPARK_HOME/bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6 degree.py ./stanford_graphs/amazon.graph.large large OR
• $SPARK_HOME/bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6 degree.py ./stanford_graphs/amazon.graph.small

Centrality is a is a way to determine nodes that are important based on the structure of the graph. Here we have used Closeness Centrality which is a measure of the distance of a node to all other nodes. The output of the file is in centrality.csv

Run the file using:

• $SPARK_HOME/bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6 centrality.py

Articulation points are vertices in the graph that when removed, create more components than there were originally.

Run the file using:

• $SPARK_HOME/bin/pyspark --packages graphframes:graphframes:0.1.0-spark1.6 articulation.py 9_11_edgelist.txt