ldbc / ldbc_graphalytics Goto Github PK

Using validation graph from:
https://s3-eu-west-1.amazonaws.com/graphalytics-graphs/index.html

The result contains numeric_limits<int64_t>::max(), but the validation expects Infinity:

18:34:31.206 [INFO ] Validating contents of '/var/scratch/mcapota/output-graphmat-s/datagen-300-SSSP'...
18:34:43.340 [INFO ]  - Vertex 6 has value '9.223372036854776E18', but valid value is 'Infinity'

Edges per seconds = number of edges / runtime.

We will examine (semantical) TEPS in the future.

Add more details, e.g.:
https://www.spec.org/cpu2006/results/res2014q4/cpu2006-20140922-31653.html
http://icl.cs.utk.edu/hpcc/hpcc_record.cgi?id=382

E.g., number of nodes and edges, file size.

Allows shipping one big configuration files that includes configuration for all the known graphs.

For Java platforms, two jar files is needed, one for default, another for Granula.However, for C++ platforms, it is entirely unnecessary to have two jar files (they are identical).

Hello

I should have reported this problem to an administrator, but I did not find any contact on the project page nor on the github space. The server at atlarge.ewi.tudelft.nl which hosts graph files is down for several days. I would appreciate if you resolve the server issue or provide a secondary mirror for accessing those files.

Best regards,
Fadishei

As far as I can tell, the only way to determine if there were validation failures in a Graphalytics run is to examine each benchmark validation result. It would be useful to have an indication that the validation succeeded overall, e.g., "Validation successful for all benchmarks" printed at the end of the command line output and at the top of the experimental tab in the HTML report.

Add an option to output the benchmark report in a machine-readable format, e.g., TSV.

And mention it in the documentation.

I am unable to access the graph datasets. Is there any other place I can access them?

It's a waste of time to run the benchmark and see this at the end:

[main] ERROR nl.tudelft.graphalytics.Graphalytics - Failed to write report:
java.io.IOException: Output directory of report is non-empty: "neo4j-report".

Graphalytics benchmark launches independent JVM to execute each benchmark run, such that time-out can be guaranteed.

However, the timeout includes now the proprocess, postprocess and validation time, which is not an accurate measurement.

The prescribed path - http://atlarge.ewi.tudelft.nl/graphalytics/ in README to download the graphs returns a 404 error. Please change the link to point to the correct location.

When not a single graphs defined in "benchmark.run.graphs" cannot be found, graphs defined in "graphs.names" will be benchmarked instead.

Hello,

in the paper, you mention the following:

"To support the ability to generate graphs of different characteristics, we have extended Datagen with the capability to dynamically reproduce different distributions by means of plugins. We have already implemented those for the Zeta and Geometric distribution models, but more will be added in the
future as more real graphs are analysed."

Where can I find the modified LDBC graph gens for Zeta and Geometric distributions?

For any benchmark, the chance is high that some benchmark runs will fail. It is easier to diagnose the failures if there are well-classified states of benchmark run, e.g.,

initialization
execution
completion
validation

In graphalytics-core, it's called SingleSourceShortestPaths, e.g.:
https://github.com/tudelft-atlarge/graphalytics/blob/master/graphalytics-core/src/main/java/nl/tudelft/graphalytics/domain/algorithms/SingleSourceShortestPathsParameters.java

In graphalytics-platforms-giraph and the other platforms, it's called SingleSourceShortestPath, e.g.:
https://github.com/tudelft-atlarge/graphalytics-platforms-giraph/blob/master/graphalytics-platforms-giraph-platform/src/main/java/nl/tudelft/graphalytics/giraph/algorithms/sssp/SingleSourceShortestPathJob.java
https://github.com/tudelft-atlarge/graphalytics-platforms-graphx/blob/master/graphalytics-platforms-graphx-platform/src/main/scala/nl/tudelft/graphalytics/graphx/sssp/SingleSourceShortestPathJob.scala

The Graphalytics benchmark operates on port 8011 and 8012. The port numbers should be configurable for easier deployment.

Exception in Thymeleaf when running benchmark with an algorithm that is not yet configured for all graphs. No result is added for (graph, algorithm) pair, but there should be a "not completed"/"not started" result. Exception text:

Exception evaluating OGNL expression: "report.getResult(graph, algorithm).completedSuccessfully"

Some paths are not converted correctly by the core to absolute paths before they are passed to the platform extensions. To do: find out which paths, and fix them.

In the past month, since Build 51, builds have not run graphalytics-tests, even if their graphalytics-core compiles. Normal behavior (bc -giraph, -neo4j, etc. were not recompiled)? Fix?

Hint: could it be graphalytics-validation (was -tests)? @mihaic with the idea.

Included test report posted by Jenkins:

Refer to this link for build results (access rights to CI server needed):
http://jenkins.tribler.org//job/Graphalytics_pull_request_tester/21/

Failed Tests: 2

nl.tudelft.graphalytics:graphalytics-platforms-mapreducev2: 2

• nl.tudelft.graphalytics.mapreducev2.cd.CommunityDetectionTest.testUndirectedCommunityDetectionOnValidationGraph
• nl.tudelft.graphalytics.mapreducev2.evo.ForestFireModelTest.testDirectedForestFireModelOnValidationGraph

Graphalytics will only include core and validation (currently named graphalytics-tests). Each platform in platforms will be a separate repository. Platform configuration files will be extracted from config.
Each platform will also be a separate Maven project, depending on the core project. Once packaged, each platform code will be used by the core code for actual benchmarking. run-benchmark.sh will sill be used for running the benchmark.

Obtaining binaries will work as follows:

1. compile-benchmark.sh without flags will install core and validation in the local Maven repository.
2. compile-platform.sh (new script) in each platform to obtain platform binaries.
3. Configure path for platform binaries in graphalytics.
4. package-benchmark.sh (new script) to create a redistributable archive of core (validation?) and platforms, as well as run-benchmark.sh and platform-specific prepare-benchmark.sh scripts.

A full validation has been deployed on each benchmark run, which is quite heavy and verbose. Sometimes validation dataset might not be available. It will be easier to determine whether a benchmark execution has completed by checking if the output has the correct size.

E.g., Jenkins @ TUDelft. At least compilation + unit tests, perhaps integration tests.

The link to http://atlarge.ewi.tudelft.nl/graphalytics/ is broken.