weinberger / nosql-tests Goto Github PK
View Code? Open in Web Editor NEWNoSQL benchmark tests for documents and graphs
License: Apache License 2.0
NoSQL benchmark tests for documents and graphs
License: Apache License 2.0
PostgreSQL defaults are very conservative. Suggest using http://pgtune.leopard.in.ua to at least open up basic settings to make full use of testing platform.
It looks like the setup and run scripts are not working like they used to before.
Also, can you share the environment used for the test? For example, neo4j
doesn't support Java 11
and actually requires Java 8
.
Since my post wasn't made public I am posting here. From the discussion on the Arango website, I'd like to hold you all to your word.
We will then publish a update - also with the new results for Neo4J. That should gives as proof-able results instead of marketing statements.
I am very interested in the results of the PR from OrientDB.
Scott
nodejs benchmark neo4j -a 10.240.0.2 -t shortest
tests = tests.split(',').map(trim);
^
ReferenceError: trim is not defined
at Object.<anonymous> (/home/maxdemarzi/nosql-tests/benchmark.js:93:32)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:902:3
I am trying to run against the neo4j version 4.2 but I have problems with the transaction handing
node benchmark.js neo4j -a 127.0.0.1 -t 'warmup'
i am getting
INFO using server address 127.0.0.1
INFO start
INFO step 1/2 done
Neo4jError: Queries cannot be run directly on a session with an open transaction; either run from within the transaction or use a different session.
at captureStacktrace (/root/nosql-tests/node_modules/neo4j-driver/lib/result.js:281:15)
at new Result (/root/nosql-tests/node_modules/neo4j-driver/lib/result.js:68:19)
at Session._run (/root/nosql-tests/node_modules/neo4j-driver/lib/session.js:174:14)
at Session.run (/root/nosql-tests/node_modules/neo4j-driver/lib/session.js:135:19)
at Object.getDocument (/root/nosql-tests/neo4j/description.js:78:19)
at /root/nosql-tests/neo4j/description.js:43:28
at Object.getCollection (/root/nosql-tests/neo4j/description.js:62:5)
at /root/nosql-tests/neo4j/description.js:35:24
at processTicksAndRejections (node:internal/process/task_queues:94:5) {
code: 'N/A'
}
Can you add RethinkDB to the tests, please?
In the tests you use the JSON type, while for a "real" environment that was performance-sensitive it's likely that JSONB would be used instead. Optionally, you could also add a GIN on the structure
field which would likely improve the relation matching performance.
The majority of the comments on the current benchmark 2018 suggest Dgraph as comparison and I fully agree on that, especially since Neo4J is known as a comparatively slow graph database, while surely a widespread one.
I am very interested in a comparison to dgraph. :)
When you try to execute a single task, the benchmark.js return next message:
`[root@localhost nosql-tests]# node benchmark mongodb -a 127.0.0.1 -t singeRead
/home/dsuser/nosql-tests/benchmark.js:93
tests = tests.split(',').map(trim);
^
ReferenceError: trim is not defined
at Object. (/home/dsuser/nosql-tests/benchmark.js:93:32)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:393:7)
at startup (bootstrap_node.js:150:9)
at bootstrap_node.js:508:3`
Instead of this code:
tests = tests.split(',').map(trim);
I use this:
tests = tests.split(',').map(function (val) { return val; });
Can you add AgensGraph to the tests, please?
It's a proper implementation of a Graph Database on top of PostgreSQL and possibly a better fit in the comparison against other graph native database engines.
Wasted several days on arango (3.8), because it is twice as fast as neo4j, wow (it is not)...
general reliability and maturity.... neo4j is production ready, arango 3.6 had terrible bugs with shortestpath.
It can be measured as "days since critical bugs". Arango doesn't shine here also...
So, tests look very synthetic
the code is not executing from this line..dont know whats wrong..can you please help
module.exports = {
name: 'PostgreSQL tabular',
isPostgreSQL: true,
server: {},
singleWriteQuery : "",
startup: function (host, cb) {
var insertColumns = ['id'];
var binds = [];
I would love to see couchbase 4 (both map reduce and N1QL) added to the tests. When I have some free time will try to put something togather.
Hi.
This is a great project. Thanks for putting it together.
MongoDB is performing pretty poorly on most of the benchmark tests provided. Despite this we're still considering it as the right solution, partly due to this one killer feature, which is perfect for our use case:
https://docs.mongodb.com/v3.0/reference/operator/update/set/#set-fields-in-embedded-documents
All of the other datastores that are being benchmarked would require the client to retrieve the document, update it and reinsert it, which I estimate would be will be much slower than the single 'update' operation provided by MongoDB above. This would be particular true for large, highly nested documents.
It would be really interesting and useful to add a benchmark test for this!
I use today 20171106 version nosql-tests to test orientdb-community-importers-2.2.29.tar.gz .
and I have solve the no user and password error,
but it still report an error below:
module.js:471
throw err;
^
Error: Cannot find module '/root/Documents/NimbiLin/Programs/Databases/NoSQL/Install/InsReal/DLBenchmark/node_modules/orientjs/benchmark'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:149:9)
at bootstrap_node.js:509:3
I have followed "https://github.com/weinberger/nosql-tests" 's readme.MD to install and without any error, how to solve this?
I get the following error while trying:
nodejs benchmark postgresql -a myhost1 -t all
INFO using server address myhost1
INFO start
INFO version [{"version":"PostgreSQL 9.5.6 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609, 64-bit"}]
ERROR error: relation "relations" does not exist
I get the feeling I am missing something out. Do I have to preload the database? How?
Ubuntu 16.04 on VMware 6.5 U1.
Some OrientDB transformers failed.
Anyone see this? I'll try again to see if this is reproducible.
Is this warning is safe to ignore?
DOWNLOAD DIRECTORY: /tmp/nosqlbenchmark/downloads
Downloading PROFILES
--2018-02-15 13:49:24-- https://snap.stanford.edu/data/soc-pokec-profiles.txt.gz
Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80
Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 435512787 (415M) [application/x-gzip]
Saving to: ‘/tmp/nosqlbenchmark/downloads/soc-pokec-profiles.txt.gz.tmp’
/tmp/nosqlbenchmark/downloa 100%[==========================================>] 415.34M 1.16MB/s in 8m 32s
2018-02-15 13:57:57 (830 KB/s) - ‘/tmp/nosqlbenchmark/downloads/soc-pokec-profiles.txt.gz.tmp’ saved [435512787/435512787]
Downloading RELATIONS
--2018-02-15 13:58:58-- https://snap.stanford.edu/data/soc-pokec-relationships.txt.gz
Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80
Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 132454730 (126M) [application/x-gzip]
Saving to: ‘/tmp/nosqlbenchmark/downloads/soc-pokec-relationships.txt.gz’
/tmp/nosqlbenchmark/downloa 100%[==========================================>] 126.32M 1.23MB/s in 2m 53s
2018-02-15 14:01:52 (747 KB/s) - ‘/tmp/nosqlbenchmark/downloads/soc-pokec-relationships.txt.gz’ saved [132454730/132454730]
Converting PROFILES
Converting RELATIONS
OrientDB etl v.2.2.29 (build 9914189f972103907c24377a1567897e68642920) https://www.orientdb.com
[file] INFO Load from file /tmp/nosqlbenchmark/downloads/soc-pokec-profiles-orientdb.txt
[file] INFO Reading from file /tmp/nosqlbenchmark/downloads/soc-pokec-profiles-orientdb.txt with encoding UTF-8
2018-02-15 14:02:47:629 WARNI {db=pokec} Transformer [csv] returned null, skip rest of pipeline execution [OETLPipeline]
2018-02-15 14:02:47:630 WARNI {db=pokec} Transformer [vertex] returned null, skip rest of pipeline execution [OETLPipeline]
2018-02-15 14:09:19:868 WARNI {db=pokec} Transformer [vertex] returned null, skip rest of pipeline execution [OETLPipeline]
2018-02-15 14:09:19:868 WARNI {db=pokec} Transformer [edge] returned null, skip rest of pipeline execution [OETLPipeline]
...
...evidently, since the author hasn't used his GitHub profile since 2015.
For comparison:
I understand you guys are a business and want to make arango look good. Having arango fixed to 100% makes it appear like the most consistently performing database. Though it also makes the benchmark appear more biased than it needs to be.
Either way thanks for making this benchmark, there's a lot of talk on this topic and not enough measurement.
The postgresql.conf file should be included for reliable reproduction of this benchmark and validation of settings. Postgres is notorious for extremely conservative resource settings by default and does no inspection of the underlying platform so the settings in this file will define how the query planner represents the underlying system resources.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.