Comments (6)
Ok, did some first further analysis here with the MPT benchmarks and couldn't find any signification deterioration on first run.
First run on current master
with npm run build && ts-node benchmarks/index.ts
, results on my local PC:
1. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 799.467ms
benchmarks/checkpointing.ts | average execution time: 0s 206.067ms
2. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 835.272ms
benchmarks/checkpointing.ts | average execution time: 0s 199.895ms
3. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 849.828ms
benchmarks/checkpointing.ts | average execution time: 0s 204.285ms
I then checkout the commit mentioned above with git checkout c3155665c19b5106991cd708137d7d8272e5d13f
, reinstalled dependencies with npm i
, had to manually install a missing dependency npm i semaphore-async-await
.
I then copied over the benchmarks
folder from master
to benchmarks-new
(did the older-version runs in a copy of the repo) to have some unified benchmark code basis for comparison and then ran npm run build && ts-node benchmarks-new/index.ts
, following results:
1. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 1094.841ms
benchmarks/checkpointing.ts | average execution time: 0.05s 197.308ms
2. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 871.622ms
benchmarks/checkpointing.ts | average execution time: 0s 219.057ms
3. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 958.409ms
benchmarks/checkpointing.ts | average execution time: 0s 251.631ms
On a third round I checked out the commit mentioned as parent commit of c315566 by GitHub with git checkout
git checkout 029b5fe`, results:
1. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 955.015ms
benchmarks/checkpointing.ts | average execution time: 0s 227.782ms
2. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 853.373ms
benchmarks/checkpointing.ts | average execution time: 0s 255.643ms
3. Run
benchmarks/random.ts | rounds: 1000, ERA_SIZE: 1000, sys: 860.376ms
benchmarks/checkpointing.ts | average execution time: 0s 209.612ms
So everything is pretty comparable here, no great deterioration or deviation recognizable.
//cc @s1na
from merkle-patricia-tree.
I now added a trie.commit()
(since this was not executed and at the same time commit()
is suspected to do the speed deterioration) to benchmarks/checkpointing.ts
:
for (let i = 0; i < numOfIter; i++) {
await trie.put(vals[i], keys[i])
trie.checkpoint()
await trie.get(Buffer.from('test'))
trie.commit()
numOfOps++
if (numOfOps === numOfIter) {
const hrend = process.hrtime(hrstart)
resolve(hrend)
}
}
also adopted to 50
iterations and 10
samples, since runs were so slow, results:
c315566 :
1. Run
benchmarks/checkpointing.ts | average execution time: 0s 203.428ms
2. Run
benchmarks/checkpointing.ts | average execution time: 0s 211.407ms
3. Run
benchmarks/checkpointing.ts | average execution time: 0s 211.639ms
On c315566 :
1. Run
benchmarks/checkpointing.ts | average execution time: 0s 208.688ms
2. Run
benchmarks/checkpointing.ts | average execution time: 0s 216.011ms
3. Run
benchmarks/checkpointing.ts | average execution time: 0s 210.697ms
@s1na Where/how did you actually detect this speed deterioration? Is this reproduceable? 🤔
from merkle-patricia-tree.
@holgerd77 My set-up was a bit twisted.
I had 2 clones of the VM:
- VM's master branch which included the most recent MPT release to run the benchmarks
- Compared that to the VM at commit ethereumjs/ethereumjs-monorepo@b6beae4 which is right before this PR ethereumjs/ethereumjs-monorepo#787 was merged.
Then I had a clone of MPT and was checking previous commits to see which one caused the slowdown. So what I did was revert to those commits and then publish that commit (with a new version number) to verdaccio and then used that version in the VM.
Looking now I had a second script for only testing insertions and checkpoint/commit flow. Posting it as a gist here.
from merkle-patricia-tree.
@holgerd77 I noticed in #124 (comment) you would have to await
the commit. I am wondering though, is there a reason to do the checkpoint after the put? I would think one would want to do checkpoint -> put -> commit
from merkle-patricia-tree.
@ryanio ah no, sorry for the confusion, the put -> checkpoint
order was just there and I didn't have a close-enough look and just added the commit
afterwards. You are right, checkpoint -> put -> commit
makes more sense.
from merkle-patricia-tree.
Nothing found on further investigation, will close. Work on MPT performance will continue and new dedicated issues can be opened along this work.
from merkle-patricia-tree.
Related Issues (20)
- missing typescript types HOT 3
- Trie only accepts one `put` call HOT 2
- Rename Trie.prove to Trie.createProof HOT 7
- Verify Proof fails if there is an extension with an embedded branch
- Library fails branch-value-update test HOT 5
- update checkpointing API HOT 2
- Support Question, headline replaced by admin
- better proof/verification support HOT 3
- Support Promise and callbacks for async functions HOT 1
- Extract db-related logic from baseTrie HOT 4
- Replace istanbul with nyc HOT 1
- More consistent and better structured directory/file layout HOT 2
- Data gets lost when using string as a key. (memdown + 2 tries) HOT 3
- Tape v4.10.0 breaks tests
- Delete node error HOT 6
- nibbles.js matchingNibbleLength potential error HOT 1
- Replace sha3 with keccak256 HOT 1
- npm run build fails HOT 1
- Add Ethereum name or address to package.json
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from merkle-patricia-tree.