rovome / jdrum Goto Github PK

Java implementation of the disk repository with update management (DRUM) framework as presented by Hsin-Tsang Lee, Derek Leonard, Xiaoming Wang, and Dmitri Loguinov in the paper "IRLbot: Scaling to 6 Billion Pages and Beyond"

License: MIT License

Java 100.00%

jdrum's People

Contributors

Stargazers

Watchers

Forkers

hongshunyang

jdrum's Issues

publish Maven archetypes

Either pushing to Sonatype or the repository now provided by GitHub would definitely help users try (and use) JDrum.

Is that something you'd consider?

Thanks

No license defined

It would be great to have this library under Apache or MIT license. This could allow to use it in many projects without reimplementing same idea from scratch.

64 bit key is not enough

Original algorithm uses 80 bit. When you deal with data what can hash into 64 bit with almost no collisions you probably should store all the data in RAM.

Live lock in FlippableDataContainer.put()

I've tried to call drum.checkUpdate() about a million times and FlippableDataContainer falls into live lock in put method. Making all methods synchronized fixes the issue.

ps It happens more often w/o debugger attached and maybe related to throughput. When you move
"this.berkeleyDB.sync();" out of loop in compareDataWithDataStore() performance goes from ~100 checks per second to ~35000 checks per second what probably helps to reproduce the issue.

Recommend Projects