Comments (4)
dipu-bd
commented on June 9, 2024
These look fairly easy to implement. I will add support for the algorithms in the above tasklist
from hashlib.
cbenhagen
commented on June 9, 2024
@dipu-bd damn you're fast! :) I just tested your xxHash64 implementation and it seems to return wrong hashes for data longer than 31 bytes.
Is there a way you can optimize the speed? It would be awesome to achieve at least 1GB/s. More is always better of course.
from hashlib.
dipu-bd
commented on June 9, 2024
I just tested them with some basic set of test cases. I will add more cases once I am done implementing all of them.
Dart has a lot of limitations. I am doing the best I can do within those limitations. Can not promise to achieve at least 1GB/s.
If you can provide some test cases, it would be helpful too.
from hashlib.
dipu-bd
commented on June 9, 2024
Current benchmark result:
$ dart compile exe ./benchmark/xxhash.dart -o ./build/xxhash && ./build/xxhash
--------- XXH3 ----------
XXH3 (512 x 100000): 534.881 ms => nb# 186957 @ 95.72MB/s
XXH3 (32768 x 10000): 2803.8 ms => nb# 3567 @ 116.87MB/s
XXH3 (8388608 x 10): 718.028 ms => nb# 14 @ 116.83MB/s
--------- XXH32 ----------
XXH32 (512 x 100000): 113.605 ms => nb# 880240 @ 450.68MB/s
XXH32 (32768 x 10000): 674.364 ms => nb# 14829 @ 485.91MB/s
XXH32 (8388608 x 10): 173.158 ms => nb# 58 @ 484.45MB/s
--------- XXH64 ----------
XXH64 (512 x 100000): 117.831 ms => nb# 848676 @ 434.52MB/s
XXH64 (32768 x 10000): 683.766 ms => nb# 14625 @ 479.23MB/s
XXH64 (8388608 x 10): 171.683 ms => nb# 58 @ 488.61MB/s
--------- XXH128 ----------
XXH128 (512 x 100000): 514.697 ms => nb# 194289 @ 99.48MB/s
XXH128 (32768 x 10000): 2802.43 ms => nb# 3568 @ 116.93MB/s
XXH128 (8388608 x 10): 721.401 ms => nb# 14 @ 116.28MB/s
Benchmark of original implementation on the same machine:
benchmarking large inputs : from 512 bytes (log9) to 128 MB (log27)
xxh3 , 24798, 30406, 32255, 34043, 34979, 35244, 34881, 35331, 35457, 35569, 35266, 35240, 34807, 35173, 35349, 35438, 31891, 31502, 32324
XXH32 , 7419, 7542, 7658, 7667, 7884, 7886, 7864, 7836, 7775, 7852, 7862, 7826, 7857, 7870, 7761, 7962, 7777, 7766, 7623
XXH64 , 11935, 12752, 13125, 13216, 13356, 13559, 13573, 13485, 13326, 13615, 13444, 13721, 13250, 13333, 13449, 13464, 13218, 13001, 13036
XXH128 , 20893, 27057, 30375, 32641, 34044, 34518, 34821, 35189, 35489, 35323, 34665, 33807, 33404, 34843, 34691, 33877, 30740, 31530, 30862
---------------------
For a string of size 2^23 (= 8388608) :
xxh3 = 35349 @ 35.35 GB/s
XXH32 = 7761 @ 7.76 GB/s
XXH64 = 13464 @ 13.46 GB/s
XXH128 = 33877 @ 33.88 GB/s
from hashlib.
Related Issues (17)
- Implementation sources
- Move codecs out of utils
- Implement RIPEMD algorithm series HOT 2
- Implement Streebog HOT 2
- Implement Whirlpool HOT 2
- Implement ShangMi-3 (SM3) HOT 2
- Implement Blake3
- Implement BCrypt HOT 1
- Implement MD4 HOT 1
- pub.dev shows owner as "unverified uploader" HOT 1
- Expose current time in TOTP
- Support scrypt
- Improve Poly1305 runtime HOT 1
- Argon2 is much slower (x146) with `dart run` than `dart compile && .exe` HOT 7
- HOTP: An HMAC-Based One-Time Password Algorithm
- TOTP: Time-Based One-Time Password Algorithm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hashlib.