mikispag / bitiodine Goto Github PK

A Rust Bitcoin blockchain parser with clustering capabilities, allowing to group together addresses in ownership clusters. Please contact @mikispag if interested in using BitIodine for any real-world use case.

Home Page: https://miki.it

License: MIT License

Rust 100.00%

bitcoin-analytics blockchain-analytics bitcoin blockchain

bitiodine's People

Contributors

Stargazers

Watchers

bitiodine's Issues

Error while executing generate_users_graph.py

Hi,
I was executing the python script generate_users_graph.py to get the user graph from my previous parsed blockchain, but I got the following error that I believe to be a bug:

xyz@machine:~/Documents/bitiodine/deploy/grapher$ ./generate_users_graph.py
Clusters loaded - 24938637 addresses.
Singletons stripped - 22475995 addresses.
Scanning 31592433 transactions, starting from 1.
Traceback (most recent call last):
  File "./generate_users_graph.py", line 83, in <module>
    dest = users.get(dest_addr)
NameError: name 'dest_addr' is not defined

Could this dest_addr be replaced by the out_address from the line 82 as in that line it was previous checked if out_address is not None? Do you have some clue or suggestion to overtake what I'm seeing as a little bug?

Thank you.

Issue in running

I'm facing the below issues, kindly suggest a solution:-

± ./bitiodine-rust -b /mnt/ssd/bitcoincore/blocks -v 2

thread 'main' panicked at /home/hornet/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clap_builder-4.4.16/src/parser/error.rs:32:9:
Mismatch between definition and access of `v`. Could not downcast to TypeId { t: 7428646492878894209665195255548636123 }, need to downcast to TypeId { t: 263171357195085286585474072027052583236 }

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I don't quite understand the contents of clusters. CSV

I don't quite understand the contents of clusters. CSV,Can you explain.

time:0.2.7 cannot find function `now` and `strftime` in crate `time`

error[E0425]: cannot find function now in crate time
--> src\main.rs:59:27
|
59 | let t = time::now();
| not found in time

error[E0425]: cannot find function strftime in crate time
--> src\main.rs:63:23
|
63 | time::strftime("%Y-%m-%d %H:%M:%S", &t).unwrap(),
| not found in time

Repeated missing parent block warning

I am trying to run the parser against the blockchain, from a bitcoind version 0.11.x

I am repeatedly getting a warning about missing the same parent block, at various depths

warning: at depth 197069 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 208160 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 74333 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 52879 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 205035 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 1305 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 74923 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 82277 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 232969 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 235970 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 68304 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 141161 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 49804 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 13132 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 167728 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 41026 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 208287 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 230974 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 26962 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 46349 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 100063 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 42488 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 13793 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048
warning: at depth 138265 in chain, failed to locate parent block 00000000000064ac7c1b57c1d3b4fd3a3aeb2f26c96c2dde853eef230a271048

ad nauseum

It appears to continue running however, but I wanted to check to see if this is going to block the rest of the parser from completeing successfully..

CPU Utilization

Trying to run the "Clusterizer" and graph generators (both transaction and user graphs) on a fully updated block-chain takes extremely long. Looking into CPU and memory usage, reveals the python code which generates the graphs takes advantage of only one CPU core. We're running the scripts on a server with 2 CPUs(Xeon 2690 v2), 768GB RAM and SSD storage. But we never managed to generate graphs properly due to unpractical needed time.

Can you suggest us any solution to boost the running speed and force to use all system resources?

bitiodine.net, never worked for me

I tried 3+ months ago and it would say the server are updating, same thing today.

Transactions Info along with clustering of addresses

Hello,
Can we also obtain the transactions information like sending party/receiving party bitcoin amounts and time stamps along with the cluster information? I need to construct a transaction graph in a project. And how is the clustering done? Are you clustering the input addresses of the individual transactions into 1 address? Are the change addresses are being considered during clustering?

memory allocation of 23400000000 bytes failed

cargo build --release

...
[skip]
...
warning[E0502]: cannot borrow *slice_inputs_and_outputs as immutable because it is also borrowed as mutable
--> src\transactions.rs:138:13
|
136 | sha256_hasher1.input(read_slice(
| ---------- mutable borrow later used by call
137 | &mut slice_inputs_and_outputs,
| ----------------------------- mutable borrow occurs here
138 | slice_inputs_and_outputs.len() - slice.len(),
| ^^^^^^^^^^^^^^^^^^^^^^^^ immutable borrow occurs here
|
= warning: this error has been downgraded to a warning for backwards compatibility with previous releases
= warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future
= note: for more information, try rustc --explain E0729

warning[E0502]: cannot borrow *init_slice as immutable because it is also borrowed as mutable
--> src\transactions.rs:164:48
|
164 | slice: read_slice(&mut init_slice, init_slice.len() - slice.len())?,
| ---------- --------------- ^^^^^^^^^^ immutable borrow occurs here
| | |
| | mutable borrow occurs here
| mutable borrow later used by call
|
= warning: this error has been downgraded to a warning for backwards compatibility with previous releases
= warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future
= note: for more information, try rustc --explain E0729

warning[E0502]: cannot borrow *init_slice as immutable because it is also borrowed as mutable
--> src\transactions.rs:201:48
|
201 | slice: read_slice(&mut init_slice, init_slice.len() - slice.len())?,
| ---------- --------------- ^^^^^^^^^^ immutable borrow occurs here
| | |
| | mutable borrow occurs here
| mutable borrow later used by call
|
= warning: this error has been downgraded to a warning for backwards compatibility with previous releases
= warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future
= note: for more information, try rustc --explain E0729

warning[E0502]: cannot borrow *init_slice as immutable because it is also borrowed as mutable
--> src\transactions.rs:226:48
|
226 | slice: read_slice(&mut init_slice, init_slice.len() - slice.len())?,
| ---------- --------------- ^^^^^^^^^^ immutable borrow occurs here
| | |
| | mutable borrow occurs here
| mutable borrow later used by call
|
= warning: this error has been downgraded to a warning for backwards compatibility with previous releases
= warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future
= note: for more information, try rustc --explain E0729

Finished release [optimized] target(s) in 1m 11s

run bitiodine-rust.exe

C:\bitiodine-master\target\release>bitiodine-rust.exe
memory allocation of 23400000000 bytes failed

Some questions of The Clusterizer

I found that the clusterizer.py don't apply the Heuristic 2, Is that right? And Is there some new version complete that? I didn't find in the project of "rusty-blockparser".
Thank you very much for provide this tool.

a question about bitiodine running

Hello Mr.

I learn your excellent tool, and complie it, use command:

./bitiodine-rust -b /root/.bitcoin/blocks/blocks/ -o ./clusters.csv

meet a error:

2019-12-09 23:15:51.8646 - INFO - Parsing the blockchain: block file 0/1861...
2019-12-09 23:15:57.7749 - INFO - Parsing the blockchain: block file 1/1861...
2019-12-09 23:16:04.1321 - INFO - Parsing the blockchain: block file 2/1861...
2019-12-09 23:16:11.0132 - INFO - Parsing the blockchain: block file 3/1861...
2019-12-09 23:16:19.6497 - INFO - Parsing the blockchain: block file 4/1861...
2019-12-09 23:16:26.2890 - INFO - Parsing the blockchain: block file 5/1861...
2019-12-09 23:16:33.8223 - INFO - Parsing the blockchain: block file 6/1861...
2019-12-09 23:16:39.3330 - INFO - Parsing the blockchain: block file 7/1861...
2019-12-09 23:16:46.2270 - INFO - Parsing the blockchain: block file 8/1861...
memory allocation of 2214592528 bytes failedAborted

Is it right as my runing way ?How I check the export result,see the ./clusters.csv?But I didn't find it.
Thanks

Latest version fails with 'called `Result::unwrap()` on an `Err` value: Invalid'

Built the latest version, machine is Windows, with the latest version of the blockchain:

C:\dev\bitiodine\target\release>bitiodine-rust --blocks-dir "G:\BitcoinData\blocks" --output "../../clusters.csv"
2018-12-08 15:24:20.2447 - INFO - Parsing the blockchain: block file 0/1455...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Invalid', libcore\result.rs:1009:5
stack backtrace:
   0: std::sys::windows::backtrace::set_frames
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\sys\windows\backtrace\mod.rs:104
   1: std::sys::windows::backtrace::set_frames
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\sys\windows\backtrace\mod.rs:104
   2: std::sys::windows::backtrace::set_frames
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\sys\windows\backtrace\mod.rs:104
   3: std::sys::windows::backtrace::set_frames
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\sys\windows\backtrace\mod.rs:104
   4: std::sys::windows::backtrace::set_frames
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\sys\windows\backtrace\mod.rs:104
   5: std::panicking::default_hook
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\panicking.rs:227
   6: std::panicking::default_hook
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\panicking.rs:227
   7: std::panicking::continue_panic_fmt
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\panicking.rs:390
   8: std::panicking::rust_begin_panic
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libstd\panicking.rs:325
   9: core::panicking::panic_fmt
             at /rustc/abe02cefd6cd1916df62ad7dc80161bea50b72e8\src/libcore\panicking.rs:77
  10: <unknown>
  11: main
  12: <unknown>
  13: main
  14: invoke_main
             at f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:64
  15: invoke_main
             at f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:64
  16: BaseThreadInitThunk
  17: RtlUserThreadStart

generate_transaction_graph.py

Getting the following error:

:~/bitiodine/deploy/grapher$ ./generate_transaction_graph.py
[Errno 2] No such file or directory: 'None.dat'
Traceback (most recent call last):
  File "./generate_transaction_graph.py", line 40, in <module>
    print("Scanning %d transactions, starting from %d." %(amount_txids, min_tx_id))
TypeError: %d format: a number is required, not NoneType

Since my attempts failed, might you have a work around suggestion?

Seg fault

Hi, I'm trying to get bitiodine up and running on ubuntu and am getting a seg fault:

info: starting command "sqldump"
info: Dumping the blockchain...
Segmentation fault (core dumped)

Any idea what I could be doing wrong? Thanks!

What are the dependencies?

I'm unable to make the blockparser - are there dependencies that are different from znort's version?

mikispag / bitiodine Goto Github PK

bitiodine's People

Contributors

Stargazers

Watchers

Forkers

bitiodine's Issues

Recommend Projects

Recommend Topics

Recommend Org