Giter VIP home page Giter VIP logo

libnspv's Introduction

Icon

Documentation update in progress

libnspv – a portable C library for creating and manipulating data structures and interacting with the p2p network on Komodo-compatible Smart Chains including the KMD main chain.

Based on Libbtc

=============================================================

Build Status Coverage Status

What is libbtc?

Libbtc is a very portable C library for creating and manipulating bitcoin data structures and interacting with the p2p network.

Current features

  • Generating and storing private and public keys
  • ECDSA secp256k1 signing and verification (through libsecp256k1 included as git subtree)
  • Generate recoverable signatures (and recover pubkey from signatures)
  • BIP32 hierarchical deterministic key derivation
  • Transaction generation, manipulation, signing and ser-/deserialization including P2PKH, P2SH, multisig
  • Address generation
  • Base58check encoding
  • Native implementation of SHA256, SHA512, SHA512_HMAC, RIPEMD-160 including NIST testvectors
  • Native constant time AES (+256CBC) cipher implementation including NIST testvectors
  • Keystore (wallet) databases (through logdb https://github.com/liblogdb/liblogdb)
  • Event based bitcoin P2P client capable of connecting to multiple nodes in a single thread (requires libevent)

Advantages of libbtc?

  • No dependencies in case no p2p network client is required (only dependency is libsecp256k1 added as git subtree)
  • The only dependency for the p2p network client is libevent (very portable)
  • optimized for MCU and low mem environments
  • ~full test coverage
  • mem leak free (valgrind check during CI)

The bitcointool CLI

Generate a new privatekey WIF and HEX encoded:
./bitcointool -command genkey
> privatekey WIF: KwmAqzEiP7nJbQi6ofQywSEad4j5b9BXDJvyypQDDLSvrV6wACG8
> privatekey HEX: 102f1d9d91fa1c8d816ef469e74c1153a6b453d2a991e77fe187e5514a7b18ac
Generate the public key and p2pkh address from a WIF encoded private key
/bitcointool -command pubfrompriv -p KwmAqzEiP7nJbQi6ofQywSEad4j5b9BXDJvyypQDDLSvrV6wACG8
> pubkey: 023d86ca58e2519cce1729b4d36dfe5a053ad5f4ae6f7ef9360bee4e657f7e41c9
> p2pkh address: 1N5ZkjyabcZLLHMweJrSkn3qedsPGzAx9m
Generate the P2PKH address from a hex encoded compact public key
./bitcointool -command addrfrompub -pubkey 023d86ca58e2519cce1729b4d36dfe5a053ad5f4ae6f7ef9360bee4e657f7e41c9
> p2pkh address: 1N5ZkjyabcZLLHMweJrSkn3qedsPGzAx9m
Generate new BIP32 master key
./bitcointool -command hdgenmaster
> masterkey: xprv9s21ZrQH143K3C5hLMq2Upsh8mf9Z1p5C4QuXJkiodSSihp324YnWpFfRjvP7gqocJKz4oakVwZn5cUgRYTHtNRvGqU5DU2Gn8MPM9jHvfC
Print HD node
./bitcointool -command hdprintkey -privkey xprv9s21ZrQH143K3C5hLMq2Upsh8mf9Z1p5C4QuXJkiodSSihp324YnWpFfRjvP7gqocJKz4oakVwZn5cUgRYTHtNRvGqU5DU2Gn8MPM9jHvfC
> ext key: xprv9s21ZrQH143K3C5hLMq2Upsh8mf9Z1p5C4QuXJkiodSSihp324YnWpFfRjvP7gqocJKz4oakVwZn5cUgRYTHtNRvGqU5DU2Gn8MPM9jHvfC
> depth: 0
> p2pkh address: 1Fh1zA8mD6S2LBbCqdViEGuV3oDhggX3k4
> pubkey hex: 0394a83fcfa131afc47a3fcd1d32db399a0ffa7e68844546b2df7ed9f5ebd07b09
> extended pubkey: xpub661MyMwAqRbcFgAASPN2qxpRgoVdxUXvZHLWKhALMxyRbW9BZbs34ca9H3LrdsKxdMD4o5Fc7eqDg19cRTj3V9dCCeM4R1DRn8DvUq3rMva
Derive child key (second child key at level 1 in this case)
./bitcointool -command hdderive -keypath m/1h -privkey xprv9s21ZrQH143K3C5hLMq2Upsh8mf9Z1p5C4QuXJkiodSSihp324YnWpFfRjvP7gqocJKz4oakVwZn5cUgRYTHtNRvGqU5DU2Gn8MPM9jHvfC
> ext key: xprv9v5qiRbzrbhUzAVBdtfqi1tQx5tiRJ2jpNtAw8bRec8sTivLw55H85SoRTizNdx2JSVL4sNxmjvseASZkwpUopby3iGiJWnVH3Wjg2GkjrD
> depth: 1
> p2pkh address: 1DFBGZdcADGTcWwDEgf15RGPqnjmW2gokC
> pubkey hex: 0203a85ec401e66a218bf1583112599ee2a1268ebc90d91b7f457c87a50f2b011b
> extended pubkey: xpub695C7w8tgyFnCeZejvCr59q9W7jCpkkbBbomjX13CwfrLXFVUcPXfsmHGiSfpYds2JuHrXAFEoikMX6725W8VgrVL5x4ojBw9QFAPgtdw1G

The bitcoin-send-tx CLI

This tools can be used to broadcast a raw transaction to peers retrived from a dns seed or specified by ip/port. The application will try to connect to max 6 peers, send the transaction two two of them and listens on the remaining ones if the transaction has been relayed back.

Send a raw transaction to random peers on mainnet
./bitcoin-send-tx <txhex>
Send a raw transaction to random peers on testnet and show debug infos
./bitcoin-send-tx -d -t <txhex>
Send a raw transaction to specific peers on mainnet and show debug infos use a timeout of 5s
./bitcoin-send-tx -d -s 5 -i 192.168.1.110:8333,127.0.0.1:8333 <txhex>

How to Build libnspv with libbtc and tools

You will need to have some dev tools prior to building libnspv for your system.

Prepare system before building

MacOS

# Install brew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
# Install Xcode, opens a pop-up window to install CLT without installing the entire Xcode package
xcode-select --install
# Update brew and install dependencies
brew update
brew upgrade
brew install libsodium libevent automake libtool git wget
Install using brew
brew tap jl777/homebrew-libnspv
brew install libnspv

Android

# Install Termux
https://play.google.com/store/apps/details?id=com.termux
# Open and run the following commands
pkg upgrade
pkg install build-essential git wget libsodium libevent

Linux

sudo apt-get -y install build-essential pkg-config libc6-dev m4 autoconf \
libtool unzip git wget automake

Clone repository using Git

Once prepared your system, get the source code:

git clone https://github.com/jl777/libnspv/
cd libnspv

Now you can follow the next instructions to compile and update

First build cryptoconditions library

MacOS & Linux

cd ./src/tools/cryptoconditions
./autogen.sh
./configure
make

Windows

cd ./src/tools/cryptoconditions
./build_win.sh

Full libnspv library including CLI tool and wallet database

Go to libnspv directory and run:

On Linux and MacOS:

./autogen.sh
./configure
make check

On Windows run

./build_win.sh

Pure library without wallet support

Go to libnspv directory and run:

./autogen.sh
./configure --disable-wallet --disable-tools
make check

Updating libnspv

cd libnspv
git pull
./autogen.sh
./configure
make

Running nspv

libnspv does all the above, it also can launch a superlite nSPV client

./nspv KMD launches a KMD superlite

./nspv BTC launches the normal libbtc headers only SPV mode

for superlite clients, it will daemonize the nspv executable and it is accessible via JSON requests sent into the localhost rpc port. you can specify any available rpc port with -p option at the end.

there is a coins JSON file that comes from the https://github.com/jl777/coins/coins which will define the default behavior of the superlite if it is a komodo "asset" and also has the following fields:

"p2p": 12985,
"magic":"feb4cb23",
"nSPV":"5.9.102.210, 5.9.253.195, 5.9.253.196, 5.9.253.197, 5.9.253.198, 5.9.253.199, 5.9.253.200, 5.9.253.201, 5.9.253.202, 5.9.253.203"

the "p2p" field is the coin's peer to peer port, the "magic" is the netmagic of that chain. the decimal value of this can be obtained from the getinfo call, convert to hex and serialize it into the 4 hexbytes. if you got the direction wrong, just flip it around. finally, the "nSPV" field is a list of one or more ip addresses of a nSPV fullnode

the JSON api is very simple and almost a direct mapping of the nSPV=1 rpc commands in the komodod, the testvectors file at ~/libnspv/src/tools/testvectors will show specifically how to call it using curl. any other method to post the JSON to the rpc port will suffice.

https://docs.komodoplatform.com/nSPV has initial docs

How To build in Termux on Android:

Check if your device will use the new Termux repos: https://github.com/termux/termux-packages#information-for-android-7-users

Update and install required packages

pkg upgrade && pkg install build-essential git wget libsodium libevent

Other packages may be required.

Clone and enter the repo

git clone https://github.com/jl777/libnspv/
cd libnspv

Update scripts so target environment determined (from https://wiki.termux.com/wiki/Building_packages):

find . -name 'config.sub' -exec chmod u+w '{}' \; -exec cp -f "${PREFIX}/share/libtool/build-aux/config.sub" '{}' \;
find . -name 'config.guess' -exec chmod u+w '{}' \; -exec cp -f "${PREFIX}/share/libtool/build-aux/config.guess" '{}' \;

To build libnspv and tools for Android use one time script:

./androidonetime.sh

Running nspv on Android

Start nspv
Then visit 127.0.0.1:7771 in your Android browser.

libnspv's People

Contributors

afk11 avatar cris-f avatar deckersu avatar deniod avatar dimxy avatar eklitzke avatar gcharang avatar icota avatar jl777 avatar jonasschnelli avatar laanwj avatar mediovia avatar milerius avatar mixa84 avatar ndrchvzz avatar ns408 avatar promag avatar satindergrewal avatar sirseveng avatar smk762 avatar theblackmallard avatar tkms1122 avatar tonymorony avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

libnspv's Issues

libnspv make delays when sending messages, apparently because of buffering

Tests show that requests via libnspv might perform with delay of several seconds.
Libnspv uses libevent2 library to send/receive messages over net. It looks like libevent2 does buffering internally before actual sending.
I search or something like a flush feature but according to this it is not possible yet:

Currently (as of Libevent 2.0.5-beta), bufferevent_flush() is only implemented for some bufferevent types. In particular, socket-based bufferevents don’t have it. http://www.wangafu.net/~nickm/libevent-book/Ref6_bufferevent.html

Maybe we need to contact libevent2 devs to get more info.

add api to send opreturn data

it might make sense to force having a prevtxid field so that the entire set of (related) opreturns can be retrieved

broadcast fails with timeout error on a specific tx hex

hex: 0400008085202f89013d8fc1f09d8a24b215296e12d434ca31e106871bdb77ff45f8cecdc9979ee26c000000006a473044022047f9f02739528df5196560003db99af1c2e3b9f82bdb2dcad0aa0fb0c9cabe8c022057e27b6fc499007bcadaa8cda7dad6a990bbfc0b7de096ab9b7acd6de78f8588012103ab871db1d39d3f3894df53bc1d3018a243b6768dffdb2a4c038fd92213854b99feffffff01d0e89600000000001976a914fc46abe3748209db896e25faf39090679f17475588acc1ee8d5d000000000000000000000000000000

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "broadcast", "params": ["0400008085202f89013d8fc1f09d8a24b215296e12d434ca31e106871bdb77ff45f8cecdc9979ee26c000000006a473044022047f9f02739528df5196560003db99af1c2e3b9f82bdb2dcad0aa0fb0c9cabe8c022057e27b6fc499007bcadaa8cda7dad6a990bbfc0b7de096ab9b7acd6de78f8588012103ab871db1d39d3f3894df53bc1d3018a243b6768dffdb2a4c038fd92213854b99feffffff01d0e89600000000001976a914fc46abe3748209db896e25faf39090679f17475588acc1ee8d5d000000000000000000000000000000"] }' -H 'content-type: text/plain;' http://127.0.0.1:7771/
{"result":"success","expected":"71ddf2b213fdc7efc60ed1ece6329578858e2c9609da783adabd7baa53fc1128","broadcast":"0000000000000000000000000000000000000000000000000000000000000000","retcode":-2,"type":"timeout","lastpeer":"78.47.196.146:7770"}

skipcount parameter for listunspent and listtransactions calls does not work

'skipcount' parameter has currently no effect.

To reporduce:

  • take address with X amount of utxos/transactions
  • call listtransactions with skipcount = 0
  • call listtransactions with skipcount = N, N > 0

result: calls have same output
expected: 2nd call should have N less transactions/utxos in output

Curls might be used to check (for KMD):

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "listtransactions", "params": ["RBCSvPzYtTZ51RYbsDLKkQ58Mj5TuRRNeh", 0, 0] }' -H 'content-type: text/plain;' http://127.0.0.1:7771/

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "listtransactions", "params": ["RBCSvPzYtTZ51RYbsDLKkQ58Mj5TuRRNeh", 0, 30] }' -H 'content-type: text/plain;' http://127.0.0.1:7771/

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "listunspent", "params": ["RBCSvPzYtTZ51RYbsDLKkQ58Mj5TuRRNeh", 0, 0] }' -H 'content-type: text/plain;' http://127.0.0.1:7771/

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "listunspent", "params": ["RBCSvPzYtTZ51RYbsDLKkQ58Mj5TuRRNeh", 0, 30] }' -H 'content-type: text/plain;' http://127.0.0.1:7771/

Parse error of coins file

Hello,
First let me say thank you for this wallet. It is nicely done. I've installed the wallet. When I run ./nspv file I get "parse error of coins file." The ./bitcointool works perfect.

nspv fails on its shutdown

If nspv executable connected and disconnected from several nodes it might fail on shutdown when node array is freed.
Fixed by clearing node->event_bev on node disconnecting

nSPV client started for coins other than KMD shuts down after a brief output

Using OUR as an example. Found similar behavior for ILN, RICK
Using the conf:

{
    "coin": "OUR",
    "asset": "OUR",
    "fname": "OUR Crypto Union",
    "rpcport": 45672,
    "p2p": 45671,
    "nSPV": "51.255.195.65, 37.187.225.231",
    "magic": "b76686a7",
    "mm2": 1
  }

"51.255.195.65, 37.187.225.231" are seed nodes from launch parameters.
The client shuts down with the following output

Found (OUR) magic.b76686a7, p2p.45671 seeds.(51.255.195.65, 37.187.225.231)
027e3758c3a65b12aa1046462b486d0a63bfa1beae327897f56c5cfb7daaae71 genesisblockhash OUR
Start NSPV_rpcloop.45672
>>>>>>>>>> NSPV_rpcloop 127.0.0.1:45672 bind sock.3 API enabled at unixtime.1580745093 <<<<<<<<<
Discover OUR peers...Trying to connect to 1 51.255.195.65:45671
Trying to connect to 2 37.187.225.231:45671
Connected to node 1 51.255.195.65:45671
Connected to node 2 37.187.225.231:45671
end of client runloop

Launch parameters of OUR for reference:

./komodod -ac_name=OUR -ac_reward=1478310502 -ac_halving=525600 -ac_cc=42 -ac_supply=100000000 -ac_perc=77700 -ac_staked=93 -ac_pubkey=02652a3f3e00b3a1875a918314f0bac838d6dd189a346fa623f5efe9541ac0b98c -ac_public=1 -addnode=51.255.195.65 -addnode=217.182.129.38 -addnode=37.187.225.231

mempool based pruning of utxos: @blackjok3r

during spend rpc function, we can find out that a utxo has been spent via the txproof result. i think if we get such a failure, we can go into a secondary spend logic, which updates the list of utxo based on the already spent. maybe even issue a mempool search.

separately, it can be useful to be able to spend utxo in the mempool. so this issue is the sane handling of the mempool status of utxo

CC tx creation: nSPV_CCtx.h, @blackjok3r, @dimxy, @mihailo

// @blackjok3r @dimxy @mihailo implement the CC tx creation functions here
// instead of a swissarmy knife finalizeCCtx, i think it is better to pass in the specific info needed into a CC signing function. this would eliminate the comparing to all the different possibilities
// since the CC rpc that creates the tx will know which vins are normal and which ones are CC, and most importantly what type of CC vin it is, it will be much simpler finalize function, though it will mean all the CC rpc calls will have to do more work. that was the rationale behind FinalizeCCtx, but i hear a lot of complaints about the complexity it has become.
// please make a new way of doing CC tx that wont lead to complaints later. let us start with faucetget

valgrind failures

There are a handful of valgrind errors reported when valgrind ./tests is run. This issue is present in libbtc, too. [but has now been fixed. also fixed was a bug where when compiled in release mode, random bytes generation could rarely silently fail and make reproducible keys.]

nspv_spend call komodod crash

./komodod -nSPV=1

komodo daemon may crash on nspv_spend call.

Script example to help with reproduction:

#!/bin/bash

./komodo-cli nspv_login  _wif_
sleep 1
while :
do
  ./komodo-cli nspv_logout
  sleep 1
  ./komodo-cli nspv_login  _wif_
  sleep 1
  ./komodo-cli nspv_spend  _spend_addr_  0.01
  sleep 1
done

After 2 to 6 iterations you should get segfault.

Note: komodod does not crash if you just do:
nspv_login
snpv_spend in loop
Issue happens only in case of successive log_in log_out log_in calls.

backtrace examples:
https://paste.ubuntu.com/p/9CzV52djhv/

Upstreaming with libbtc

Hi, I recently acquired merge access with libbtc. However, I have only a little capacity to maintain it.

Your fork seems more active nowadays. Do you have any thoughts on sharing work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.