Giter VIP home page Giter VIP logo

komodoplatform / komodo-defi-framework Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jl777/supernet

93.0 20.0 81.0 216.72 MB

This is the official Komodo DeFi Framework repository

Home Page: https://developers.atomicdex.io

C 0.01% Shell 0.18% Dockerfile 0.01% Rust 99.77% Ruby 0.01% Python 0.01%
blockchain p2p dex atomic-swap atomic-swaps utxo ethereum erc20 qtum qrc20 bitcoin komodo cross-chain cross-chain-swap cross-chain-bridge cross-chain-dex cross-chain-applications

komodo-defi-framework's Introduction

downloads last commit
issues issues closed pulls pulls closed
build status release version chat on Discord follow on Twitter

What is the Komodo DeFi Framework?

The Komodo DeFi Framework is open-source atomic-swap software for seamless, decentralised, peer to peer trading between almost every blockchain asset in existence. This software works with propagation of orderbooks and swap states through the libp2p protocol and uses Hash Time Lock Contracts (HTLCs) for ensuring that the two parties in a swap either mutually complete a trade, or funds return to thier original owner.

There is no 3rd party intermediary, no proxy tokens, and at all times users remain in sole possession of their private keys.

A well documented API offers simple access to the underlying services using simple language agnostic JSON structured methods and parameters such that users can communicate with the core in a variety of methods such as curl in CLI, or fully functioning desktop and mobile applications like Komodo Wallet Desktop.

For a curated list of Komodo DeFi Framework based projects and resources, check out Awesome AtomicDEX.

Features

  • Perform blockchain transactions without a local native chain (e.g. via Electrum servers)
  • Query orderbooks for all pairs within the supported coins
  • Buy/sell from the orderbook, or create maker orders
  • Configure automated "makerbot" trading with periodic price updates and optional telegram alerts

System Requirements

  • 64-bit MacOS, Windows, or Linux operating system
  • 2GB of free RAM (or more)
  • User account with admin/root privileges

Building from source

On Host System:

Pre-built release binaries are available for OSX, Linux or Windows.

If you want to build from source, the following prerequisites are required:

  • Rustup
  • OS specific build tools (e.g. build-essential on Linux, XCode on OSX or MSVC on Win)
  • (Optional) OSX: install openssl, e.g. brew install openssl.
  • (Optional) OSX: run LIBRARY_PATH=/usr/local/opt/openssl/lib
  • (Optional) Linux: Install libudev-dev (dpkg) or libudev-devel (rpm) package.
  • Download or compile protoc 3.21.x+ and add it to your PATH env. It is also available via package managers depending on the OS.

To build, run cargo build (or cargo build -vv to get verbose build output).

For more detailed instructions, please refer to the Installation Guide.

From Container:

If you want to build from source without installing prerequisites to your host system, you can do so by binding the source code inside a container and compiling it there.

Build the image:

docker build -t mm2-build-container -f .docker/Dockerfile .

Bind source code into container and compile it:

docker run -v "$(pwd)":/app -w /app mm2-build-container cargo build

Just like building it on your host system, you will now have the target directory containing the build files.

Building WASM binary

Please refer to the WASM Build Guide.

Configuration

Basic config is contained in two files, MM2.json and coins

The user configuration MM2.json file contains rpc credentials, your mnemonic seed phrase, a netid (8762 is the current main network) and some extra optional parameters.

For example:

{
  "gui": "core_readme",
  "netid": 8762,
  "rpc_password": "Ent3r_Un1Qu3_Pa$$w0rd",
  "passphrase": "ENTER_UNIQUE_SEED_PHRASE_DONT_USE_THIS_CHANGE_IT_OR_FUNDS_NOT_SAFU"
}

The coins file contains information about the coins and tokens you want to trade. A regularly updated version is maintained in the Komodo Platform coins repository. Pull Requests to add any coins not yet included are welcome.

To facilitate interoperability with the mm2 service, there is the adex-cli command line utility. It provides a questionnaire initialization mode to set up the configuration and obtain the proper coin set through the internet. It can also be used to start or stop the service.

Usage

To launch the Komodo DeFi Framework, run ./mm2 (or mm2.exe in Windows)

To activate a coin:

curl --url "http://127.0.0.1:7783" --data '{
	"coin": "KMD",
	"method": "electrum",
	"servers": [
		{"url": "electrum1.cipig.net:10001"},
		{"url": "electrum2.cipig.net:10001"},
		{"url": "electrum3.cipig.net:10001"}
	],
	"required_confirmations":10,
	"requires_notarization":true,
	"mm2":1,
	"userpass": "$userpass"
}'

To view the orderbook for a pair:

curl --url "http://127.0.0.1:7783" --data '{
    "userpass":"$userpass",
    "method":"orderbook",
    "base":"KMD",
    "rel":"BTC"
}'

To place a buy order:

curl --url "http://127.0.0.1:7783" --data '{
  "userpass": "$userpass",
  "method": "buy",
  "base": "KMD",
  "rel": "DOGE",
  "volume": "10",
  "price": "4"
}'

Refer to the Komodo Developer Docs for details of additional RPC methods and parameters

Project structure

mm2src - Rust code, contains some parts ported from C as is (e.g. lp_ordermatch) to reach the most essential/error prone code. Some other modules/crates are reimplemented from scratch.

Additional docs for developers

Disclaimer

This repository contains the work in progress code of the brand new Komodo DeFi Framework (mm2) built mainly on Rust.
The current state can be considered as a alpha version.

WARNING: Use with test coins only or with assets which value does not exceed an amount you are willing to lose. This is alpha stage software!

Help and troubleshooting

If you have any question/want to report a bug/suggest an improvement feel free to open an issue or join the Komodo Platform Discord dev-marketmaker channel.

komodo-defi-framework's People

Contributors

artemgr avatar artemii235 avatar borngraced avatar ca333 avatar caglaryucekaya avatar deckersu avatar fadedreamz avatar flamingice avatar freexploit avatar himu007 avatar jl777 avatar kolobus avatar laruh avatar lukechilds avatar milerius avatar miyurz avatar onur-ozkan avatar oxarbitrage avatar ptyx11 avatar rozhkovdmitrii avatar satindergrewal avatar sergeyboyko0791 avatar shailesh09 avatar shamardy avatar siulynot avatar smk762 avatar st3rling0x avatar tonymorony avatar usamir avatar vineetbhargav86 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

komodo-defi-framework's Issues

Send ETH/ERC20 TX data

Currently in the update socket messages and the tradestatus completion method for ETH/ERC20 swaps we get the ETOMIC TXIDs/amounts but not the ETH/ERC20 values.

It would be great if you could expose the ETH/ERC20 TX data to us.

As an example, this is a current TX update object:

{
  "name": "bobdeposit",
  "coin": "ETOMIC",
  "tx": "0100000001581946dcc0ca9e473d17a7199e2104678d39436dafc25d4002db641039e3cc34010000006b483045022100c311c0efe7ab3665cd8ada46e610e58c262ed079225a108776d171992047c5a602204aa3b22fcefa5c3ddbb2766a023577588daad88a02a13a2a42879e14a755a00e012102f047e14e64689a3464b62d7ed551f5228f7c6d38f61980ab89ace38525fe199effffffff02efa4b4060000000017a9142be18265142ed155d7df201f187ee7642a9ee19287a74f45b9000000001976a914719adc7ec7ad318b2ebfcc500822d2483ab8f8d688acd70f4e5b",
  "txid": "b856c6d7fc0e4bdc89dcafc0e1607fef1f4070e6d7a028226d8c52038e413d87",
  "Bdeposit": "0x2be18265142ed155d7df201f187ee7642a9ee192",
  "expiration": 1531842519,
  "uuid": "413403a1a1907bdf18cd0fbc4ebcfeb8850bb7af640750751e80e29626c7dd73",
  "iambob": 0,
  "bobcoin": "MYTH",
  "bobtomic": "0x277ab4b9dde09a8e710fd755deeb9d0d9532d047",
  "etomicsrc": "0xe675e3bed50682297adc795bf7756e618c32a8f3",
  "bobDepositEthTx": "0xf70a6e90c7cf854cb36df488e733cc6ccea554b8465932c483a0e8ea2420311a",
  "aliceRealSat": "0",
  "bobRealSat": "972872033",
  "alicecoin": "KMD",
  "etomicdest": "0x7b0f414f040f59a743716027f8a24a37dbe1d16f",
  "lock": 0,
  "amount": 1.12502,
  "Apayment": "bTNHjB86Ex1W4m9DEEXUrfppbAcxmsWRHB",
  "redeem": "6304d70f4e5bb17582012088a914891324366afabe2922e518b5c5088a412f9cf1f4882102581cc589fd63fe464a5a60e337f0214e4fcad747c7ecfbb7ecaab84d6f47c83eac6782012088a91460283920463fe8fb6b8f0f0b84097805afab696a882103fb4bea54e3ce957466917e16a5b034dd556fdf4ea971db49593b0b44523d39b7ac68",
  "method": "update",
  "update": "bobdeposit",
  "requestid": 1980583534,
  "quoteid": 2269013680
}

I see there is bobDepositEthTx for the ETH TXID and bobRealSat.

However it would be better if bobDepositEthTx could just be exposed as something like txidEth so we can aways use that over txid if it exists, then we don't have to manually search for the ETH TXID property for each TX type e.g bobDepositEthTx, bobPaymentEthTx etc.

Also bobRealSat appears to be denoted in "satoshis" which doesn't really make sense for ETH/ERC20. This could cause issues if the token has a non standard decimal amount. It would be great if mm could calculate the value based on it's internal ERC20 decimal value and just expose amountEth.

With these solutions we can just rely on using txidEth and amountEth if they exist, if not, fallback to txid and amount.

We also regenerate all trade data from the completed tradestatus message.

An example:

{
  "uuid": "413403a1a1907bdf18cd0fbc4ebcfeb8850bb7af640750751e80e29626c7dd73",
  "expiration": 1531842519,
  "tradeid": 2724183059,
  "requestid": 1980583534,
  "quoteid": 2269013680,
  "iambob": 0,
  "Bgui": "",
  "Agui": "hyperdex",
  "gui": "hyperdex",
  "bob": "MYTH",
  "bobtomic": "0x277ab4b9dde09a8e710fd755deeb9d0d9532d047",
  "etomicsrc": "0xe675e3bed50682297adc795bf7756e618c32a8f3",
  "srcamount": 9.72872033,
  "bobtxfee": 0.00001,
  "alice": "KMD",
  "etomicdest": "0x7b0f414f040f59a743716027f8a24a37dbe1d16f",
  "destamount": 1.00008999,
  "alicetxfee": 0.00001,
  "aliceid": "6016495696859496449",
  "result": "success",
  "status": "finished",
  "finishtime": 1531827408,
  "bobdeposit": "b856c6d7fc0e4bdc89dcafc0e1607fef1f4070e6d7a028226d8c52038e413d87",
  "alicepayment": "c1f2fc2d3f04d114b2369f779b2998d27843f97fea66cf853cdfc9c0c8663dc1",
  "bobpayment": "76b77af9951e12d46f009aa7c97378e9ca76961860aed3271b06adb889b818fb",
  "paymentspent": "e53dcd6f1ae43e8a0404263ec67e552fc2f699ba86a0620b446c924d8837cd0e",
  "Apaymentspent": "0000000000000000000000000000000000000000000000000000000000000000",
  "depositspent": "0000000000000000000000000000000000000000000000000000000000000000",
  "alicedexfee": "3229097506b4baab932412db463441268eea26c37c1c4571aedc3bda07f21a98",
  "method": "tradestatus",
  "values": [
    1.00000999,
    0,
    1.00002,
    1.00011,
    1.12502,
    0,
    0.00128711,
    0,
    0,
    0,
    0
  ],
  "sentflags": [
    "alicespend",
    "bobpayment",
    "alicepayment",
    "bobdeposit",
    "myfee"
  ]
}

This is to ensure we have correct trade data if the mm node went offline, restarted, was reverted etc. We can currently completely rebuild a BTC protocol trade with this data but not an ETH/ERC20 trade.

We need ETH/ERC20 TXIDs and amounts included to be able to do this, currently we only have ETOMIC.

segfault when calling balances

when i call curl --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"balances\"}"
marketmaker v1.0.244 segfaults

Thread 6 "marketmaker" received signal SIGSEGV, Segmentation fault.
__GI_____strtol_l_internal (nptr=0x0, endptr=0x0, base=0, group=<optimized out>, loc=0x7ffff759d560 <_nl_global_locale>) at ../stdlib/strtol_l.c:292

Can't sell ETH or ERC20 tokens

{"error":"cant find a deposit that is close enough in size. make another deposit that is just a bit larger than what you want to trade"}

I have funded the ERC20 tokens and I think ETOMIC UTXOs should be requested automatically from the faucet?

I get the following output from marketmaker:

address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
                other coins loop elapsed  148001.24 millis > threshold    5000.00, ave     519.27 millis, count.6316 postprice
                   KMD coin loop elapsed   45012.81 millis > threshold   10000.00, ave     281.27 millis, count.11760 postprice
LP_autobuy KMD/MYTH price 2.50000000 vol 10.00000000 nonce 0
no more electrum data when expected2 len.0 n.0
continue with select ...
no more electrum data when expected2 len.0 n.0
bindflag.0 iguana_socket mismatch (13.236.219.225) -> (electrum2.cipig.net)
continue with select ...
RECONNECT ep.0x10805a000 EQL numerrors.10 too big -> new 159.65.91.235:10801 sock.98
RECONNECT ep.0x107457000 KMD numerrors.10 too big -> new electrum2.cipig.net:10001 sock.92
no more electrum data when expected2 len.0 n.0
bindflag.0 iguana_socket mismatch (52.63.107.102) -> (electrum1.cipig.net)
continue with select ...
RECONNECT ep.0x107056000 KMD numerrors.19 too big -> new electrum1.cipig.net:10001 sock.106
added 124 of 124 from ETOMIC listunspents
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738
address_myutxopair couldnt find ETOMIC RQm8BNygniNVqVPRYXEosznRZ7uEiooqSm targets 1.00030000 0.00158738

bob waits for failed bobdeposit forever

I saw this happening a couple of times. When bob fails to send out bobdeposit it does not stop, but waits for bobdeposit forever:

send CONNECT for 3224943839-1439130809
MISMATCHED required confirms me.(1 0) vs (0 1) max.(6 6) othermax.(6 6)
NUMCONFIRMS for SWAP alice.1 bob.1, otheristrusted.0 othertrusts.0
CUT VERIFIED
added 25 of 25 from KMD listunspents
second couldnt add address_utxo 46b2222ac40348162b187f0ef05f9efe98c81cccce1e9908a0d1ab16de2f3256/v0 after not finding
dustcombine.1 numpre.0 min0.(nil) min1.(nil) numutxos.25 amount 8.10202447
change 0.00000000 = total 5.36788848 - amount 8.10202447, adjust 0.00000000 numvouts.1, txfee 0.00001000
not enough inputs  5.36788848 < for amount 8.10202447 txfee 0.00001000
rawtx withdraw error? ({"utxotxid":"46b2222ac40348162b187f0ef05f9efe98c81cccce1e9908a0d1ab16de2f3256","utxovout":0,"locktime":1532552422,"txfee":"1000","outputs":[{"bKSrbnmkQ9eM8jGmmZCyVcAFQ9mAEcJqAp":8.10201447}]})
error bob generating 0x7f7c981559ec deposit.23
ETH txid received: 0x0000000000000000000000000000000000000000000000000000000000000000
swapdata rawtx has null txbytes
error from basilisk_swapdata_rawtx.bobdeposit 0x7f7c981559ec len.0
error sending bobdeposit
0 wait for bobdeposit bKSrbnmkQ9eM8jGmmZCyVcAFQ9mAEcJqAp numconfs.1 KMD 0000000000000000000000000000000000000000000000000000000000000000
0 wait for bobdeposit bKSrbnmkQ9eM8jGmmZCyVcAFQ9mAEcJqAp numconfs.1 KMD 0000000000000000000000000000000000000000000000000000000000000000
0 wait for bobdeposit bKSrbnmkQ9eM8jGmmZCyVcAFQ9mAEcJqAp numconfs.1 KMD 0000000000000000000000000000000000000000000000000000000000000000

I used version v1.0.329 of mm.

[v2] Store swaps in a database

Because of Marketmaker v1 limitations, we are forced to store swaps in our own local database in HyperDEX, which causes a lot of complications and it can easily get out of sync with Marketmaker.

Marketmaker v2 should store swaps in its own database, so clients can just query it for swaps. Preferably using SQLite, since it's easy to embed and battle-tested.

Add proper check of addresses equality.

If token address is set in checksum form in coins config the check of token address on etomic swap contracts execution will fail.
Alice ERC20 fee 0x6aacbbac911d8b0e56e4d30f1102290b94e95fbf85fee344b57017f0ef4f9f38 token address 0xe41d2489571d322189246dafa5ebde1f4699f498 is not equal to expected 0xE41d2489571d322189246DaFA5ebDe1F4699F498

Unstable `eth_withdraw` behaviour

eth_withdraw has returned a txid after setting broadcast to 1. However the transactions didn't seem to actually process and I can never view the txid on a block explorer.

Also I'm getting regular marketmaker crashes after issuing eth_withdraw.

Etomic trades don't execute successfully

Describe the bug
A clear and concise description of what the bug is.

We think we are all set. Komodo native, Etomic active. Eth funded (both sides) HYD funded. Etomic funded (plenty utxos). KMD funded. sufficient utxos and Zcredits.
Smart address: RPgJffb45kmNjpfBjkQgsfwsCaXk1q5mta

Autotrade script:

source userpass
curl --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"HYD\",\"rel\":\"KMD\",\"fixed\":0.5,\"margin\":0.00001}"`

We'll start Komodod, Etomic, Marketmaker, Enable, Auto trade.
HYD enters the order book.

Alice (ETH funded, KMD funded, plenty utxos) puts in order.
Alice runs 1.0.7.rc GUI

Bob's console:

1920 uuid.47f55a1bc820de6af4cb7307c685e13b      request   HYD/KMD     1.00039999 ->   0.50060000 ( 0.50040485) | RT.2 3196 n0
1921 uuid.47f55a1bc820de6af4cb7307c685e13b     reserved   HYD/KMD     1.00079863 ->   0.50060000 ( 0.50020552) | RT.2 3196 n0
1922 uuid.47f55a1bc820de6af4cb7307c685e13b      connect   HYD/KMD     1.00079863 ->   0.50060000 ( 0.50020552) | RT.2 3196 n0

And a bit later.

1924 uuid.47f55a1bc820de6af4cb7307c685e13b    connected   HYD/KMD     1.00079863 ->   0.50060000 ( 0.50020552) | RT.2 3196 n0
AUTOPRICE numautorefs.1

Then it just stops.

Meanwhile in Alice' BarterDEX GUI the trade shows as:

realtime
Quote ID: 1299518380
Request ID: 2625546241

As soon as the swap enters 'trade status' in Alice' GUI it reports 'swap never started'
When you try to kickstart it says 'finished. file removed'

Please answer following questions and attach requested info - it'll help to solve issue faster

  • What OS do you use?
    Ubuntu 18.04 LTS (GNU/Linux 4.15.0-20-generic x86_64)
    Alice on Mac 10.13.5
  • What marketmaker version do you run?
    v1.0.261
  • Attach your coins.json config.
    check
  • Provide your enable script with response.
    script:
source userpass
./setpassphrase > /dev/null
curl -s --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"enable\",\"coin\":\"KMD\"}"
curl -s --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"enable\",\"coin\":\"ETOMIC\"}"
curl -s --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"enable\",\"coin\":\"ETH\"}"
curl -s --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"enable\",\"coin\":\"HYD\"}"
curl -s --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"enable\",\"coin\":\"IOP\"}"

respons:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 53781    0 53606  100   175   127k    427 --:--:-- --:--:-- --:--:--  128k
[{"coin":"KMD","installed":true,"height":903375,"balance":4.98598968,"KMDvalue":4.98598968,"status":"active","smartaddress":"RPgJffb45kmNjpfBjkQgsfwsCaXk1q5mta","rpc":"127.0.0.1:7771","pubtype":60,"p2shtype":85,"wiftype":188,"txfee":1000,"zcredits":0,"zdebits":{"swaps":[],"pendingswaps":0}}]
[{"coin":"ETOMIC","installed":true,"height":19711,"balance":1114.31191333,"KMDvalue":0,"status":"active","smartaddress":"RPgJffb45kmNjpfBjkQgsfwsCaXk1q5mta","rpc":"127.0.0.1:10271","pubtype":60,"p2shtype":85,"wiftype":188,"txfee":1000}]
[{"coin":"ETH","installed":false,"height":-1,"balance":0.02643242,"status":"active","smartaddress":"0x72aa81351317c7f677df3abcb52078744d72b649","rpc":"127.0.0.1:80","pubtype":0,"p2shtype":85,"wiftype":188,"txfee":1000}]
[{"coin":"HYD","installed":false,"height":-1,"balance":17.69837806,"status":"active","smartaddress":"0x72aa81351317c7f677df3abcb52078744d72b649","rpc":"127.0.0.1:80","pubtype":0,"p2shtype":85,"wiftype":188,"txfee":1000}]
{"error":"couldnt find coin locally installed","coin":"IOP"}

Note, we don't use IOP in this test.

  • Provide other curl scripts (with responses) which were executed prior to error.
    Autotrade script:
source userpass
curl --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"HYD\",\"rel\":\"KMD\",\"fixed\":0.5,\"margin\":0.00001}"`

respons:
{"result":"success"}

  • Attach full marketmaker console logs (start collecting right after marketmaker execution).
    check

  • Make sure that you don't send your passphrase, userpass and privkeys. Your funds might be stolen if you reveal this info publicly!

  • Provide info for all nodes involved (e.g. if error occurs during atomic swap you should provide info for both Bob and Alice).

mm_console.txt
coins.json.txt

Windows build doesn't run without installing extra dependencies

screen shot 2018-04-07 at 18 56 15

screen shot 2018-04-07 at 18 56 20

Getting this error when running the windows builds. This release specifically: https://github.com/artemii235/SuperNET/releases/tag/marketmaker-v1.0.65

Seems to be depending on the C++ Redistributables.

We're bundling pre-built mm binaries in our GUI. It's not a very good user experience to have them install stuff like the C++ Redistributables too.

Is it strictly required if it’s just pure C code? If you're compiling with Visual Studio, maybe it's something it pulled in during compilation that isn’t really needed?

If it's possible to get a standalone executable, that would be great.

[v2] Call graph

There SHOULD be a way to explore the list of MM functions and see which functions were ported and which are still in C.
Also which ported functions are pure and which are stateful.

It's also important to mark the exits into any programming libraries that are stateful or using I/O.

Portfolio requests block `marketmaker` when network connectivity is lost

Following on from #82

This fix works, if I turn off my internet connection, the next portfolio requests show the old balance.

However the first portfolio request after losing connectivity seems to take a very long time to resolve (~1.3mins). After that initial offline request, all offline requests using the cache are very fast.

This is a big issue due to the fact that marketmaker is single threaded so it completely blocks everything and makes the app unusable for over a minute.

It could possibly be caused by losing network while a request is happening.

Is this just due to an internal timeout? If so do you think you could decrease it?

[v2] Only 1 user can use 1 public IP at once.

ah right
yeah mm only works on one WAN IP
so for example, if someone runs it on a university WiFI
no one else on the university WiFi can use mm until the first person stops
I haven't confirmed this issue personally but someone mentioned it in Slack and James confirmed
and I believe someone else said this is due to nanomsg
I don't know if that's correct though
it doesn't sound right

[v2] Expose values as an object with named properties

In trade objects there is a value property which is an array of numbers:

values: [  
  1.04275855,
  0,
  1.04285855,
  0.10039849,
  1.17319086,
  0,
  0.00012918,
  0,
  0,
  0,
  0
]

It's not clear what these numbers represent without reading the source:

static char *txnames[] = { "alicespend", "bobspend", "bobpayment", "alicepayment", "bobdeposit", "otherfee", "myfee", "bobrefund", "bobreclaim", "alicereclaim", "aliceclaim" };

I think it would be neater if these values were exposed as an object with named properties. That way it's self documenting, very easy to understand and easier to update without breakage.

"values":{
  "alicespend":1.04275855,
  "bobspend":0,
  "bobpayment":1.04285855,
  "alicepayment":0.10039849,
  "bobdeposit":1.17319086,
  "otherfee":0,
  "myfee"0.00012918,
  "bobrefund":0,
  "bobreclaim":0,
  "alicereclaim":0,
  "aliceclaim":0
}

Previous discussion here: jl777#764

show `avgvol`/`maxvol` in ETH/ERC20 orders

Currently avgvol/maxvol values for ETH/ERC20 order book entries are always 0.

Is there any way we can show data on this?

I’m aware ETH doesn’t actually use UTXOs so we can’t have UTXO count. But couldn’t we get avgvol from previous ETOMIC trades? And maxvol could just be the balance? Or does it not work like that?

[v2] Transparently auto-split UTXOs for orders

If user has the correct balance to place a trade, they shouldn't have to worry about having correctly sized UTXOs, marketmaker should auto split into the correct size.

Say for example I have a single UTXO with a value of 15 KMD. If I want to trade 10 KMD => 0.2 LTC, I should be able to just place a trade and mm would instantly respond with a success message because it can see I have enough balance. Then it should split:

1. UTXO 15 KMD

into:

1. UTXO 10 KMD (for alicepayment)
2. UTXO 0.01 KMD (for dexfee)
3. UTXO 4.99 KMD (change)

And use those new UTXOs for the trade.

This should be completely transparent to the user. It could maybe even be factored into the first step of the swap protocol as an autosplit TX.

So you would have:

autosplit, myfee, bobdeposit, alicepayment, bobpayment, alicespend.

Maybe autosplit and myfee could be combined into the same TX?

Even if the user already has a UTXO that would work wit the existing logic (because it's over the required size), autosplit should still happen. This prevents a user with two large UTXOs locking up a large UTXO in a small swap.

Unclear error feedback

When trying to do an instantdex_deposit I noticed we filled in amount with 1, I get back an error message like the below.

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Origin: *

{"error":"instantdex_deposit needs to have weeks and amount"}

searching the docs, I read that the minimum is 10 KMD. The feedback is unclear and could cause a lot of time being wasted by the one using the application

[v2] Gather crash reports

  • Functional test (those things tends to be flaky, we need to automatically test that crashes are caught).
  • should] Targeted at the binary Docker deployments and internal testing there should be an option of letting the OS dump the core instead of manually collecting the stack trace. No need to upload the core anywhere, at least not withing this issue timeframe: the testing developers will get it manually from the image.
  • Handle Windows exceptions (example, _EXCEPTION_POINTERS, ru, StackWalker). This should speed up the development by allowing me to test the crash handling code from my primary development environments. And will work for the Windows deployments of MM. => POC.
  • -> A full Windows build is likely necessary. Otherwise we'll be getting linking errors with things like os_portable::OS_init missing.
  • -> -> Try to link the C MM1 library to the MM2 Rust binary instead of the other way around.
  • Handle Unix signals (having the Linux and macOS deployments in mind).
  • should] Capture not just the C crashes but also the Rust panics. => Global panic handlers aren't stable yet (#[panic_handler]). We could set thread-local hooks, but I'll try a simpler (?) route first of using RUST_BACKTRACE and getting crash reports by capturing the standard output.
  • -> Dump C/Rust backtraces to standard output and save-to-share log.
  • -> -> Rehash invariants regarding the logging of sensitive information.
  • Backtrace without line numbers first, to improve reliability? => Not an option.
  • Scan the logs folder and see if some of the logs might constitute a crash or a failure.
  • won't] Watchdog. Mark the log dirty when we're doing something and clean when we're finished. That way we can know that there was a failure even if we were too dead to capture the backtrace in the log. We'll probably need a helper, a forked process that leaves a trace that the computer and filesystem were online. If the system was online but MM failed to leave the clean mark then something went wrong and the log might help us figure it out. It's important to leave the clean mark whenever MM is killed. Won't do it in this issue, but should probably create a separate one.
  • Make sure it works on macOS, HyperDEX team mostly deploys there.
  • -> Test the macOS build?
  • -> CI macOS build?
  • -> Remove the strip from the builds, turn debug options on.
  • Figure out where to send/store them. => I'd like to use the Google Cloud Storage for that. Should ask Artem for alternatives. => For starters it might be good enough if they're printed to stdout.
  • Consider implementing this for MM1 too. => The plan is to make a new release of MM reusing the same CI build chain.

quote validate error -3

16410897112016945153 >>>>>>> myprice 0.34920189 qprice 0.34939785 r.42 range 0.00019595 -> 0.34928419, bestprice 0.34928419 counter.-1
priceiter.0 voliter.0 price 0.34928419 vol 0.34956428
Could not find tx out: bfe38eda022ba85fca5c9aa8cd9cd973bfe0a6c4dafc891096ddf4a9a0398bed 1
alice not eligible 1.00085188 -> dest 0.34956428 0.34926675 (0.45041572 0.00217062) bfe38eda022ba85fca5c9aa8cd9cd973bfe0a6c4dafc891096ddf4a9a0398bed/v1 4187f93708aa4068c328d6c30fe1b91e6f9073d619a8352b4e262cb62c2b5016/v1
quote ZILLA/KMD validate error -3

This happens on Bob's side when it's validating Alice quote KMD utxos with low number of confirmations.

Second couldn’t add address error

Excerpt output from mm. Trade did not go through, but dex fee was sent.

CONNECTED mpnet.0 fill.0 gtc.0 numpending.0 tradeid.794971430 requestid.267556293 quoteid.240213557 pairstr.tcp://173.212.225.176:29292
ETH/KMD bid 400.40000000 ask 0.00000000 values 0.00000000 0.00000000
>>>>>>>>>> jumblrflag.-2 <<<<<<<<< r.267556293 q.240213557, 1.00000000 bobconfs.1, 30.00009000 aliceconfs.0 taddr.0 0
etomic src (0x0000000000000000000000000000000000000000 0x4e623b150b847da59f064f7c7fcc9f787dae2229) dest ( 0xec3ede410f3ed859bf53da2706e437683dfd17e8)
alice pairstr.(tcp://173.212.225.176:29292) pairsock.12 pthread_t 8
start swap iamalice pair.12
MISMATCHED required confirms me.(0 1) vs (1 1) max.(6 6) othermax.(6 6)
NUMCONFIRMS for SWAP alice.1 bob.1, otheristrusted.0 othertrusts.0
CUT VERIFIED
added 18 of 18 from KMD listunspents
second couldnt add address_utxo ffd07a5a31c0e34cf51105adbfcff039d44aa03dd653f3f27b984e444f90ea13/v0 after not finding
dustcombine.1 numpre.0 min0.0x0 min1.0x0 numutxos.18 amount 30.00012000
84bd1ddf1dc2d545023a8b887cfa489871889c930d1009c5bd159fd6f4e08c7a/3 73.12500000 interest 0.21505385 -> sum 0.21505385
added 18 of 18 from KMD listunspents
second couldnt add address_utxo 8a073aaedecb5d1705f98667dbbf29c04a7d0c16bf830bd3af1d019b05bfe6ea/v1 after not finding
dustcombine.1 numpre.0 min0.0x0 min1.0x0 numutxos.17 amount 0.03862015

Make `eth_withdraw` more similar to `withdraw`

We use withdraw with {broadcast: 0} to get the txfee. Then we show the txfee to the user and get a manual confirmation before we broadcast with sendrawtransaction.

Currently we can't do this with ETH/ERC20 because eth_withdraw just isntantly broadcasts.

Would it be possible to make it behave the same way as withdraw so we can allow the user to confirm the txfee?

Balances are not showing up

When running the balances rpc method, ETH and ERC20 balances don't show up.

curl --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"balances\"}"

[v2] CMC paid API

cipi quotes from CMC: "The Public API will be taken offline on December 4th, 2018. Please migrate your application to the free tier of the Professional API."

We might ask users to register for a free API key, but this might be suboptimal in terms of UX.
Can we register automatically?
Can we share the information across the network?
Do we need a private paid CMC proxy?
Are there alternatives to CMC?

"Except as expressly permitted by CMC above, you shall not, and shall not permit any End User, to: ... Copy, manipulate or aggregate any Content (including data) for the purpose of making it available to any third party; Use or introduce to the Service any data mining, crawling, "scraping", robot or similar automated or data gathering or extraction method, or manually access, acquire, monitor or copy any portion of the Service, or download or store Content (unless expressly authorized by CMC);" - https://pro.coinmarketcap.com/user-agreement

Interestingly, there's a lot of negative feedback on CMC at https://news.bitcoin.com/8-alternatives-to-coinmarketcap/


Artem's todo list.

  • Setup an (automatic) test that would reach the autopricing code.
  • → Log stack traces to verify how the CMC code is being invoked.
  • → Allow for two MMs to run on the same host and netid (on different IPs).
  • Top-down port towards LP_CMCbtcprice.
  • Coingecko implementation.
  • "allow the user to configure coinmarketcap login info, so if it is there it is used, if not just the free services" ("then serious marketmakers would get a CMC account for pricing").
  • → should] Test the CoinMarketCap mode.
  • Add logging into the autopricing code, allowing us to better test it from the integration test.
  • → Dump dashboard to log on drop.
  • → should] Log the underlying pricing service delays if any.
  • could] Produce a call-graph of MarketMaker functions invoked by that code (#125 and it might also help with packaging design).
  • Create a separate crate for autoprices (we're splitting the MM2 code into crates in order to reduce the costs of the edit-compile-test cycle).
  • → Add test modules to the binary crate (tests are binaries and need to link with the C libraries, therefore the test modules should be a part of the binary crate, mm2.rs, but it also makes sense to move them into subcrate-specific module files). ⇒ Moving C bindings to the helpers crate might be a better idea, it will allow subcrates to use the C code without duplicating the build.rs and will let them keep their test too.
  • should] Make sure the GUI gets a specific error code [and message] if the underlying pricing service fails to respond. This will allow the user to switch the services in a pinch.
  • could] Test lp_autoprice_iter under a faulty network and see if it recovers.
  • Port the rest of LP_autoprice_iter and the arbitrary coin LP_CMCbtcprice invocation there. Refactoring the price fetching to work with a list of arbitrary coins.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.