comit-network / waves Goto Github PK

View Code? Open in Web Editor NEW

15.0 3.0 6.0 3.05 MB

Workspace for project waves - bringing DeFi to Liquid.

License: MIT License

Rust 58.74% Shell 1.95% JavaScript 0.50% HTML 1.27% CSS 0.55% TypeScript 36.99%

waves's Introduction

Project Waves

Building Defi for Bitcoin on Liquid.

What you will find here

This project includes:

bobtimus: a daemon which acts as an automated market-maker offering buy and sell L-BTC/L-USDt trades based on a rate pulled from Kraken. It also serves a website which acts as an interface for browser extensions to reach bobtimus' HTTP API.
waves_wallet: a Liquid wallet as a browser extension. When visiting bobtimus' website, a waves_wallet user can perform L-BTC/L-USDt atomic swaps in a couple of clicks.

Try it out on regtest

Environment Setup

Install nigiri from https://github.com/vulpemventures/nigiri

curl https://getnigiri.vulpem.com | bash

For our scripts you will need jq as well. You can get it from https://stedolan.github.io/jq/.

Start nodes:

./waves-scripts start_nodes

For our tools to work we need to know which asset ID is the native asset (i.e. Bitcoin).

./waves-scripts load_native_asset

This will create a file in your directory with an asset ID in it called .native_asset_id

If successful, we need to mint a new asset. This asset will be our USDT asset for the purpose of atomic swaps and lending. The asset ID of this new asset will be in .usdt_asset_id

./waves-scripts mint_usdt

Start the Maker (Bobtimus)

Once the environment setup is done you can go ahead and start bobtimus:

./waves-scripts start_bobtimus

This will start bobtimus in a detached mode. Its PID can be found in .bobtimus and the logs in ./logs/bobtimus.

While bobtimus is hosting a production version of waves on http://localhost:3030 you probably want a development build while working on it. For that run the following command and keep the terminal open. Your waves application will be reachable under http://localhost:3004.

Note that you can use this in the browser that includes the web-extension (see next section). For trading with Bobtimus the web extension is required, so you will have to complete running the taker web extension

./waves-scripts start_webapp

Run the Taker Web Extension

Last but not least, you will need the web extension:

Make sure you are in the directory of the web extension: cd extension
Make sure you installed nodejs dependencies: yarn install
Build the web extension, including wasm build of the web extension's wallet, run: yarn watch
- On MacOs we encountered probems with the default clang compiler.
  - You can overcome this by installing llvm: brew install llvm
  - Then run: CC=/usr/local/opt/llvm/bin/clang AR=/usr/local/opt/llvm/bin/llvm-ar yarn watch
Once yarn watch has Completed successfully run: yarn start

If everything was successfully, you now see a firefox browser with the extension enabled and the browser console being opened. The latter one is useful for degbugging purposes as it will print all kinds of stuff.

What's coming

With atomic swaps serving as the foundation, financial products such as borrowing and lending are on the horizon.

Where to reach us

If you have any questions about this project or want to keep up with its development, join the COMIT-Liquid Matrix channel.

waves's People

Contributors

Stargazers

Watchers

Forkers

thomaseizinger bonomat kukks liquidao klochowicz

waves's Issues

Yarn test blows up

When running yarn test a giant error message is shown. This does not effect the test though:

See waves_test here for example: https://github.com/comit-network/droplet/pull/110/checks?check_run_id=1660725116

  ● Console

    console.error
      Warning: An update to ConfirmSwapDrawer inside a test was not wrapped in act(...).
      
      When testing, code that causes React state updates should be wrapped into act(...):
      
      act(() => {
        /* fire events that update state */
      });
      /* assert on the output */
      
      This ensures that you're testing the behavior the user would see in the browser. Learn more at reactjs.org/link/wrap-tests-with-act
          at ConfirmSwapDrawer (/home/runner/work/droplet/droplet/waves/src/ConfirmSwapDrawer.tsx:34:45)
          at div
          at App (/home/runner/work/droplet/droplet/waves/src/App.tsx:172:21)
          at Router (/home/runner/work/droplet/droplet/waves/node_modules/react-router/cjs/react-router.js:99:30)
          at BrowserRouter (/home/runner/work/droplet/droplet/waves/node_modules/react-router-dom/cjs/react-router-dom.js:67:35)
          at SSEProvider (/home/runner/work/droplet/droplet/waves/node_modules/react-hooks-sse/dist/cjs/SSEContext.js:34:23)

      at printWarning (node_modules/react-dom/cjs/react-dom.development.js:67:30)
      at error (node_modules/react-dom/cjs/react-dom.development.js:43:5)
      at warnIfNotCurrentlyActingUpdatesInDEV (node_modules/react-dom/cjs/react-dom.development.js:24064:9)
      at dispatchAction (node_modules/react-dom/cjs/react-dom.development.js:16135:9)
      at node_modules/swr/dist/index.js:1:7507
      at node_modules/swr/dist/index.js:1:9245
      at step (node_modules/swr/dist/index.js:1:3918)
      at Object.throw (node_modules/swr/dist/index.js:1:3203)

    console.error
      Warning: An update to App inside a test was not wrapped in act(...).
      
      When testing, code that causes React state updates should be wrapped into act(...):
      
      act(() => {
        /* fire events that update state */
      });
      /* assert on the output */
      
      This ensures that you're testing the behavior the user would see in the browser. Learn more at reactjs.org/link/wrap-tests-with-act
          at App (/home/runner/work/droplet/droplet/waves/src/App.tsx:172:21)
          at Router (/home/runner/work/droplet/droplet/waves/node_modules/react-router/cjs/react-router.js:99:30)
          at BrowserRouter (/home/runner/work/droplet/droplet/waves/node_modules/react-router-dom/cjs/react-router-dom.js:67:35)
          at SSEProvider (/home/runner/work/droplet/droplet/waves/node_modules/react-hooks-sse/dist/cjs/SSEContext.js:34:23)

      at printWarning (node_modules/react-dom/cjs/react-dom.development.js:67:30)
      at error (node_modules/react-dom/cjs/react-dom.development.js:43:5)
      at warnIfNotCurrentlyActingUpdatesInDEV (node_modules/react-dom/cjs/react-dom.development.js:24064:9)
      at dispatchAction (node_modules/react-dom/cjs/react-dom.development.js:16135:9)
      at node_modules/swr/dist/index.js:1:7507
      at node_modules/swr/dist/index.js:1:9245
      at step (node_modules/swr/dist/index.js:1:3918)
      at Object.throw (node_modules/swr/dist/index.js:1:3203)

PASS src/RateService.test.tsx

[Wallet] Allow arbitrary assets

At the moment our wallet understands two different assets: L-USDT and L-BTC (sometimes referred to as the native asset).
The according asset IDs are configured in the browser's localstorage and can be changed through the extension's option page our through setting certain environment variables during compile time.

There are a few limitations here:

Unknown assets are not detected by the wallet
If an asset is wrongly configured it is not shown (as it is basically unknown)
Bobtimus might have a different opinion on asset Ids

This leads to some random errors where the wallet cannot participate in a swap because it does not know about the assets Ids.

Ideally, the wallet knows about all assets for which it owns the key to.

waves-scripts: Allow to restart individual components without tearing down the whole environment

Recently, I did have to restart bobtimus several times while developing. Starting bobtimus requires the USDT asset ID.

Unfortunately, there is currently no way of restarting bobtimus with the same asset ID. It would be nice if one could do that.

Frontend: use bobtimus rate service

Frontend should get the rate from bobtimus via SSE

Wallet: Make `AddressParams` configurable through env variable with default to `Liquid`

Wallet: Handle 404 from esplora when fetching UTXOs

Wallet: `WithdrawAll` Fails

Broadcasting the transaction failed with:

sendrawtransaction RPC error: {"code":-26,"message":"bad-txns-in-ne-out, value in != value out (code 16)"}

Frontend

Mock: https://www.figma.com/proto/MxcADlWgz6Z8XdwXwFuzVU/BTC%2FUSDT?node-id=2%3A81&scaling=scale-down

implement simple ui according to the mock without the browser extension
build variables
- ~~for asset ids (bitcoin, usdt)~~. not needed as returned by wallet
- ~~for esplora (blockstream vs localhost)~~ dealt with in wallet

Extend CI to upload fuzzer results in case it fails

Improve "swapped" page

After a successful swap, we redirect the user to a "swapped" page where they can click on a link to the blockexplorer.

Other than that, this page does not display anything useful.

We should improve that by saying something like: "Your swap is complete" etc.

Bobtimus

Bobtimus is in the role of Bob

rate feed: publish fixed rate every X seconds
api to implement protocol steps
- interaction with elementsd
- user provides an elementsd node with an existing wallet
- we expect the wallet to remain unlocked forever

Show swap data based on decomposed transaction

Basically this TODO here: https://github.com/thomaseizinger/droplet/blob/8b65bd1dc40818cb9e16cfc27f70f3a8baea3744/waves/src/ConfirmSwapDrawer.tsx#L45-L49

Move code from `elements-fun` into `rust-elements`

It turns out that there isn't as much need for most of the re-modelling that we did in elements-fun as we initially thought. See more here.

This is a tracking issue for moving all of what we did in elements-fun back into rust-elements (non-exhaustive list):

Add a dependency to rust-secp256k1-zkp and model asset/value commitment using types from rust-secp256k1-zkp
Create TxOut constructors for new_last_confidential / new_non_last_confidential
Introduce .unblind API on TxOut
Introduce ExplicitTxOut and ConfidentialTxOut along with into_{explicit,confidential} on TxOut

WithdrawEverything fails

This needs some more investigation:

When calling withdraw_everything_to I receive the following error in the browser console:

WalletInfo.tsx:41 Uncaught (in promise) error decoding response body: missing field `1` at line 1 column 2: missing field `1` at line 1 column 2
withdraw @ WalletInfo.tsx:41
async function (async)
withdraw @ WalletInfo.tsx:39
callCallback @ react-dom.development.js:3945
invokeGuardedCallbackDev @ react-dom.development.js:3994
invokeGuardedCallback @ react-dom.development.js:4056
invokeGuardedCallbackAndCatchFirstError @ react-dom.development.js:4070
executeDispatch @ react-dom.development.js:8243
processDispatchQueueItemsInOrder @ react-dom.development.js:8275
processDispatchQueue @ react-dom.development.js:8288
dispatchEventsForPlugins @ react-dom.development.js:8299
(anonymous) @ react-dom.development.js:8508
batchedEventUpdates$1 @ react-dom.development.js:22396
batchedEventUpdates @ react-dom.development.js:3745
dispatchEventForPluginEventSystem @ react-dom.development.js:8507
attemptToDispatchEvent @ react-dom.development.js:6005
dispatchEvent @ react-dom.development.js:5924
unstable_runWithPriority @ scheduler.development.js:646
runWithPriority$1 @ react-dom.development.js:11276
discreteUpdates$1 @ react-dom.development.js:22413
discreteUpdates @ react-dom.development.js:3756
dispatchDiscreteEvent @ react-dom.development.js:5889

Wallet: Add function to decode Bob's response so the frontend can display the verification screen

Frontend: create & sign swap

Implement process to create swap:

call create swap on wallet #67
Post create swap message as received from wallet to bobtimus.
sign and publish transaction with wallet

Reduce custom styling

We currently using several custom styles for buttons and other things by introducing "variants" in the theme:

https://github.com/comit-network/droplet/blob/85094e61af25d6894803e8eaad45b90bc9e06cee/waves/src/theme.tsx#L34-L63

This easily leads to inconsistent usage of colors and doesn't scale.

We should pick a theme (i.e. a color palette) to our liking and use the built-in variants of buttons instead.

Run tests on MacOS CI

This PR adds a build for MacOS but does not run tests because MacOS on Github CI does not have docker.

Add api prefix to bobtimus endpoints

Verify frontend wallet's {BTC, USDt} asset ID at compile time

Originally posted by @thomaseizinger in #134 (comment)

With #134 we no longer verify the validity of {NATIVE, USDT}_ASSET_ID at compile time. This means we can build the application with wrong values and only fail during execution time.

Make wallet more discoverable

Introduce "Connect" button at the same location as balances if wallet is not available or locked
Center button always says "Swap", is greyed out if wallet is locked / not available

Re-enable MacOS CI caching

Infrastructure: Development environment with local esplora and elementd

Esplora URL for the wallet can be set using ESPLORA_URL env variable at build time: https://github.com/comit-network/droplet/blob/96687ef6f0c9acd3795cfcab03bb6fc427c5ddab/waves/wallet/src/esplora.rs#L9

Bobtimus: Embed frontend and serve via warp

Bobtimus seems to not be able to handle more than one swap

Getting this error when trying to do a 2nd swap:

[2021-01-06T00:32:40Z ERROR bobtimus::http] Failed to handle create swap: failed to prove surjection

Frontend: use `debug` instead of console.{log,warn, ...}

https://github.com/visionmedia/debug

It's not possible to type decimal numbers into the amount field

Once deleted, one can only sell a fraction of a BTC by using the arrow keys.

Infrastructure: Automated e2e test

Build wallet in release mode (with ESPLORA_URL set to local node)
Build bobtimus in release mode (that should embed the frontend build artifacts, see #33)
Start elementsd, esplora and bobtimus
Run e2e test (written using cypress) against frontend served through bobtimus
do funding through fake-bobtimus faucet endpoint from within the test

Cannot let warp handle deserialisation of CreateSwapPayload

Because it contains a SecretKey, which only can be deserialised from borrowed strings.

Bobtimus erronously returns a 404 if creating a swap fails

Related to #91.

Getting a 404, should probably be a 400 or 500, depending on what the root-cause is.

Update metadata of app

We should update HTML description, title and keywords to meaningful things.

Wallet: Format amounts as in nominal currency instead of sats

Reactivate fuzz tests

They currently fail and I believe that is because I am using debug_assert_eq within deserialize functions. These should be regular deserialization errors instead.

Make forms submittable

All forms should be submittable with Enter.

Do do this, all event handlers of forms should be <form onSubmit={}> and the button inside the form has to be type = submit.

elements-fuzz does not build on MacOS

The crate elements-fuzz does not build on MacOS. See here: https://github.com/comit-network/droplet/runs/1587089387

excerpt:

  Compiling wasm-bindgen-backend v0.2.69
error: failed to run custom build command for `afl v0.3.2`

Caused by:
  process didn't exit successfully: `/Users/runner/work/droplet/droplet/target/debug/build/afl-030fb788fb95fc15/build-script-build` (exit code: 101)
  --- stdout
  rm -f afl-gcc afl-fuzz afl-showmap afl-tmin afl-gotcpu afl-analyze afl-as as afl-g++ afl-clang afl-clang++ *.o *~ a.out core core.[1-9][0-9]* *.stackdump test .test test-instr .test-instr0 .test-instr1 qemu_mode/qemu-2.10.0.tar.bz2 afl-qemu-trace
  rm -rf out_dir qemu_mode/qemu-2.10.0
  /Applications/Xcode_12.2.app/Contents/Developer/usr/bin/make -C llvm_mode clean
  rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 
  rm -f ../afl-clang-fast ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../afl-clang-fast++
  /Applications/Xcode_12.2.app/Contents/Developer/usr/bin/make -C libdislocator clean
  rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
  rm -f libdislocator.so
  /Applications/Xcode_12.2.app/Contents/Developer/usr/bin/make -C libtokencap clean
  rm -f *.o *.so *~ a.out core core.[1-9][0-9]*
  rm -f libtokencap.so
  [*] Checking for the ability to compile x86 code...
  [+] Everything seems to be working, ready to compile.
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-gcc.c -o afl-gcc 
  set -e; for i in afl-g++ afl-clang afl-clang++; do ln -sf afl-gcc $i; done
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-fuzz.c -o afl-fuzz 
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-showmap.c -o afl-showmap 
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-tmin.c -o afl-tmin 
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-gotcpu.c -o afl-gotcpu 
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-analyze.c -o afl-analyze 
  cc -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" afl-as.c -o afl-as 
  ln -sf afl-as as
  [*] Testing the CC wrapper and instrumentation output...
  unset AFL_USE_ASAN AFL_USE_MSAN; AFL_QUIET=1 AFL_INST_RATIO=100 AFL_PATH=. ./afl-clang -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/lib/afl\" -DDOC_PATH=\"/share/doc/afl\" -DBIN_PATH=\"/bin\" test-instr.c -o test-instr 
  echo 0 | ./afl-showmap -m none -q -o .test-instr0 ./test-instr
  echo 1 | ./afl-showmap -m none -q -o .test-instr1 ./test-instr

  Oops, the instrumentation does not seem to be behaving correctly!

  Please ping <[email protected]> to troubleshoot the issue.


  --- stderr
  make: *** [test_build] Error 1
  thread 'main' panicked at 'assertion failed: status.success()', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/afl-0.3.2/build.rs:29:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
Error: Process completed with exit code 101.

Rounding error when computing USDT?

This doesn't look right 😅

[UI Test] Wait for balance in a smart way

Right now we use cypress to wait for the swap button to be enabled which happens once we have enough funding. We hard-coded a max timeout of a few seconds which might or might not be enough as we could see on some CI runs.

We should wait for the balance to be recognized in a smart way.

Fix precision for USDT

Currently it defaults to 0 because the registry doesn't return anything.

Related #106 #102.

Wallet: withdraw all function

Consider serving waves frontend under its own namespace

Rationale: future-proof against route-conflicts between api, assets and frontend routes

adapt warp to serve assets (excluding index.html) under namespace
re-configure webpack to generate appropriate relative links (likely a package.json config)
actual name of namespace is not that important (i.e. choose something meaningful)

Use local block explorer on regtest

We have a block-explorer running locally using nigiri, use that instead of defaulting to blockstream.info

Wallet: Initialize logger only if specific variable is set in local storage

Listen on localstorage key wallet_log
Parse value as log level
No key means off

[UI Test] Verify amounts in sign-with-wallet screen

See TODO in code: https://github.com/comit-network/droplet/blob/f58612c1bc5d22ecd0127471f77e421315fe832e/waves/cypress/integration/e2e.spec.ts#L28

Infrastructure: Configure nigiri with a "registry" so the registry endpoint returns ticker labels & precisions

Disable swap button if balance lower than swap amount

Otherwise it blows up.

Frontend Wallet

only manage certain assets (i.e. bitcoin, usdt)

create new wallet which takes some kind of randomness
- save seed in localstorage
get new address from wallet to fund
get balance for multiple assets
protocol steps
- create_new_swap: to initiate a new swap
- finalize_swap: to finalize a swap
  - publish_transaction
utxo management against esplora
- fetch utxos for address
- store utxos in localstorage
(optional) encrypt local storage