Giter VIP home page Giter VIP logo

doppler's Introduction

Doppler (A Lightning Domain-Specific Language)

  • GOAL: Create a DSL for Lightning that enables users to easily experiment with and test against various Lightning implementations. Whether it's for discovery testing, inclusion in a comprehensive integration testing suite, or simply investigating the interactions between different components. Given the potential occurrence of intriguing and unique issues across different implementations of Lightning, this project aims to simplify the lives of application developers reliant on the network.

The DSL should empower developers to compose a concise script that configures an entire cluster of nodes running a single docker network, even if they are of different implementation types, to suit precise testing requirements. This should provide a sensation similar to working with a set of Lego blocks, where all the necessary components are at your fingertips, ready to be assembled based on the idea at hand.

Additionally, this DSL can be used against a cluster of remote LND nodes (more implementations will follow) to generate activity across them. This could be payments or channel related activity at the moment, but more work can be done to further expand this to include starting and stopping the remote nodes as well as funding them from a configured faucet. Still would not recommend running these doppler files on a mainnet cluster of nodes, but we wont stop you from being reckless 😉

How to use:

  • More information on how to use this tool can be found here: USAGE.md

Supports:

  • creating a cluster of bitcoind nodes
  • setting up one or many as miners on a provided time interval
  • setting up and funding a cluster of LND nodes, backed by a specified bitcoind node
  • outputing the cluster configuration as a docker-compose
  • making all the logs/data of the nodes available to the running of doppler
  • allowing to set values in LND's native configuration file
  • setup all the networking deterministically
  • OPEN_CHANNEL
  • SEND_LN (amp/keysend/bolt11 via subcommand)
  • SEND_ONCHAIN (only taproot addresses)
  • LOOP a set of commands over an optional interval
  • CLOSE_CHANNEL
  • support multiple node implementations (supports LND, CoreLN, Eclair)
  • add a cluster level UI to see how all the nodes connect (comes from https://github.com/litch/lightning-conformance/tree/master/operator)
  • SETTLE_HOLD_LN -- only works for LND nodes
  • SEND_HOLD_LN -- only works for LND nodes
  • SEND_COINS -- to send from a btc miner to any of the L2 node types (helpful in making sure there are enough funds for channels to open)
  • FORCE_CLOSE_CHANNEL - forces an L2 node to close a give channel
  • TAG - allows for hodl invoices payment hashes to be stored between doppler files being run, enable a shared state between files (these will be used in the future to enable closing a specific channel with another node instead of just picking one at random that the two nodes share)
  • STOP_BTC - stops a BTC container
  • START_BTC - starts a BTC container
  • STOP_LN - stops a LN container
  • START_LN - starts a LN container
  • WAIT BLOCKS - uses one of the hooked up LND nodes and waits to proceed in the script until a certain block height is reached

Interesting Simulations

Acknowledgments

  • Thank you polar for having such easy docker images to work with
  • Thank you litch for creating lightning-conformance which helped inspire this project
  • Thank you w3irdrobot for getting this project started and helping pick the tool to build the grammar in rust

Disclaimer

  • This is very much a work in progress, but feel free to use what works today and PR's are welcome!

doppler's People

Contributors

litch avatar tee8z avatar thebrandonlucas avatar thinkmassive avatar w3irdrobot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

doppler's Issues

Add/Remove nodes in doppler file

We should make it possible to add or remove nodes in a for loop during the actual "script running" section of the doppler file. If we can enable this, some much more interesting scenarios will be able to be created just in the doppler file.

Add binary builds to release

Add a github action that builds the Doppler binary and packages up the example files + readmes into the tar file on the release page when a new release is created.

Improve grammar for CLOSE_CHANNEL

We should add a channel alias to make it easy to define which channel we actual want to close. This should be an optional alias, maybe we don't care about the channel alias (will allow files to be shorter).

Add grammar for BOLT12

This should make it possible to send BOLT 12 invoices through different implementations. It is okay if the PR does not add support for LND as that implementation requires LNDK to be running along side of it which is a much bigger change required. However, if someone has the inclination to tackle both adding LNDK plus all the other implementations bolt12 for the grammar it would be appreciated. To start, take a look at SEND_LN command and see if there would be a good way to signify bolt12 maybe like how we do for amp or keysend at the moment. Any addition needs to be backwards compatible with the grammar already created so run through the many example script with the new grammar to verify everything works as expected.

Get Subcommands working for SEND_LN

Right now any subcommand/flags being passed to the SEND_LN is being ignored. This leads to a confusing situation where you're expecting to see an amp or keysend payment when really a normal invoice is made instead. Either we need to make a custom grammar for the different type of payments or we need to get the subcommands to work.

Add grammar to docs

It would be nice if there was a table of each doppler command and its definition & usage somewhere, with definitions mapped to their bitcoin-cli/lnd etc. counterparts.

A lot of it is fairly intuitive, but it would speed up the creation of custom doppler files.

Is it possible to use this for onchain only?

I am trying to setup 2 wallets with UTXOs in order to do a payjoin, like so:

BITCOIND_MINER sender
BITCOIND receiver

UP
sender MINE_BLOCKS 200
sender SEND_ONCHAIN receiver AMT 100

Here is the output:

RUST_BACKTRACE=1 cargo run --bin doppler -- -f "doppler_files/payjoin_setup.doppler" -d
warning: value assigned to `subcommand` is never read
   --> src/lnd_actions/lnd_cli.rs:414:17
    |
414 |         let mut subcommand = String::from("");
    |                 ^^^^^^^^^^
    |
    = help: maybe it is overwritten before being read?
    = note: `#[warn(unused_assignments)]` on by default

warning: `doppler` (lib) generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/doppler -f doppler_files/payjoin_setup.doppler -d`
[2024-05-14T23:18:52.178896000Z INFO] rest false
thread 'main' panicked at src/workflow.rs:60:10:
parse error: Error { variant: ParsingError { positives: [ln_node_action_type, btc_node_action_type], negatives: [] }, location: Pos(90), line_col: Pos((7, 8)), path: None, line: "sender SEND_ONCHAIN receiver AMT 1", continued_line: None }
stack backtrace:
   0: rust_begin_unwind
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::expect
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1030:23
   4: doppler::workflow::run_workflow_until_stop
             at ./src/workflow.rs:59:18
   5: doppler::main
             at ./src/main.rs:66:5
   6: core::ops::function::FnOnce::call_once
             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I'm assuming something like this would be possible? If so, what am I doing wrong?

Thanks

Expand documentation

Add a dictionary of the different grammar allowed and give better details into how to string different commands together. Could even link to doppler_file examples that show the gramma in action.

Add grammar for BUMP_FEE

We should make it possible to BUMP_FEE from a doppler file instead of requiring a bash script.

Add how many sats a lightning node should get

We give a random amount of sats to each node at the beginning of the process. Instead, we should allow the doppler file to set how many sats each individual node gets on start up.

Remove special IP networking

Instead of creating a custom gateway and the raw ip's being mapped, we should just be port forwarding. This currently causes issues trying to call into the doppler containers from mac since mac doesn't have a docker0 networking interface on it. This should be a pretty quick fix.

Doppler not running on MacOS (Intel)

This is what I get when I try to run the basic_setup script. I had docker desktop open when running this. Also tried it with docker-compose and docker compose. Cheers!

Doppler issue:
plebdev@plebdevs-MacBook-Pro doppler % cargo run --bin doppler -- -f "doppler_files/simple_example/basic_setup.doppler"
warning: value assigned to `subcommand` is never read
   --> src/lnd_actions/lnd_cli.rs:391:17
    |
391 |         let mut subcommand = String::from("");
    |                 ^^^^^^^^^^
    |
    = help: maybe it is overwritten before being read?
    = note: `#[warn(unused_assignments)]` on by default

warning: `doppler` (lib) generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 0.48s
     Running `target/debug/doppler -f doppler_files/simple_example/basic_setup.doppler`
version: 0.5
 Apr 12 22:00:34.115 INFO rest false
 Apr 12 22:00:36.696 INFO connect to doppler-lnd-lnd1 via rest using https://localhost:9097 and via grpc using 9098 with admin.macaroon found at localhost:/Users/plebdev/Desktop/doppler/data/lnd1/.lnd/data/chain/bitcoin/regtest/admin.macaroon
 Apr 12 22:00:36.696 INFO connect to doppler-lnd-lnd2 via rest using https://localhost:9100 and via grpc using 9101 with admin.macaroon found at localhost:/Users/plebdev/Desktop/doppler/data/lnd2/.lnd/data/chain/bitcoin/regtest/admin.macaroon
 Apr 12 22:00:36.696 INFO connect to doppler-lnd-lnd3 via rest using https://localhost:9103 and via grpc using 9104 with admin.macaroon found at localhost:/Users/plebdev/Desktop/doppler/data/lnd3/.lnd/data/chain/bitcoin/regtest/admin.macaroon
 Apr 12 22:00:36.698 INFO (start up): docker compose -f doppler-cluster.yaml up -d
 Apr 12 22:00:43.380 INFO (create wallet): docker compose -f doppler-cluster.yaml exec --user 1000:1000 doppler-bitcoind-miner-bd1 bitcoin-cli --datadir=/home/bitcoin/.bitcoin createwallet doppler-bitcoind-miner-bd1
 Apr 12 22:00:43.567 INFO (load wallet): docker compose -f doppler-cluster.yaml exec --user 1000:1000 doppler-bitcoind-miner-bd1 bitcoin-cli --datadir=/home/bitcoin/.bitcoin loadwallet doppler-bitcoind-miner-bd1
 Apr 12 22:00:43.756 ERRO failed to load wallet for bd1:  service "doppler-bitcoind-miner-bd1" is not running
 Apr 12 22:00:43.756 INFO (getnewaddress): docker compose -f doppler-cluster.yaml exec --user 1000:1000 doppler-bitcoind-miner-bd1 bitcoin-cli -rpcwallet=doppler-bitcoind-miner-bd1 --datadir=/home/bitcoin/.bitcoin getnewaddress
thread 'main' panicked at src/workflow.rs:31 :44:
failed to start the cluster: failed to create new address
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Apr 12 22:00:43.942 ERRO Failed to start cluster from generated compose file: failed to create new address

Fix security issues found by github actions

github actions found a few issues with dependencies in this project that either need to be updated or removed to fix the security issues. Until these are fix PLEASE DO NOT USE ON MAIN NET.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.