As shared in the screenshot we get coinbase errors upon starting, i tried with a new database and still got the error. Might be linked to pruned issue.

setup a DB using the schema shared in the spec in the spec document.

Write a script to start btc nodes and ensure that they connect to a specific set of peers by whitelisting that set in the bitcoin.conf file.

we need to implement a way to dump forkscanner logs to specified file location. this would help us to debug and monitor the forkscanner.

this ticket is. to keep track of 4 things we need to add.

1. Lagging nodes implementaion (specs added in spec folder) here
2. Invalid block pubsub socket (Spec added in the spec sheet)
3. Get block from peer implementation. here
4. Pubsub socket endpoint to return single chaintip. here
5. Double list response from get forks endpoint. here
6. Pubsub channel break with a panic error bug. here
7. Documentation and a list and examples of endpoints here
8. Check up on matching parent and children relationships. here
9. Address Watcher, we need a pub sub, which would accept an address and notify when there is an inward transaction in that address. (details added in Specs)
10. keep a track of peers our btc nodes are connected to. and provide an endpoint which would give us the current peers for each of our nodes.
11. An api endpoint to push a block to forkscanner.

please find the details in the spec sheet

Please find more details in the spec sheet

we require a web socket (pubsub). which would send a notification whenever anything changes in the chaintips table.

I am getting this response from subscribe_forks websocket:

recv: {"jsonrpc":"2.0","result":10550142073689083033,"id":1}

recv: {"jsonrpc":"2.0","method":"forks","params":[[{"block":"000000000000000000023ad1f1c37363b224c11d039bd27ff9d01e734b239f2f","height":742542,"id":1,"node":1,"parent_chaintip":null,"status":"active"},{"block":"000000000000000000023ad1f1c37363b224c11d039bd27ff9d01e734b239f2f","height":742542,"id":2,"node":2,"parent_chaintip":null,"status":"active"}]]}

Issue 1
I am not able to understand what and why I am getting this:

recv: {"jsonrpc":"2.0","result":10550142073689083033,"id":1}

Can I please have just 1 response of chaintip?

Issue 2

{
	"jsonrpc": "2.0",
	"method": "forks",
	"params": [
		[{
			"block": "000000000000000000023ad1f1c37363b224c11d039bd27ff9d01e734b239f2f",
			"height": 742542,
			"id": 1,
			"node": 1,
			"parent_chaintip": null,
			"status": "active"
		}, {
			"block": "000000000000000000023ad1f1c37363b224c11d039bd27ff9d01e734b239f2f",
			"height": 742542,
			"id": 2,
			"node": 2,
			"parent_chaintip": null,
			"status": "active"
		}]
	]
}

Params array is actually coming in as array in an array, if there is no use-case of this, please return just one array or return just one block as the chaintip that forkoracle will use to send to the chain.

As mentioned in this issue. we need to maintain peer list and ensure mirror connects to the same peers as the regular node. Also all regular nodes connect to different set of peers. we need to maintain a peer table in the DB.

string "name"
integer "version"
datetime "created_at"
datetime "updated_at"
string "rpchost"
integer_array = ["node_id's connected to this peer"]

please find the details in spec sheet

right now we have an RPC endpoint 'get_block' which returns one block at the given height. what we want is another endpoint which would take a value (depth) and would return all the block s uptil that depth. so if depth is 10 that would mean the latest 10 blocks.

We need to implement an API endpoint, which would give us stale blocks upto the specified depth, it would mean running query to get blocks where there are more then 1 block at the same height.

https://github.com/twilight-project/forkscanner/blob/main/src/scanner.rs#L1026

At the above mentioned location, we need a check which sees if a block is marked as invalid. if it is, we don't have to add that block into the stale candidate table.

this means that we will not have to process blocks which are already marked as invalid and save us processing power

Bootstrapping is done by creating a service that will query the nodes and update the DB:

• Setup A DB using the Schema shared in the spec sheet.
• Setup some bitcoin nodes in different geographical locations.
• automate setting up of btc nodes
• setup a system so that nodes can get a list of peers from other nodes.
• Retrieve data from nodes and populate the DB.

An endpoint which will take a block hash and will share the parent/previous of that block.

we need more detailed documentation on how to set up and run forkscanner. along with a list of pusub and json rpc it supports. with example on how to use each of these.

We have 6 validators running forkscanner and all of them get this error

on each validator we have a tool called forkoracle which subscribes to fork scanner's web sockets, forkoracle is a Golang based tool and the source code can be seen at the link mentioned above

It seems tests were not updated with the latest push

We need to add header_only(boolean) field in block schema. This is necessary as it lets us know which node has the complete block and which node only has the block headers.

later on when we will query other nodes to update the current node which only has the headers. we will need this field to filter and not send requests for blocks we already have.

https://github.com/twilight-project/forkscanner/blob/add-chaintips-queries/src/schema.rs#L6

we will also need to mark this as true when we process valid header and header-only status from getchaintips RPC call
https://github.com/twilight-project/forkscanner/blob/add-chaintips-queries/src/scanner.rs#L114

Setup preferably at-least 3 bitcoin full nodes, ensuring that the nodes are geographically in different zones which would allow us to see chain splits due to network latency

This service validates folks and makes roll backs where needed.

• find any missing blocks
• Chaintip Level Validation
• Block level validation
• make Active on mirror and roll backs

forkscanner's get_block rpc endpoint returns a block along with transaction id,

the transaction ids shared are in bytes arrays, please convert them back to hex strings so it matches the same format as block ids and are easily readable.

the subscription channel breaks with and panic error. more details in the screen shot attached.

the address watching endpoint throws parsing error.

Invalid parameters. Expected list of addresses to watch.

this should not be generic for now it throws this error for every case

also i am using the below params while subscribing to pubsub

{ "jsonrpc": "2.0", "id": "watched_address_checks", "method": "watched_address_checks", "params": { "watch": [ "3Q92diEguqddwBc17Q2DpMXp1nPFx5nCtF" ], "watch_until": "2023-09-30T00:00:00" } }

this is a copy of the example shared and still get the list error. even though it is a list

Ideally forkscanner will be connected to pruned btc nodes, we should have a check which ignores a block if gets pruned status back from btc node for the first get.

right now forkscanner keeps on sending the same get block request again and again, and goes into an infinite loop and gets blocked by the btc node.

An api endpoint, which would take a block hash or height and will return the parent block hash and height

Ideally, we require Forkscanner's full nodes to be connected to unique peers. This requires knowledge of peers who are already connected to other full nodes at the time of initialising Forkscanner on a new machine.

In an offline brainstorm with @AhmadAshraf2 , we came up with 2 ideas:

1. bitnodes.io has a public API for a leaderboard. This could return a list of upto 100 live nodes. At the time of initialisation, one could pick a random set of peers. Alternative to this is running a local crawler, which maybe an overkill.

2. Forkscanner can send their list of connected peers in a msg to Tendermint. At the time of initialisation, one could query Tendermint for a list of peers already connected to other full nodes and then randomly pick peers for their full nodes. This exposes the IPs for connected peers on-chain, which sacrifices their privacy.

We intend to avoid any p2p layer among Forkscanner full nodes.

if we can have 3 configurations for address watcher mentioned below

1. enable or disable address watcher
2. monitor only inflow of funds in the address (can run on pruned node)
3. monitor only outflow of funds from the address (needs atleast one full btc node)
4. monitor both inflow and out flow (needs atleast one full btc node)

details added in the spec sheet

This issue is for tracking the tasks remaining on the forkscanner. please follow the list provided below

Bugs

below is the list of issues found with the current forkscanner.

1. Forkscanner has 3 bitcoin nodes in its DB. When you run the scanner, it only connects to 2 of these nodes. there are no errors regarding error connection failures. Although normally forkscanner does show errors in case of connection failures. This needs some investigating.

2. During testing Forkscanner was given 3 bitcoin nodes to connect to. Information from one of the nodes got stuck and was not being updated in the forkscanner DB. Again no connection errors were found. Considering no errors and the fact that the bitcoin node itself was running fine, An assumotion is that the request was never made to get the data from the node. it could be that one of the thread gets stuck.

3. Forkscanner has an pubsub endoint. which publishes new chaintips to all its subscribers. it was noticed that the Forkscanner picked up new chaintip and updated the DB a few minutes before it publishes the new chaintip to the subscriber. this should not be the case. it should be instant.

4. While running forkscanner we faced some lock poisoning errors. screen shots are attached with this issue

Missing Features and Endpoints

Below is the list of features missing and need to be added in the Forkscanner.

1. Forkscanner will be used as a service in a system, where other services might need to submit a block back to the bitcoin node. to do this we need a RPC endpoint, using which another service can submit a block to forkscanner and then forkscanner can send it to bitcoin nodes using the submit block rpc endpoint.

2. As of now Forkscanner expects the bitcoin node and its mirror to be running on the same machine. This works for testing purposes but in production both of these should run on different machines. so we need to add this functionality in.

3. Forkscanner checks if a node is lagging behind or not. we would also need to check the mirrors for liveness. a detailed spec will be share for this

4. Forkscanner needs to have a pubsub connection which would provide a single chaintip to other services, and if there is a valid fork then show the active chaintip and the valid fork tip. more details on this will be provided.

5. Forkscanner needs to keep a track of each bitcoin node's peers and should have an endpoint to provide a list of peers for each node.

6. Documentation should have an example for every rpc endpoint and for every pubsub, also it should have all the steps required to run the system.

In Progress

This is a list which contains the features currently being worked on

1. Address Watcher
2. stale candudate endpoint

Attachments

Spec for lagging mirror nodes.docx

We will run nodes and mirror of these nodes. to do this we need to ensure that a mirror nodes connects to the same peers as the regular node. we also need to ensure that our regular nodes connect to different set of peers. overlapping set or peers will work but exact same set will create issues.

For this we need the below mentioned issues resolved.

#3
#4

the code is quite clean and understandable, but can we add some comments/doc strings to explain what each struct and method are for.

This would allow for better documentation and will help other developers in the future.

when forkscanner uses bitcoin-cli getchaintips, forkscanner considers chaintip with status "headers-only" as a valid block. This is not right, and when the address watcher is activated it tries to look for a block which does not exist. and throws an exception

Since each node will have a mirror node. we need to save the connection information about the mirror node. so we need to add the below information.

string "mirror_rpchost"
integer "mirror_rpcport"

https://github.com/twilight-project/forkscanner/blob/add-chaintips-queries/src/schema.rs#L36

We try to find the missing blocks among our peers, the blocks which are marked as valid headers/ headers only in our DB

please find the details in spec sheet

We need a few API/RPC calls explained below

• An API/RPC endpoint to add or remove nodes.
• An API/RPC endpoint to retrieve latest active chaintip.
• An API/RPC endpoint to retrieve latest chaintips with each status from all connected btc nodes.
• An API/RPC endpoint to check if a transaction is in active chain.
• An API end point to get a list of blocks from DB, or get by hash or height.

we create a gbfp_blocks list and populate it as well. the purpose of this list was to retrieve these blocks from peers. that part is missing and in fact nothing is done with this list right now.

Trying out the validation_checks pubsub, it's not returns a random json rather than what was expected.

above the results i get for this {"jsonrpc": "2.0", "id": 1, "method": "validation_checks", "params": [20]}

what we require is the Chaintip height, stale candidate height, timestamp for stale candidate, chaintip, stalecandidate tip, branch length of stale branch and stale branch root (block where the branch started)

right now address watcher keeps track of both inflow and outflow of funds from the addresses under watch.

to check the outflow we have to download a lot of transactions, which slows the process down and takes a lot of storage space. it would be beneficial if we keep outflow of funds as a feature which can be turned off or on.

This will allow user who do not have a use case to watch outflow of funds save storage and processing.

only error are showing up on console. I am unable to see any other console logs e.g. info

Responsibility of this Fork Scanner will be to detect forks in BTC chain. this will provide means to check if a specific transaction is in the active chain-tip or in a valid/invalid forks. Fork Scanner will also check double spent transactions (transactions which are in more then one chain-tip) and ensure that the transaction is secure.

Purpose behind writing this is to monitor the chain and making sure that the transaction is concrete and here is no possibility of it reverting.

This is a progress tracking issue and below are the subtasks:

• Initial startup and boot strapping.
• Validating forks and Rollbacks.
• Detecting Double spent transactions
• Api/RPC end point

Need a websocket pubsub to pickup latest active chaintip. The idea is that forkoracle (that I am currently writing in golang) should be able to subscribe to the notifications, that will allow forkoracle to not work on timers.

Currently, I am just using get_forks to pick the active chaintip but more implementation is required to notify the subscriber that a new active chaintip entry was made in the database.

For Bootstrapping we will create a service that will query the nodes and update the DB. For this we use the rpc::getchaintips Json-Rpc call. We query all nodes to share their latest chain tips and we save those in the DB

complete detail are shared in the spec sheet

Right now the pubsubend point returns the amount deposited in the address under watch. we would like to know the address from where the funds came from as well

https://github.com/twilight-project/forkscanner/blob/add-chaintips-queries/src/scanner.rs#L651

Currently to find the block where the branch split happened. we pick the latest block from each split and start traversing backwards (to the block's parents). and keep going backwards until the block hash is same for both splits. this approach might be faulty on some specific corner cases

since we are maintaining a parent-child relation ship and can find the descendants of a specific block. I would recommend that we traverse one chain and keep moving backwards until we find a block which is an ancestor of the tip block of both the chain-tips.

this would be a more robust approach.

https://github.com/twilight-project/forkscanner/blob/main/src/scanner.rs#L927

this check should be other way around to ensure that the relation between parent and child is correct
same goes for the method match_parent