it only handles issue and basic transfers nothing else is handled
it terminates on first error
so normally the saved transaction buffer is not restored

measurement of libp2p traffic load by tcpdump is not as expected. Comparing with zeroMQ, it creates a lot more traffic.

Currently unit test coverage rate is 0

I'd like to take a look at ethereum's fast sync (ethereum/go-ethereum/pull/1889), learn the idea and try to copy this idea to bitmark node's initial synchronization.

There is no activation mechanism for transitioning to new consensus rules in the Bitmark blockchain. The changes could be activated without buffer time. The deployments of new versions to full nodes can't be executed until primary node runners successfully upgrade the version of bitmarkd.

There are 3 supported chains in bitmarkd configuration: bitmark, testing, and local. The local chain is designed for running regression tests.

The current local chain doesn't support generating blocks on demand for testing purposes. Almost every test case is coupled with recorderd, which makes the regression tests time-consuming. Most of the time is spent in checking if recorderd is working. The local chain should enable developers to have full control over the environment and test the bitmarkd functions without extra dependencies.

Besides, the local chain should be working with bitcoin/litecoin regression test mode. There are 3 modes of running payment modules: p2p, discovery, rest. Currently, the p2p mode can't be tested in the local chain because it automatically tries to connect to DNS seeds of bitcoin and litecoin for peer discovery on either mainnet or testnet.

While running Bitmark Linux version I come across this issue:

root@moby:~/bitmark-0.0.7-linux# ./Bitmark
./Bitmark: error while loading shared libraries: libgmodule-2.0.so.0:

It looks like it cannot find the shared objects, which are not in any of the dependencies

Starting from 0.12, it seems that there is a memory leak in bitmarkd.

Current logger level is determined when program started, as using bm-tester, often with situation that some problem happens, but log level is set to critical thus cannot see actual system log message.

If log level can be changed while program is running, then we can change log level to get more detailed information when some unexpected problem happens.

can't load package: package github.com/bitmark-inc/bitmarkd: no buildable Go source files in /home/echo/workspace/Go/src/github.com/bitmark-inc/bitmarkd

bitmarkd relies on go-libucl, and go-argon2.

libucl is installed by running the Makefile in github.com/bitmark-inc/go-libucl and moving the headers and shared libs in the appropriate spaces.

libargon2 is installed by updating the submodules of go-argon2 and running the Makefile.

Installing these 2 libraries required some debugging and could ideally be automated to lower the bar for new users.

In order to validate transactions in an efficient way, we decide to create a way to validate bitcoin payment faster. This part is to first to benchmark the resource usage for bitcoind.

A pending tx is supposed to be expired after 3 days. Several hundred outdated transactions are still circulating on livenet (v0.12.4).

Following the instruction at the bottom of https://bitmark.com/ on Mac OSX 10.11.2

~$ go install github.com/bitmark-inc/bitmarkd
# pkg-config --cflags libzmq
Package libzmq was not found in the pkg-config search path.
Perhaps you should add the directory containing `libzmq.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libzmq' found
pkg-config: exit status 1

Increase coverage rate of a consensus package

I'm interested in running bitmarkd without having to have a GUI. After running the commands in the README and the website installation guide, I am presented with the following message:

root@moby:~# go get github.com/bitmark-inc/bitmarkd
package github.com/bitmark-inc/bitmarkd: no buildable Go source files in /root/go/src/github.com/bitmark-inc/bitmarkd

Reproduced on both Linux and Mac.

I see references to a package main in github.com/bitmark-inc/bitmarkd/command/bitmarkd. Is this the package that should be installed?

Using the existing frameworks for testing P2P distributing system to make Network more stable.

• Investing distribute system testing frameworks
• Research on Jepsen
• Research on Muxy
• Demo on Jepsen
• Demo on Muxy
• Propose a framework for bitmark distributed system

Acceptance criteria:

• Sequence diagrams for the following operations
  • class operations: create, transfer, and update
  • token operations: create and transfer
• Block diagram (wallet, bitmark blockchain, resolver, and resource server)

References:

• https://github.com/decentralized-identity/github-did/blob/master/docs/did-method-spec/index.md#crud-operations
• https://github.com/blockstack/stacks-blockchain/blob/stacks-1.0/docs/blockstack-did-spec.md

--

Overall goal

1. High-dimensional thinking about all needs

   Role, NFT, endpoint
   Decide on NFT Controller
   Determine the USER controller
   Is the item owning the "person" or the person owning the item.

2. Convert all requirements into modules

   Convert into specific objects and modules
   Decide whether the verification module after CRUD is organized by node or an external company.

3. Convert the type to UTXO

   Even if the verification mechanism of an external company organization is adopted. The most basic UTXO transaction verification mechanism is also required
   The mining mechanism of UTXO transactions is the same as BTC?LTC? or new tokens are to be issued. If there are new tokens in the future, support should be retained.

4. Deploy the new UTXO to Bitmarkd's system
   • lib builds a new type of UTXO
   • lib reads UTXO in new state
   • lib Verifies the new state of UTXO
   • lib get rpc UTXO
   • RPC receiving type UTXO
   • RPC builds a new type of UTXO
   • RPC read UTXO in new state
   • RPC verifies the new state of UTXO
   • p2p synchronization implementation

If the btc peers.json contains unreachable IPv6 bitmarkd will use approximately 50% CPU.

Workaround:

• stop bitmarkd
• remove peers.json
• restartd bitmarkd

This shows up in local test mode when IPv6 address was lost after switch to WIFI (IPv4 only)

Build a local cluster to test the correctness of the consensus protocol.

We're only able to test this case: the block with more backing nodes wins.

Current logger package needs to be initialised before using. This increase some complexity when writing unit test.

Not only every test case need to initialize logger, but also when background task is running while main test problem is terminating and finalize logger, it might cause unexpected problem of logger not initialised.

If logger can be default send to stdout, then it provides two benefits:

• no need to setup logger for ever test
• when ever unit test error, with go test -v shows details message of process, facilitate easier debugging

Original Problem Description

In Bitmarkd configuration, if we want to listen on all address of IPV4 and IPV6, we may do this
listen = {
"0.0.0.0:2136",
"[::]:2136"
}
Then we get a bind error that said the port has already Bind.
We have 2 modules have the issue Peer and RPC

Propose base on """Linux"""

Use [::] to represent all interface
In configuration,
In RPC module
In Peer module

"BUT" ---> This is a os related and a library issue ... please see follow

Investigation on the Issue

according to this reference: https://lwn.net/Articles/688462/
In kernel Level,
OS do IPV4 IPV6 Mapping. In Linux it is a default, therefore, "[::]" means listen All
However, according to the article, BSD may not be the default. (OS dependent)

In Go Library Level, (Test on Ubuntu Linux Only )
In RPC, ipType can be specified "tcp" -- > listen all
l, err := tls.Listen(ipType[i], listen, tlsConfiguration)

If we want to specify this
listen = {
"0.0.0.0:2136",
"[::]:2136"
}
It is also ok, we should use
tcp4 -- > "0.0.0.0:2136",
tcp6 --> "[::]:2136"

However, In Peer ZMQ Library , If we can not bind both in NewBind. To listen to all, we shall use [::]:2136 (Notice: Maybe a deep investigation on ZMQ library may can )

// allocate IPv4 and IPv6 sockets
lstn.socket4, lstn.socket6, err = zmqutil.NewBind(log, zmq.REP, listenerZapDomain, privateKey, publicKey, c)

What should we do ??

Check on differences in in BSD, MAC and Linux and find the rules

• Will IPV4 map IPV6?
• What the parameters to reflect this on RPC and ZMQ setting

Bitmarkd depends on a series of other packages. Suggest to vendor these package and provide an easy way for installing, updating and maintaining them.
A good solution (also used in btcd) is Glide

Here is a sample glide.yaml file.

package: github.com/bitmark-inc/bitmarkd/command/bitmarkd
import:
- package: github.com/bitmark-inc/bitmarkd
  version: v2.10
  subpackages:
  - announce
  - asset
  - block
  - chain
  - configuration
  - fault
  - mode
  - payment
  - payment/bitcoin
  - peer
  - proof
  - reservoir
  - rpc
  - storage
  - util
  - version
  - zmqutil
- package: github.com/bitmark-inc/certgen
- package: github.com/bitmark-inc/exitwithstatus
- package: github.com/bitmark-inc/getoptions
- package: github.com/bitmark-inc/listener
- package: github.com/bitmark-inc/logger
- package: golang.org/x/crypto
  subpackages:
  - sha3

Let me know if you would like my help with this.

Full-Node documentation has complete 40% in Q2. It needs to combine into the whole bitmark-documentation. The goal is working on launching full-node documentation.

• Review the bitmark documentation to see how full-node documentation can merge into the documentation.
• Work on the way to coordinating document structure with QA
• Close this OKR to let the QA leading the document making process

Add additional request fucntion to proof/submission.go. This currently only supports returning a nonce. Add the txZero field to the request and add a hash field to the response.
The submission must verify that txZero is a valid foundation record before accepting the submission and only return the hasg if this record is valid.

This allows for two possibilities

1. recorderd can sign its own txZero and thus provide its own payment addresses

2. a mining pool program can be created to use the returned hashes to apportion rewards to individual miners

This is an extended issue from #105.

By removing the discovery dependency, the bitmarkd program doesn't rely on the discovery service hosted by Bitmark Inc.

bitmark-0.0.7-linux.zip contains artifacts from Mac's Finder utility. They usually contain useless information and sometimes also personal information on your desktop environment. Suggest to remove them

Currently, hash rate calculation only considers leading zeros from difficulty, but actually remaining digits still have impact on hash rate calculation.

Better way is to calculate all those digits to have more accurate number.

Take difficulty 4.8 as an example, leading zeros are 8 + log2(4.8), hash rate is pow(2, 8+log2(4.8))/120

Current discovery is using DNSLookup for TXT record and routing is using binary tree with X algorithm. There are more existing choices of discovery and routing. Explore the widely used algorithm come with a stable p2p library can be an improvement of P2P module.

• Research on P2P algorithms
• Research on gossip protocol on fault detection for routing
• research and testing on specific algorithms (decided later
• Propose a new algorithm

Non-mining nodes don't need to verify if fees for bitmark transactions are already paid or not. The status of a transaction can go to confirmed directly from pending. Disabling the payment module can remove additional dependencies on bitcoin and litecoin nodes and also reduce network bandwidth usage.

Remove all the publish code as this was only ever used to forward data to a database collector. It is not necessary to have this in every bitmarkd as this program could just use the normal p2p connection and receive identical data. (this ports was simply publishing the exact same binary data as the normal p2p so is only adding extra processing and network overhead)

Current bitmark node requires public IP to connect with other nodes. This limits the utilization rate of bitmark nodes. The bi-directional connection enables that two nodes require only one public IP to connect with each other. Further more, nat transversal can enhance the requirement of using public ip. The task is to research on existing golang libraries for bidirectional and nat transversal.

• Research on bidirectional tcp
• Research on Nat transversal issue (20%)
• Research on golang library
• Testing on the libraries in bidirectional tcp
• Testing on nat transversal feature
• Create a simple P2P module to illustrate the bidirectional connection works

Currently this package coverage rate is 23%

The new config that uses "*" for TPC46 dual stack does not work for RPC (2130) on 0.12.4-beta.3.
Caused by change from old listener to plain Go library listen and separation to "tcp4" and "tcp6" options.

Detect the "::" and "0.0.0.0" case ad use the dual "tcp" config instead.

Testing fix in 0.12.4-beta.4