encrypt-s / kauri-api Goto Github PK

• configure .goreleaser.yml for release builds (from local machine)
• configure .travis.yml for release builds (via Travis CI)
• run a build matrix with multiple Go versions and/or multiple OS's
• generate RPM, Deb and Snap packages
• test locally by running goreleaser from project dir
• test in CI by letting Travis do its magic (via test PR)
• automatically sign the generated artifacts
• download and extract created archives, make sure they are legit

notes on GoReleaser:

• builds Go binaries for several platforms
• creates a GitHub release
• pushes a Homebrew formula to a tap repository
• wrapped in Travis
• Profit.

unused code carried over from kowhai project should be removed

docs for this repo should be viewable on godoc like this https://godoc.org/github.com/micro/go-config

• prototype interface pattern for multi-daemon interface
• port existing functionality to use multi-daemon interface
• add new currency config
• verify both currencies can be supported on endpoint
• write tests for multi-daemon interface

travis.yml should have go test -v -race ./... included in script block

The race detector is integrated with the go tool chain. When the -race command-line flag is set, the compiler instruments all memory accesses with code that records when and how the memory was accessed, while the runtime library watches for unsynchronized accesses to shared variables. When such "racy" behavior is detected, a warning is printed.

• running go run app/main.go should configure daemon
• use alternative data-dir in server config (preserve default data dir)
• make sure release binary can be run seamlessly
• create pattern for multi-daemon support

the following errors should be fixed:

app/api/api.go:62:1: exported function RouteBuilder should have comment or be unexported
app/api/api.go:68:1: exported function OpenRouteHandler should have comment or be unexported
app/conf/appconfig.go:22:6: exported type Coins should have comment or be unexported
app/daemon/daemonapi/vars.go:5:1: comment on exported var NavConfig should be of the form "NavConfig ..."
app/daemon/daemonrpc/rpc.go:14:6: exported type RpcRequestData should have comment or be unexported
app/daemon/daemonrpc/rpc.go:14:6: type RpcRequestData should be RPCRequestData
app/daemon/daemonrpc/rpc.go:19:6: exported type RpcResp should have comment or be unexported
app/daemon/daemonrpc/rpc.go:19:6: type RpcResp should be RPCResp
app/daemon/daemonrpc/rpc.go:60:1: exported function RPCFailed should have comment or be unexported
app/daemon/daemonrpc/rpc.go:69:2: var respJson should be respJSON
app/middleware/adapter.go:15:1: comment on exported function Adapt should be of the form "Adapt ..."
app/middleware/notify.go:8:1: exported function Notify should have comment or be unexported```

• add more detail to release notes
• provide setup info for Kauri-API
• enhance release notes

use getRawTransactionsHandler as initial functionality to model the pattern:

• in handler func, call deamonManager.GetTransctions(incomingTxs)
• manager should implement DaemonInterface = [currencyA, currencyB, etc..]
• range through active daemons and return transaction data
• navdaemon.go will check for 'nav' in returned data and get raw tx
• btcdaemon.go will check for 'btc' data, etc...

• write swagger spec for getrawtransactions POST
• validate spec
• publish on Swagger Hub
• add to readme
• add response / models to Swagger API

Details to be discussed on optimal deployment strategy

Ideas, Possible Approaches:

• https://kubernetes.io/docs/concepts/workloads/pods/pod/
• DigitalOcean Droplet

Configurations:

• load balancer
• health check

• Define clearly what testnet/devnet is and what it's usage is
• Decide if we need more nets

• convert an internal errors to an RPC error with the appropriate code set
• return meaningful response

new response should omit results obj and have the following structure:

{"data":[{"currency":"NAV","addresses":[{"address":"NW7uXr4ZAeJKigMGnKbSLfCBQY59cH1T8G","transactions":[{"txid

• Investigate benchmarks to include for API
• Decide how to implement each benchmark
• Implement benchmark on project
• Implement CI/CD

struct names should make more sense and follow our spec:
https://app.swaggerhub.com/apis/Encrypt-S/kauri-api/0.0.1#/developers/getRawTransactions

successful transaction response has the following structure:

• Determine documentation style (Go & Markdown)
• Update developer workflow markdown
• Strategy for internal documentation
• Swagger API and web portal (including CI/CD)

• Supports selecting a specific server to query for blockchain data
• Setting to add servers (by ip or url) including local
• Supports if this is a trusted server / local server

From integration tests, measurements over time can be taken to create valuable metrics to fine-tune and optimise the API (Latency over time, latency on particular requests, etc..)

Further discussion needed into what metrics will be the most valuable for Kauri API.

1. Test the interaction between the Kauri Wallet UI app and the Kauri API.
2. Ensure all exposed endpoints are covered.
3. Instead of using mock data, test against a real Docker daemon that is destroyed after testing.
4. Test environment should be easy to scale and support
5. Performance testing, load generation, concurrent requests, response latency

Initial Approach (to be detailed out as separate issues):

• Docker setup (boot up ephermal docker images for tests - Docker Compose)
• Auth setup
• Error handling
• Leverage Dockertest lib to spin up containers / daemons for tests
• Configure scripts for pipeline automation - CI
• Configure performance testing - load generation
• Configure test reporting

The Go app should leverage the built-in profiler that supports CPU, memory, goroutine and block (contention) profiling.

easily enabled:

import (
    "net/http"
    _ "net/http/pprof"
)

advantages:

• low overhead
• can be used in a production environment
• can be used with live traffic without any noticeable performance penalties

dev tasks (issues):

• report the number of incoming requests
• report how long every request takes.
• report request details - (URL, IP address and user-agent)
• send all stats to metric aggregator (e.g. StatsD) In addition, the service needs to
• measure how many requests per second the API can handle (Apache Benchmark tool)
• profile the CPU to determine which functions spend most time
• run heap profiler for in-use memory and number of allocated objects
• block profiling to show function calls that lead to blocking on sync primitives like mutexes and channels

• Define error style (Internal API & Client error responses)
• Define error capture plan (logging, alerts)
• Utilise something like Rollbar to capture errors server side
• Report external services only for testnet/dev

Strategy

We will log in development / testnet only, not in production

Logging can be expensive in Go without a good strategy. Resist the temptation of logging almost everything. Gather as much information as possible, and do it properly. Avoid the white noise of logs containing useless elements. Ensure that our logs report meaningful data that improves the overall health and longevity of the Kauri API app.

Tasks

• decide on log data shipper/catcher (Loggly, Logstash, Logmatic, etc...)
• configure log data shipper/catcher
• configure logs to report in dev / test only
• structure logs in JSON format
• log event types :: bind log output to specific line of code
• log output type :: JSON_PRETTY | JSON_ONELINE | DEBUG
• stream log data through TCP (SSL) using Logrus or something similar
• request key :: track requests across multiple services / daemons

Setup CI for optimal development:

• configure Travis for build / PR pipeline
• add build status badge to readme
• configure Coveralls for test coverage
• add coverage status badge to readme

• Drone for the win

https://app.swaggerhub.com/apis/Encrypt-S/kauri-api/0.0.1

• Utilise 12-factor app methodology (https://12factor.net/)
• Package app with GoReleaser (cutting releases locally w/ config)
• Package app for multiple runtimes (osx, linux, windows, arm, etc..)