trustwallet / trust-ray Goto Github PK

:cloud: API for the Trust Wallet. Project no longer supported and current version used as source of transactions and address tokens in Blockatlas https://github.com/trustwallet/blockatlas/blob/master/config.yml#L64

Home Page: https://trustwallet.com/

License: GNU General Public License v3.0

TypeScript 98.30% Python 1.07% Shell 0.48% Dockerfile 0.14%

ethereum backend blockchain typescript cryptocurrency dapps erc20 erc20-tokens erc721

trust-ray's Introduction

☁️ Trust Ray ☁️

API for the Trust Ethereum Wallet.

Features

Parsing entire blockchain
Retrieving transactions with operations field for ERC20 contract actions
Retrieving ERC20 token balances
Push notification service (not yet implemented)

API wiki

Deploy on Heroku

Locally (without docker)

Install required modules: $ npm install
Compile TypeScript: $ npm run build
Start the app: $ node dist/server.js
Run tests: $ npm run build && npm test

Docker containers

Install docker and docker-compose.

Set in ~/.bashrc export COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml

Set in .env MONGODB_URI=mongodb://mongodb:27017/trust-wallet

Dev tool:

Run build for npm install and build ./trust build
Start app in docker ./trust run
Stop docker containers ./trust stop
App logs ./trust logs

Authors

Contributing

We intend for this project to be an educational resource: we are excited to share our wins, mistakes, and methodology of iOS development as we work in the open. Our primary focus is to continue improving the app for our users in line with our roadmap.

The best way to submit feedback and report bugs is to open a GitHub issue. Please be sure to include your operating system, device, version number, and steps to reproduce reported bugs. Keep in mind that all participants will be expected to follow our code of conduct.

Code of Conduct

We aim to share our knowledge and findings as we work daily to improve our product, for our community, in a safe and open space. We work as we live, as kind and considerate human beings who learn and grow from giving and receiving positive, constructive feedback. We reserve the right to delete or ban any behavior violating this base foundation of respect.

trust-ray's People

Contributors

Stargazers

Watchers

Forkers

kolya182 rip32700 buildonviction makitra vischub alexintosh 872409 borisbutakov hewigovens jinchung oreosfly bitcoinone1 ictoken flypaper0 millerapps duyquoc johnnynanjiang pandeykumar cryptosports opensauce-io albererre wl-tsui pengpengliu c-rtx1210 spirans diop blueplanet ellaism water-bamboo davidqhr rahmanazhar valsoray07 imchain168 kaliwdsn elagerway gettyio zerxon macho321 debitoken beshkenadze 4xxi hboon dogecrypto papers-ch zlben dinneo blockchain-zju b26 blockchainresource huangyaya27 valeryva sjgga candangios catwallet faithgithub bitfyio multitoken kknight99 beehive-project tdhungit youdianwuliao enzodudu eddywm ruben135 cobain irmabf algorithmic-trading nonren jlebron3 fadysi92 quangdo3112 mason0510 akropolisio shundev ricburton xilibi2003 kubulus1997 liuhuanhui loomnetwork wangshayne riptl cryptobuks baophucct leoneparise knolan39666420 dappstore123 yaoyuhang devilbits dily3825002 djaramillo021 becorp naiemk prazd banklife xero-official naboxcom albertleecn bonomali freemocker blockchainidentityvault

trust-ray's Issues

Fix RPC Error

Mongoose bulk operations for one block

Consider using bulk operations for all transactions for one block. This should speed up the upserts.

Avoid adding null values for to or from fields

Doesn't hurt, but would be nice to have

Just flat map on addresses field I guess

Remove list on starting up a build on heroku

Extra time and effort to startup production build

Split parsing and RESTPI API into separate command

Once we stabilize functionality it would be good to start just REST API, and parsing separately.

something like:

npm run start - this would just run REST API
npm run start --parser - REST API + Parser

Wrong value format

https://trust-wallet.herokuapp.com/transactions/0x455cae2178da848f320566af74798acb108d8d014ff693eb19c5f4f7c59aa636

operation: {
_id: "5a055b4c7946851ff3f85217",
transactionId: "0x455cae2178da848f320566af74798acb108d8d014ff693eb19c5f4f7c59aa636",
type: "token_transfer",
from: "0x3f5017f9884a2b47b72592bebe4256d5879d42cd",
to: "0x1117e4e38f9122ca4c00f2c4214e4f400db8e221",
value: "3.3565e+22",
contract: {
_id: "5a055b4c7946851ff3f851fd",
address: "0x8f8221afbb33998d8584a2b05749ba73c37a938a",
name: "Request Token",
totalSupply: "999999999244592134526985951",
decimals: 18,
symbol: "REQ"
}
},

value: "3.3565e+22", this has to be fixed

Integrate push service

Eventually ingerate the push service into the backend.

Add caching to travis: npm install command

Lowercase addresses in DB

Let's keep all the addresses and contracts in database in lowercase, makes it easier to find needed values

Parsing is behind

Something happened and parser was behind

19.00 UTC - 20.30 UTC
blocksToSync: 400

Proper input sanitzation

Populate list of operations

Pull pending transactions from the RPC server

to make UX amazing for the user we would want to show pending transactions as well, for that we need to call

https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_getblockbynumber, use pending as parameter

@rip32700 I think this should be called when blockchain in sync and just checking for new blocks and etc.

Doc the setup requirements

Please outline the setup requirements. I deployed to Heroku and associated an mLab mongoDB instance. Still doesn't work. What is needed to get a deployment to a working state like this -
https://trust-wallet-kovan.herokuapp.com/

Save blockNumber as a type "Number"

Sorting or indexing by type Number would be slightly faster than a string normally

Return error message

Some transaction could fail like this one: https://etherscan.io/tx/0xafe482affe9836ddff09553561ad71debec46e5c7e79bd4d4d3abf7f6a239e9e

we should have status for representing that in Transaction model

"blockHash" key on transactionSchema

What is the purpose of saving it ?. Do we really need it ?.

saveTransactions - wait until it's completed

@rip32700 can we make this function to return promise or do async?

It would be to have wait for saving transactions and then go to the next block, otherwise we get too many requests piled up and running into timeout issues.

Could not parse input for contract: The returned value is not a convertible string. Cannot read property '_id' of undefined

Parsing yearly blocks got this error:
[Node] info: blockInDB: 231100, blockInChain: 4452694
[Node] info: processing block: 231100
[Node] error: Could not parse input for contract 0xc1D9Be24114726b5D11ed7eF85c23Ce22A083592 with error: Error: ERROR: The returned value is not a convertible string:.
[Node] error: Could not find contract by id for 0xc1D9Be24114726b5D11ed7eF85c23Ce22A083592 with error: TypeError: Cannot read property '_id' of undefined
[Node] error: Could not parse input for contract 0x2CA7b7140BDB846CA272D7392B3DD6E2dC1a6843 with error: Error: ERROR: The returned value is not a convertible string:.
[Node] error: Could not find contract by id for 0x2CA7b7140BDB846CA272D7392B3DD6E2dC1a6843 with error: TypeError: Cannot read property '_id' of undefined
[Node] error: Could not parse input for contract 0x2CA7b7140BDB846CA272D7392B3DD6E2dC1a6843 with error: Error: ERROR: The returned value is not a convertible string:.
[Node] error: Could not find contract by id for 0x2CA7b7140BDB846CA272D7392B3DD6E2dC1a6843 with error: TypeError: Cannot read property '_id' of undefined
[Node] info: processing block: 231920
[Node] info: processing block: 232740
[Node] info: processing block: 233560
[Node] info: processing block: 234380
[Node] info: processing block: 235200
[Node] error: Could not parse input for contract 0x14FEA711DCa8fCEa6bE47E14Ea1406384f18c7cc with error: Error: ERROR: The returned value is not a convertible string:.
[Node] error: Could not find contract by id for 0x14FEA711DCa8fCEa6bE47E14Ea1406384f18c7cc with error: TypeError: Cannot read property '_id' of undefined
[Node] info: processing block: 236020
[Node] error: Could not parse input for contract 0x17c7D136bDfC4371f989076bB3842be2e73C3Ec1 with error: Error: ERROR: The returned value is not a convertible string:.
[Node] error: Could not find contract by id for 0x17c7D136bDfC4371f989076bB3842be2e73C3Ec1 with error: TypeError: Cannot read property '_id' of undefined

Remove /api

from: /api/transactions/0xc343bdaee7cdfee3b87aa90f56835e44f39c4005262223dbcc01ab8632883ecb

to: /transactions/0xc343bdaee7cdfee3b87aa90f56835e44f39c4005262223dbcc01ab8632883ecb

in most cases this API would be deployed to subdomain such as api.example.com, api sounds little redundant

Implement endpoint /tokens logic

Handle multisend functionality

An example: https://etherscan.io/tx/0x0696523adeb7fdcd1ef21c5fab30a00f8fbd6935a7fd1031f4635a949b3e5de2

Check TS completion before deploying

I had this error and fixed it with 1345ca4 pretty much.
TS was failing to complite even though tests were passing.

Can we make compiling prod version as a test before deploying? to make sure it won't crash. It's tricky because mac os case insensitive, but ubuntu is, that's why it caused this error.

Invalid JSON RPC response

2017-11-03T05:34:20.684575+00:00 heroku[router]: at=info method=GET path="/" host=trust-wallet-kovan.herokuapp.com request_id=ae2413cc-6a71-4b45-a3c9-35f008192f55 fwd="67.188.0.87" dyno=web.1 connect=1ms service=32ms status=404 bytes=383 protocol=https
2017-11-03T05:34:20.686338+00:00 app[web.1]: �[0mGET / �[33m404 �[0m16.177 ms - 139�[0m
2017-11-03T05:34:30.650529+00:00 app[web.1]: info: Retrieving new transactions between blocks 4578142 and 4580574
2017-11-03T05:34:39.342638+00:00 app[web.1]: error: Could not get block 4579282 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.490494+00:00 app[web.1]: error: Could not get block 4579311 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.490685+00:00 app[web.1]: error: Could not get block 4578995 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.490818+00:00 app[web.1]: error: Could not get block 4579303 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.490947+00:00 app[web.1]: error: Could not get block 4579279 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491074+00:00 app[web.1]: error: Could not get block 4579276 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491198+00:00 app[web.1]: error: Could not get block 4579270 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491322+00:00 app[web.1]: error: Could not get block 4578818 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491445+00:00 app[web.1]: error: Could not get block 4579272 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491568+00:00 app[web.1]: error: Could not get block 4578958 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491690+00:00 app[web.1]: error: Could not get block 4579250 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491815+00:00 app[web.1]: error: Could not get block 4579242 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.491933+00:00 app[web.1]: error: Could not get block 4578615 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492063+00:00 app[web.1]: error: Could not get block 4579248 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492185+00:00 app[web.1]: error: Could not get block 4579310 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492342+00:00 app[web.1]: error: Could not get block 4579343 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492466+00:00 app[web.1]: error: Could not get block 4579335 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492588+00:00 app[web.1]: error: Could not get block 4579347 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492712+00:00 app[web.1]: error: Could not get block 4579296 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492838+00:00 app[web.1]: error: Could not get block 4579312 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:39.492962+00:00 app[web.1]: error: Could not get block 4579290 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.432075+00:00 app[web.1]: error: Could not get block 4578920 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.432442+00:00 app[web.1]: error: Could not get block 4578842 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.432648+00:00 app[web.1]: error: Could not get block 4578915 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.432877+00:00 app[web.1]: error: Could not get block 4579164 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.433097+00:00 app[web.1]: error: Could not get block 4579098 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:40.433858+00:00 app[web.1]: error: Could not get block 4579524 from blockchain with error: Error: Invalid JSON RPC response: ""
2017-11-03T05:34:45.255339+00:00 app[web.1]: info: Retrieving new transactions between blocks 4580574 and 4580575
2017-11-03T05:35:00.264785+00:00 app[web.1]: info: Retrieving new transactions between blocks 4580575 and 4580577
2017-11-03T05:35:15.312939+00:00 app[web.1]: info: Retrieving new transactions between blocks 4580577 and 4580579
2017-11-03T05:35:30.413386+00:00 app[web.1]: info: Retrieving new transactions between blocks 4580579 and 4580581
2017-11-03T05:35:45.382889+00:00 app[web.1]: info: Retrieving new transactions between blocks 4580581 and 4580583

I'm still getting this errors on kovan server

Error: ERROR: The returned value is not a convertible string:.

2017-11-10T08:02:12.028812+00:00 app[web.1]: error: Could not parse input for contract 0x84119cb33e8f590d75c2d6ea4e6b0741a7494eda with error: Error: ERROR: The returned value is not a convertible string:.
2017-11-10T08:02:12.032914+00:00 app[web.1]: error: Could not find contract by id for 0x84119cb33E8F590D75c2D6Ea4e6B0741a7494EDA with error: TypeError: Cannot read property '_id' of undefined

If you run blocks in parallel it's getting this errors, if you run just one block it looks fine

Could not parse input for contract: with error: Error: Couldn't decode uint256 from ABI: 0x.

error: Could not parse input for contract 0x5f100EBe48ae9c5396DD059DD0d8f4447f12a9E0 with error: Error: Couldn't decode uint256 from ABI: 0x.
[Node] error: Could not parse input for contract 0xaB3652FD492FbB0d6b63acB742f3eD12AfBAEf52 with error: Error: Couldn't decode uint256 from ABI: 0x.
[Node] error: Could not find contract by id for 0x5f100EBe48ae9c5396DD059DD0d8f4447f12a9E0 with error: TypeError: Cannot read property '_id' of undefined
[Node] error: Could not find contract by id for 0xaB3652FD492FbB0d6b63acB742f3eD12AfBAEf52 with error: TypeError: Cannot read property '_id' of undefined
[Node] error: Could not parse input for contract 0x5f100EBe48ae9c5396DD059DD0d8f4447f12a9E0 with error: Error: Couldn't decode uint256 from ABI: 0x.
[Node] error: Could not find contract by id for 0x5f100EBe48ae9c5396DD059DD0d8f4447f12a9E0 with error: TypeError: Cannot read property '_id' of undefined

Kovan network server has missing indexes

I only see _id as an index

https://trust-wallet-kovan.herokuapp.com/api/transactions?address=0x00360d2b7D240Ec0643B6D819ba81A09e40E5bCd crashing the app

Could not get transaction receipt for tx hash

Example: https://etherscan.io/tx/0x04bce300aabff6351d60dc5c1491ccdd5cc8143090233ad1340601d70b39d4eb

Setup Deploy to heroku button

You select chain ID, RPC Server mongodb and etc... and you will have server running

Implement pagination for /transactions

error: failed to parse: TypeError: Cannot read property 'transactions' of null. restart again ...

I believe it's related to getting empty block which doesn't have transactions array, we just need to add extra check, @rip32700 I think you had this check in the past but I removed it

Adjust README

Missing operation for transfer token

URL https://api.trustwalletapp.com/transactions/0x53786a476f8b54d4ff353ce0a413d2ed212526f6f1faf24fd43521eb045f7487

response:

{  
   "_id":"0x53786a476f8b54d4ff353ce0a413d2ed212526f6f1faf24fd43521eb045f7487",
   "blockNumber":4547616,
   "timeStamp":"1510614114",
   "nonce":5,
   "from":"0xbb210d2dcf73fa534d645761497bda3f60bf0c6f",
   "to":"0x28c8d01ff633ea9cd8fc6a451d7457889e698de6",
   "value":"0",
   "gas":"124000",
   "gasPrice":"1000000000",
   "input":"0xa9059cbb0000000000000000000000000c6f41113abb3fa62cd5c361cbfe77322d3acd820000000000000000000000000000000000000000000000000000000000000031",
   "gasUsed":"2590171",
   "operations":[  

   ],
   "addresses":[  
      "0xbb210d2dcf73fa534d645761497bda3f60bf0c6f",
      "0x28c8d01ff633ea9cd8fc6a451d7457889e698de6"
   ]
}

Even though this transaction has transfer token: https://etherscan.io/tx/0x53786a476f8b54d4ff353ce0a413d2ed212526f6f1faf24fd43521eb045f7487

UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Decoded int exceeds width: 160 vs 256

https://github.com/TrustWallet/trust-wallet-backend/blob/master/src/common/blockchain.utils.ts#L193

In here we should have a check if this is ERC20 token then store token transfer :) otherwise API decoder freak out if it's something else

Order transactions by timestamp DESC

Build errors on Ubuntu 16.04

I'm on Ubuntu 16.04. How are you building this and is there a problem on my end?

I run:
sudo apt-get update
sudo apt-get install nodejs

Then:
sudo npm install

Then I receive these errors ...
npm ERR! Linux 4.11.0-1013-azure
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
...
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the scrypt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild

Remove PORT env value

can we remove PORT value from env? in most cases it will run on 80, dev in 8080. just to simplify for heroku deployment and us as well

Mongoose remove deprecation warnings

2017-11-03T05:34:18.413031+00:00 app[web.1]: (node:53) DeprecationWarning: `open()` is deprecated in mongoose >= 4.11.0, use `openUri()` instead, or set the `useMongoClient` option if using `connect()` or `createConnection()`. See http://mongoosejs.com/docs/connections.html#use-mongo-client
2017-11-03T05:34:18.413686+00:00 app[web.1]: (node:53) DeprecationWarning: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html

Setup Monitoring

Once all the services up and running, we need to setup monitors in the case any service goes down.

Create unit tests

Optionally setup code coverage.

Come up with better name for this repo!

backend sounds too boring!

etherParse
etherScan
eScan
etherChain
etherWatch
...

anything that make sense

Replace _id with hash

Since we are not going to use _id we can make hash as main ID

Store timestamp in Timestamp type

https://docs.mongodb.com/manual/reference/operator/query/type/

Should be faster to pull data from DB

Return transactions from and to

https://github.com/TrustWallet/trust-wallet-backend/blob/master/src/controllers/transaction.controller.ts#L14

we should both include from or to, which will include received and sent transactions

Consider using parity ethereum api

https://github.com/paritytech/js-api/blob/master/README.md.

Might be easer, light-weigh, more support for promises.

Startblock and endblock property on transactions API

For pagination we need to use startBlock as default 0, and endBlock = 9999999999

When user come back to the app, I fetch transactions from the last transaction block.

@kolya182 would you like to tackle this?

Add confirmations field to transaction model

confirmations: Int = currentBlockNumber - transaction.blockNumber

Fix Ubuntu setup for project

Following error occurs for $ npm install

npm ERR! Linux 4.11.0-1013-azure
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
...
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the scrypt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild