Giter VIP home page Giter VIP logo

indy's Introduction

Indy

An Ethereum Super Node Indexing Server

js-standard-style

Indy is a an Ethereum blockchain indexer and REST API server, built with NodeJS, Express and Parity. The current DB implementation uses CouchDB.

The indexer is running on the same machine as the Parity, creating a database of transactional data. The data can then be queried using a REST API.

REST API

Pending transactions

GET /mempool/txs/[address]

Parameters:

  • address – optional account address.

Returns an array of unconfirmed (pending) transactions in the format below. If address is specified, only return transactions that have a from or to address that match address.

{
    status: 1,
    message: "OK",
    count: 316,
    result: [
        {
            blockNumber: 5120525,
            timeStamp: 1519072377,
            blockHash: "0xdc4df7a3dfaaa5cb115bccee2d15ddd241c057cc8ffbb8f47bb3f761f2b7ebe8",
            confirmations: 0,
            hash: "0xebcbdef39ff48a05f8e114fc5b5342c71f31ac7d35c8532edbdf632c6c3be3dc",
            nonce: 6771854,
            transactionIndex: 2,
            from: "0xEA674fdDe714fd979de3EdF0F56AA9716B898ec8",
            to: "0xDcb04eaD3c8439B7c740bC30b84CdF11Ce236AEa",
            value: "50046013704764736",
            gas: 50000,
            gasPrice: "1000000000",
            input: "0x",
            isError: 1,
            gasUsed: 21000,
            cumulativeGasUsed: 63000,
            contractAddress: null
        }
        ...
        ...
    ]
}      

Account transactions

GET /account/{address}/[startBlock]/[endBlock]/[limit]
GET /from/{address}/[startBlock]/[endBlock]/[limit]
GET /to/{address}/[startBlock]/[endBlock]/[limit]

Parameters:

  • address – mandatory account address.
  • startBlock
  • endBlock
  • limit – max result count. Range is 10 to 10000 (default).

Returns an array of pending and history transactions that have a from or to address that match address, in the format below. Limit can be speficied. The 12 Live blocks are first priority. If indexer module is not available, then the last 12 live blocks are not included.

{
    status: 1,
    message: "OK",
    count: 50,
    includeLiveBlocks: false,
    result: [
        {
            blockNumber: 3344530,
            timeStamp: 1489413225,
            blockHash: "0x034c2662fd9a4769f2772862241adf51c6110ba7aac1395e660c717e7b90cd45",
            confirmations: 1776309,
            hash: "0x0000474e48357b8e10ddece815732e631f3256d614c442a6094b1c25f8792bdc",
            nonce: 40,
            transactionIndex: 8,
            from: "0xE41d417c9E55Cc87B80Af94b6759bb9F9a036306",
            to: "0x6Fc82a5fe25A5cDb58bc74600A40A69C065263f8",
            value: "212222140000000000",
            gas: 47008,
            gasPrice: "20000000000",
            input: "0x",
            isError: 0,
            gasUsed: 31984,
            cumulativeGasUsed: 335176,
            contractAddress: null
        }
        ...
        ...
    ]
}

Token transactions

GET /tokens/{address}/{contractAddress}/[startBlock]/[endBlock]/[limit]

Parameters:

  • address – mandatory account address.
  • contractAddress – mandatory contract address.
  • startBlock
  • endBlock
  • limit – max result count. Range is 10 to 10000 (default).

Returns an array of pending and history contract transactions in the format below. Limit can be speficied, range is 50-1000. The 12 Live blocks are first priority. If indexer module is not available, then the last 12 live blocks are not included.

{
    status: 1,
    message: "OK",
    includeLiveBlocks: true,
    count: 1
    result: [
        {
            blockNumber: 3126792,
            timeStamp: 1486288489,
            blockHash: "0x0a2657b7ac67f520068f68b715662c39084bb65c1d588ec9dac04f7f6268c2b5",
            confirmations: 1994240,
            hash: "0x5b5bd839b2b6280a3052b7f4716764abad5bba54c48506cc89a3da6ef8f67b4d",
            nonce: 19,
            transactionIndex: 5,
            from: "0x000001f568875F378Bf6d170B790967FE429C81A",
            to: null,
            value: "0",
            gas: 1700000,
            gasPrice: "20000000000",
            input: "0x60606040525b60008054600160a060020a03191633600160a060020a0316179055
            [.. ..]
            fffffffffffff1916600160a060020a0383161790555b5b505600a165627a7a723058204a5af",
            isError: 0,
            gasUsed: 1647192,
            cumulativeGasUsed: 1882081,
            contractAddress: "0x088973C56EB9B5E616A5184b015642e574582c31"
        }
    ]
}

Getting Started

Install nodejs (v 9.0+) and npm (v 5.3+)

https://nodejs.org/en/download/package-manager/

  • make sure to install build tools.

Install Parity

bash <(curl https://get.parity.io -Lk) 

Install Choubase

Install couchdb. Make sure to create admin user on installation.

http://docs.couchdb.org/en/stable/install/unix.html

Config couchdb to run with enough open files descriptors to prevent "no DB shards could be opened" errors.

  1. http://docs.couchdb.org/en/stable/maintenance/performance.html#system-resource-limits

  2. In couch config local or remote

    local db: http://127.0.0.1:5984/_utils

    remote db: ssh -L5984:127.0.0.1:5984 [username]@yourserverip.

    Then in your broswer:

    http://localhost:5984/_utils/#_config/[email protected]

    max_dbs_open = 15000
    

    Increase limits as described here:

    https://superuser.com/questions/1200539/cannot-increase-open-file-limit-past-4096-ubuntu

  3. add /etc/systemd/system/couchdb.service.d/limits.conf file with the value

    [Service]
    LimitNOFILE=64000
    
  4. Make sure to change couchdb folders to your datadisk - inside /opt/couchdb/etc/default.ini or local.ini.

    *note that deault.ini will be override on couchdb upgrade.

    [couchdb]
    database_dir = /datadisk/couchdb/data
    
    [couchdb]
    view_index_dir = /datadisk/couchdb/data
    

    You can also do that by adding the values using managemnt (make sure to opne 5984 port)

    ssh -L5984:127.0.0.1:5984 [username]@yourserverip.

    Then in your broswer:

    http://localhost:5984/_utils/#_config/[email protected]

  5. Make sure to give couchdb permissions to write your dirs - it will not work without it. https://github.com/apache/couchdb/blob/master/INSTALL.Unix.md

Run Parity

Make sure to fully sync Parity before start indexing. Also, no-warp is important to get full history.

parity --no-warp --cache-size 2048

Clone git

git clone https://github.com/adyshimony/ether-super-node.git

Run NPM install

npm install

Build project

npm run build

Configuration

There are two ways to configure. Change the values in the config files and build, or pass parameters in command line.

Config file

Config file for Indexer

indexer/src/config/config.ts

Config file for REST

rest-api/src/config/config.ts

In Config class constructor, change default values if needed, for Indexer and REST

    this._provider = new web3.providers.HttpProvider('http://127.0.0.1:8545')
    this._port = 3000 
    this._useIpc = true
    this._DBUrl = 'http://admin:123456@localhost:5984'  

Make sure to build after config files change.

npm run build

Running

Both indexer and REST must run to get history and live data.

Run indexer

npm run start-indexer

Run rest

npm run start-rest

Command line options

Run npm run start-indexer -- --paremeter Where 'parameter' can be one or more of the following:

DB Url

dburl='http://admin:123456@localhost:5984'

CouchDB URL with user:password.

Port

port=XXXX

Port for incoming http requests.

Ipc

ipc=true (default) 

Set this flag to false if you want to conenct to parity with HTTP provider and not with inter-process communication. Ipc is much faster, but must be on the same machine.

Http provider

httpprovider='http://127.0.0.1:8545' (default)

Parity http provider.

Example

npm run start-indexer -- --dburl='http://admin:123456@localhost:5984' --port=3000
npm run start-rest -- --dburl='http://admin:123456@localhost:5984' --port=3001

Make sure for the pre double hyphen before the parameters, npm requires it for passing the paremeters to the running script.

indy's People

Contributors

adyshimony avatar paullinator avatar

Stargazers

Reef Loretto avatar Brett Henderson avatar Haries Nur Ikhwan avatar Gábor Mihálcz avatar Andrejs Agejevs avatar Igor Ruzanov avatar

Watchers

William Swanson avatar James Cloos avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.