A set of scripts to download and update Decentralchain history data into a PostgreSQL 11.x database.
-
Clone the repository, install dependencies.
npm install
-
Create
config.yml
file in the project, usingconfig.example.yml
for reference. -
In PostgreSQL, create empty database.
-
Set environment variable
MIGRATE
totrue
(or just run crawler like this:MIGRATE=true npm run ...
), it will apply initial and all additional migrations to yours database. -
⬇️ To download a range of blocks to database:
npm run download {start} {end}, # for example npm run download 1 100000
Blocks from the range get inserted in a single transaction, so either all get inserted, or none. In our experience ranges of 10000—100000 work best.
-
🔄 To keep your database up-to-date:
npm run updateComposite
This is a continuous script, so you may want to run it in the background. We recommend using some kind of process manager (e.g.
pm2
) to restart the process on crash.
- Create migration:
./node_modules/.bin/knex --migrations-directory migrations migrate:make $MIGRATION_NAME
- Migrate latest:
./node_modules/.bin/knex migrate:latest --client postgresql --connection postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE --migrations-directory migrations # OR npm run migrate -- --connection postgresql://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE