Website | Docs | Discord

Subsquid Network FAQ

This is a quest to run a squid with four processors. Here is how to run it:

mkdir ~/global-node-packages

npm config set prefix ~/global-node-packages

CURSHELL=`ps -hp $$ | awk '{print $5}'`; case `basename $CURSHELL` in 'bash') DEST="$HOME/.bash_profile";; 'zsh') DEST="$HOME/.zshenv";; esac; echo 'export PATH="${HOME}/global-node-packages/bin:$PATH"' >> "$DEST"

export PATH="${HOME}/global-node-packages/bin:$PATH"

mkdir ~/global-node-packages

npm config set prefix ~/global-node-packages

export PATH="${HOME}/global-node-packages/bin:$PATH"

Open a terminal and run

npm install --global @subsquid/cli@latest

This adds the sqd command. Verify that the installation was successful by running

sqd --version

A healthy response should look similar to

@subsquid/cli/2.5.0 linux-x64 node-v20.5.1

1. Open a terminal and run the following commands to create the squid and enter its folder:

   sqd init my-quad-proc-squid -t https://github.com/subsquid-quests/quad-chain-squid

   cd my-quad-proc-squid

   You can replace my-quad-proc-squid with any name you choose for your squid. If a squid with that name already exists in Aquarium, the first command will throw an error; if that happens simply think of another name and repeat the commands.

2. Press "Get Key" button in the quest card to obtain the quadProc.key key file. Save it to the ./query-gateway/keys subfolder of the squid folder. The file will be used by the query gateway container.

3. The template squid uses a PostgreSQL database and a query gateway. Start Docker containers that run these with

   sqd up

   Wait for about a minute before proceeding to the next step.

   If you get an error message about unknown shorthand flag: 'd' in -d, that means that you're using an old version of docker that does not support the compose command yet. Update Docker or edit the commands.json file as follows:

            "up": {
            "deps": ["check-key"],
            "description": "Start a PG database",
   -        "cmd": ["docker", "compose", "up", "-d"]
   +        "cmd": ["docker-compose", "up", "-d"]
          },
          "down": {
            "description": "Drop a PG database",
   -        "cmd": ["docker", "compose", "down"]
   +        "cmd": ["docker-compose", "down"]
          },

4. Prepare the squid for running by installing dependencies, building the source code and creating all the necessary database tables:

   npm ci
   sqd build
   sqd migration:apply

5. Start your squid with

   sqd run .

   The command should output lines like these:

   [eth-processor] 04:41:33 INFO  sqd:processor processing blocks from 16000000
   [moonbeam-processor] 04:41:33 INFO  sqd:processor processing blocks from 3000000
   [bsc-processor] 04:41:33 INFO  sqd:processor processing blocks from 28000000
   [base-processor] 04:41:33 INFO  sqd:processor processing blocks from 3800000
   [eth-processor] 04:41:33 INFO  sqd:processor using archive data source
   [eth-processor] 04:41:33 INFO  sqd:processor prometheus metrics are served at port 38397
   [moonbeam-processor] 04:41:33 INFO  sqd:processor using archive data source
   [moonbeam-processor] 04:41:33 INFO  sqd:processor prometheus metrics are served at port 38899
   [base-processor] 04:41:33 INFO  sqd:processor using archive data source
   [base-processor] 04:41:33 INFO  sqd:processor prometheus metrics are served at port 42565
   [bsc-processor] 04:41:33 INFO  sqd:processor using archive data source
   [bsc-processor] 04:41:33 INFO  sqd:processor prometheus metrics are served at port 33067
   [api] 04:41:34 WARN  sqd:graphql-server enabling dumb in-memory cache (size: 100mb, ttl: 1000ms, max-age: 1000ms)
   [api] 04:41:34 INFO  sqd:graphql-server listening on port 4350
   [eth-processor] 04:41:36 INFO  sqd:processor 16007919 / 18250052, rate: 2572 blocks/sec, mapping: 752 blocks/sec, 1449 items/sec, eta: 15m
   [eth-processor] 04:41:41 INFO  sqd:processor 16011899 / 18250052, rate: 2712 blocks/sec, mapping: 696 blocks/sec, 1345 items/sec, eta: 14m
   [bsc-processor] 04:42:42 INFO  sqd:processor 28004339 / 32201121, rate: 64 blocks/sec, mapping: 543 blocks/sec, 1041 items/sec, eta: 18h 19m
   [moonbeam-processor] 04:42:44 INFO  sqd:processor 3099480 / 4541860, rate: 1412 blocks/sec, mapping: 897 blocks/sec, 2726 items/sec, eta: 18m
   

   The squid should sync in 30-35 minutes. When it's done, stop it with Ctrl-C, then stop and remove the auxiliary containers with

   sqd down

Sync this squid using the key from the quest card. The syncing progress is tracked by the amount of data the squid has retrieved from Subsquid Network.

This squid captures USDC Transfer events on ETH, BSC, Base and Moonbeam, stores them in the same database and serves the data over a common GraphQL API.

Data ingester ("processor") code for each network is located at the corresponding src/ subdirectory: src/eth, src/bsc, src/base andsrc/moonbeam. The scripts file commands.json contains commands for running each processor (process:eth, process:bsc, process:base and process:moonbeam correspondingly). GraphQL server runs as a separate process started by sqd serve. You can also use sqd run to run all the services at once.

The squid uses Subsquid Network as its primary data source.