Giter VIP home page Giter VIP logo

sticker's Introduction

stick

Squid based data used to index, process, and query on top of AssetHub for KodaDot NFT Marketplace.

Hosted Squids

Project structure

  • src/generated - model/server definitions created by codegen. Do not alter the contents of this directory manually.
  • src/server-extension - module with custom type-graphql based resolvers.
  • src/types - data type definitions for chain events and extrinsics created by typegen.
  • src/mappings - mapping module.
  • lib - compiled js files. The structure of this directory must reflect src.
  • .env - environment variables defined here or supplied by a shell.

Prerequisites

  • Node 18.x
  • Docker
  • npm
  • just

Quickly running the sample

# 1. Install dependencies
npm ci

# 2. Build project
just build

# 3. Start target Postgres database container
just upd

# 4. Update database with data objects
just migrate

# 5. Start the processor
just process

# 6. Open a separate terminal and launch the graphql server to query the processed data
just serve

# 7. Visit localhost:4350/graphql to see the result

Dev flow

1. Define database schema

Start development by defining the schema of the target database via schema.graphql. Schema definition consists of regular graphql type declarations annotated with custom directives. A full description of schema.graphql dialect is available here.

2. Generate TypeORM classes

Mapping developers use TypeORM entities to interact with the target database during data processing. The squid framework generates All necessary entity classes from schema.graphql. This is done by running just codegen command.

3. Generate database migration

All database changes are applied through migration files located at db/migrations. squid-typeorm-migration tool provides several commands to drive the process. It is all TypeORM under the hood.

# Connect to the database, analyze its state, and generate a migration to match the target schema.
# Launch Docker instance of the database
just upd

# The target schema is derived from entity classes generated earlier.
# Remember to compile your entity classes beforehand!
just update-db

# Apply database migrations from `db/migrations`
just migrate

# Revert the last performed migration
just revert-db

Available sqd shortcuts:

# Build the project, remove any old migrations, then run `npx squid-typeorm-migration generate`
sqd migration:generate

# Run npx squid-typeorm-migration apply
sqd migration:apply

Testing

Unit test early, unit test often

This indexer contains unit tests for utility/parsing functions we wrote.

Tests are located in the tests/ directory. To run the tests, use:

npm run test

Warning

Currently, it is impossible to unit test the whole indexer workflow as a dry run. If you encounter some problem, please head over to the telegram group HydraDevs

Architecture

The architecture of this project is following:

  • src/processable.ts - definition of Events and Extrinsic to be processed by Squid
  • src/processor.ts - processor definition
  • src/mappings/index - the main function that is called by the processor to process events and extrinsic
  • src/mappings/<pallet> - mapping functions for each event and extrinsic
  • src/mappings/<pallet>/types.ts - types for each event and extrinsic
  • src/mappings/<pallet>/getters/<chain>.ts - transformation functions for each event and extrinsic
  • src/mappings/utils - utility functions used by mappings

Misc

  1. fast generate event handlers
pbpaste | cut -d '=' -f 1 | tr -d ' '  | xargs -I_ echo "processor.addEventHandler(Event._, dummy);"
  1. enable debug logs (in .env)
SQD_DEBUG=squid:log
  1. generate metagetters from getters
pbpaste | grep 'export'  | xargs -I_ echo "_  return proc.  }"

Funding

Project was funded as a common good by

version

sticker's People

Contributors

achilles0316 avatar

Watchers

 avatar

sticker's Issues

what is typescript?

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
Screenshot from KodaDot/Explorer

Additional context
Add any other context about the problem here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.