Giter VIP home page Giter VIP logo

pragma-monitoring's Introduction

Pragma Monitoring

Prometheus Exporter

This service runs a prometheus exporter that exposes metrics on the /metrics route. It powers our internal grafana dashboards and alerts.

This service polls at a regular interval the database that is filled by our indexers. It then processes the data and computes the following metrics:

  • time_since_last_update_seconds{network, publisher, typr}: Time since a publisher has published any data. (in seconds)
  • pair_price{network, pair, source, type}: Latest price of an asset for a given source and pair. (normalized to asset's decimals)
  • time_since_last_update_pair_id{network, pair, type}: Time since an update has been published for a given pair. (in seconds)
  • price_deviation{network, pair, source, type}: Deviation of the price from a reference price (DefiLlama API) given source and pair. (in percents)
  • price_deviation_source{network, pair, source, type}: Deviation of the price from the on-chain aggregated median price given source and pair. (in percents)
  • publisher_balance{network, publisher}: Balance of a publisher. (in ETH)

Shared Public Access

Monitoring is not publicicly available yet but databases will soon be in read-only mode.

Self-Hosting

We have created a docker-compose.yml file to help with self-hosting setup:

docker compose up -d

You can then access prometheus dashboard at http://localhost:9000 and grafana at http://localhost:3000.

Make sure to first fill the envirronement file with your own config parameters:

# The database URL the application will use to connect to the database.
DATABASE_URL='postgres://postgres:postgres@localhost:5432/postgres'

# (Optional) Defillama API Key
DEFILLAMA_API_KEY=

# Pragma API key
PRAGMA_API_KEY=

# RPC URL
RPC_URL=

# Indexer Service URL
INDEXER_SERVICE_URL=

# Config
NETWORK=testnet
ORACLE_ADDRESS=0x
PAIRS=BTC/USD,ETH/USD
IGNORE_SOURCES=BITSTAMP,DEFILLAMA
IGNORE_PUBLISHERS=BINANCE

# Prometheus
TELEGRAM_BOT_TOKEN=
OPSGENIE_API_KEY=

In order for the full flow to work you will need to have tables following the table schemas defined here.

You can use our indexer service on this repository to spin off your indexer in a few commands very easily.

pragma-monitoring's People

Contributors

akhercha avatar evolveart avatar jordyro1 avatar

Watchers

avatar avatar

pragma-monitoring's Issues

dev: backups

Current Behavior

The monitoring heavily depends on these 3 external dependencies:

  • Defillama API
  • Apibara (Indexing)
  • RPC Provider (Infura)

Expected Behavior

We should provide at least 1 backup solution for each of these components

  • If Defillama is down use another one (Coingecko? Coinmarketcap?)
  • If Infura is down, backup on Alchemy or Blast
  • If Apibara is down, backup on Checkpoint or DipDup ?

The hardest one being the indexer we should probably come up with an elegant design.
The main source of truth is the Postgres DB which is filled with indexed data => We always need to keeps the DB in sync with what is happening on-chain.

dev: long tail assets custom alert

For long tail assets, we don't want to compute the deviation against Defillama but rather look at the deviation between each other and trigger an alert if that gap is higher than a threshold.

dev: integration tests

We're missing a few things to be able to perform integration tests for this service:

  • Account Impersonation in devnet or katana. This is needed in order to simulate publishers providing wrong prices.
  • Historical Data on forking. This is needed in order to simulate indexing on old data. We could also probably use snapshot testing method for this.

dev: add processing function for publishers

Current Behavior

We currently loop on the sources mapping and query the latest entry given a couple (pair_id, source)>
When the same source is published by different data providers this leads to the monitoring not updating publisher-related metrics.

Expected Behavior

Move publisher-related processing to another task.

dev: fix future data deviation

Currently, deviation on future assets also uses Defillama as a reference point.
This should be changed to use an API that has future data.

dev: detect better indexer downtime

Currently when the GRPC indexer status server goes down, it's not detected by our monitoring.
Maybe this is simply solved by upgrading to latest apibara version on the indexer service, to be checked.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.