astraly-labs / pragma-sdk Goto Github PK

Current Behavior

There is a fork_devnet network in PragmaClient that's only used for integration tests.

Expected Behavior

Add a --fork=<network> to pytest that's only usable in combination with --net=devnet.
Remove the fork_devnet work and just use devnet instead.

What happened

#16 introduced a regression where providing a URL for the RPC network conflicts with many params like CHAIN_IDS and CONTRACT_ADDRESSES.

Stack trace

Steps to reproduce

SDK Version

1.0.0

Python version

3.9

What operating system are you using?

Mac

Is there an existing issue for this?

• I have searched the existing issues and verified no issue exits for this problem.

What happened

Once #48 is done, we can fix this test.

I couldn't find a way to correctly mock post requests depending on the body which is only a problem for TheGraphFetcher as it works with POST requests and not GET requests as other fetchers do.

Note that weirdly enough, it works in the async case.

Stack trace

fetcher_config = {'expected_result': [SpotEntry(pair_id="BTC/USD", price=3459885191309, timestamp=12345, source="THEGRAPH", publisher="...file': PosixPath('/Users/matthall/Documents/pragma-sdk/pragma/tests/mock/responses/thegraph.json'), 'name': 'TheGraph'}
mock_data = {'BTC': {'data': {'pool': {'feeTier': '3000', 'liquidity': '12133869505104', 'sqrtPrice': '147370456125399510359475624...500', 'liquidity': '20753232973335935007', 'sqrtPrice': '1867199352400732300487913333649122', 'tick': '201362', ...}}}}

    @mock.patch("time.time", mock.MagicMock(return_value=12345))
    def test_fetcher_sync_success(fetcher_config, mock_data):
        with requests_mock.Mocker() as mocker:
            fetcher = fetcher_config["fetcher_class"](SAMPLE_ASSETS, PUBLISHER_NAME)
    
            # Mocking the expected call for assets
            for asset in SAMPLE_ASSETS:
                quote_asset = asset["pair"][0]
                base_asset = asset["pair"][1]
                url = fetcher.format_url(quote_asset, base_asset)
    
                # TODO (#000): Fix this test
                if fetcher_config["name"] == "TheGraph":
                    query = fetcher.query_body(quote_asset)
                    print(query)
                    mocker.post(
                        url,
                        json=mock_data[quote_asset],
                        additional_matcher=lambda request: request.text
                        == '{"query": "' + query + '"}}',
                    )
                else:
                    mocker.get(url, json=mock_data[quote_asset])
    
>           result = fetcher.fetch_sync()
...
requests_mock.exceptions.NoMockAddress: No mock address: POST https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3

Steps to reproduce

1. poe test_ci_fetchers

SDK Version

1.0.4

Python version

3.9.13

What operating system are you using?

Mac

Is there an existing issue for this?

• I have searched the existing issues and verified no issue exits for this problem.

Feature Request

Current Behavior

Right now, there is no fetcher that can be used to retrieve the price of an on-chain asset only traded on Starknet.

Expected Behavior

There should be a new fetcher similar to TheGraphFetcher.
It should work with the following example asset:

{
        "type": "ONCHAIN",
        "source": "JEDISWAP",
        "key": "DEX",
        "detail": {
            "asset_name": "LORDS",
            "asset_address": "0x0124aeb495b947201f5fac96fd1138e326ad86195b98df6dec9009158a533b49",
            "metric": "price",
        },
        "decimals": 18,
}

Acceptance Criteria

An integration test suite should be added for this fetcher.
The contributor is free to use whatever indexing tool/API to query the underlying data.

Supported Metrics

price : When this metric is chosen, the fetcher should return a SpotEntry with the price given in USD terms
liquidity: USD denominated liquidity of the asset on the given source exchange.

Useful Resources

https://www.geckoterminal.com/dex-api
https://bitquery.io/products/dex
https://www.apibara.com/docs

Current Behavior

PragmaPublisherClient currently only supports v2 transactions where fees are paid in ETH.

Expected Behavior

We should also support v3 transactions given a new flag enable_strk_fees is provided.

Feature Request

Current Behavior

Linting job in CI fails.

Expected Behavior

CI should pass.

What happened

Running the fetchers scripts (cron or locally) returns the following error:

Stack trace

ClientOSError(1, '[SSL: APPLICATION_DATA_AFTER_CLOSE_NOTIFY] application data after close notify (_ssl.c:2756)')

Steps to reproduce

See CONTRIBUTING.md

SDK Version

1.1.6

Python version

3.9

What operating system are you using?

Mac

Is there an existing issue for this?

• I have searched the existing issues and verified no issue exits for this problem.

Feature Request

Current Behavior

OkxFutureFetcher is not tested in fetchers_test.py

Expected Behavior

OkxFutureFetcher should be added to the FUTURE_FETCHER_CONFIGS object and properly tested.

Feature Request

The SDK currently doesn't support bid/ask data and aggregates it to the midprice.
We should provide an easy way to skip this aggregation and return the bid/ask raw data.

The contract interface would look like this

Feature Request

Pragma adds a VRF module.

Currently there is no easy way to interact with this contract, we should add this functionality to the SDK, probably as a new Mixin.

Useful Resources
Previous implementation in old SDK version
Handler Lambda used to fulfil requests

Feature Request

Current Behavior

Right now, the SDK excepts an RPC_KEY environment variable in order to interact with testnet/mainnet.

Expected Behavior

The user shouldn't need to specify an RPC_KEY and the SDK should integrate a pre-defined sets of RPC endpoints that the SDK user can use.
The user should also be able to specify their own RPC endpoint.

Current Behavior

The offchain mixin only allows a data provider to publish data on the API.

Expected Behavior

We should create a new specific PragmaAPIClient type that takes the api key and api base url with default values for dev/prod.
This client should be able to easily call the different endpoints supported by the API and return the data in an understandable format with the associated data types.
The PragmaPublisherClient can be morphed into this new type.

Feature Request

Current Behavior

BinanceFutureFetcher is not tested in fetchers_test.py

Expected Behavior

BinanceFutureFetcher should be added to the FUTURE_FETCHER_CONFIGS object and properly tested.

Feature Request

Current Behavior

There is no fetcher using AVNU API.

Expected Behavior

Add support for fetching assets' prices with AVNU API.

Feature Request

I think it would be great if the timeout could be configured as a key parameter in the PragmaPublisherClient constructor, or as a param of the fetch method.

Feature Request

Current Behavior

TheGraph is only being used to fetch interest rate data from Aave.

Expected Behavior

TheGraph can be used to fetch SpotEntry from both Uniswap V2/V3.
It should be added to the integration tests.

What happened

The testing process on the add-randomness-tests branch returns an new error message: invalid signature, with respect to the public key, while running tests on the katana instance.

Stack trace

Transaction validation error: ValidateTransactionError(VirtualMachineExecutionErrorWithTrace { trace: "Error in the called contract (0x07b245927f60309cb2fdcbc3f7d37a21ce5bc05a4305104b43b72b91aa3f53e1):\nError at pc=0:124:\nSignature 01d55041f4c9e0ee47c3a77118ba08a7236b6f180c3d32bd8c7c30df8a601ef602f68789e52ec9a7ab574496a03c290976f211a2a0bba9d6cb0dd4f78b3c0865, is invalid, with respect to the public key 2165217942936718651316664700585810015615719713984440615960412299420767794974, \n and the message hash 1924305910628053302524635289018546823154332958160078634104751209254614372567.\nCairo traceback (most recent call last):\nUnknown location (pc=0:615)\nUnknown location (pc=0:600)\nUnknown location (pc=0:245)\n", source: CairoRunError(VmException(VmException { pc: 124, inst_location: None, inner_exc: Memory(InvalidSignature(("01d55041f4c9e0ee47c3a77118ba08a7236b6f180c3d32bd8c7c30df8a601ef602f68789e52ec9a7ab574496a03c290976f211a2a0bba9d6cb0dd4f78b3c0865", 2165217942936718651316664700585810015615719713984440615960412299420767794974, 1924305910628053302524635289018546823154332958160078634104751209254614372567))), error_attr_value: None, traceback: Some("Cairo traceback (most recent call last):\nUnknown location (pc=0:615)\nUnknown location (pc=0:600)\nUnknown location (pc=0:245)\n") })) })

Steps to reproduce

1. poe test_ci_vrf from the add-randomness-tests branch

SDK Version

latest

Python version

3.9

What operating system are you using?

Mac

Is there an existing issue for this?

• I have searched the existing issues and verified no issue exits for this problem.

Feature Request

Current Behavior

Binance and ByBit Futures APIs are banned from US IPs and unfortunately github runners are only located in the US.

Expected Behavior
We should self-host a runner for the CI.

Feature Request

Current Behavior

Currently, fetchers such as GeckoTerminal or TheGraph only support USD as a base asset.

Expected Behavior

We should add support for ETH as a base asset.
We should define a list of supported base assets.

Feature Request

Currently, the stagecoach folder sits in the main repo, we should probably either move it here or include it as a submodule.

• Test lambda functions on PR/Merge (we can mock the aws part)
• Push lambda functions docker images to ECR + ghcr on new SDK release (checkpoint + starknet-publisher)

Feature Request

Current Behavior

Currently TheGraphFetcher only queries SpotEntry from Uniswap V3.

Expected Behavior

TheGraph should be abstracted away and we should be able to create fetchers that inherit from a base TheGraphFetcher class and easily decide what data to fetch Spot/Future/Generic, which subgraph to use, which data to query..

A PR that solves this issue should create a UniswapV2Fetcher and UniswapV3Fetcher to demonstrate the work.

Fetchers currently follow this interface

class PublisherInterfaceT(abc.ABC):
    @abc.abstractmethod
    async def fetch(self, session: ClientSession) -> List[Any]:
        ...

    @abc.abstractmethod
    def fetch_sync(self) -> List[Any]:
        ...

    @abc.abstractmethod
    def format_url(self, quote_asset, base_asset) -> str:
        ...

    async def _fetch(self):
        async with aiohttp.ClientSession() as session:
            data = await self.fetch(session)
            return data

We should remove fetch_sync and leverage the add_sync_methods decorator cc here