santiment / sanpy Goto Github PK

If you invoke san.get("projects/bla") you get a cryptic error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/valentin/.pyenv/versions/3.6.3/lib/python3.6/site-packages/san/get.py", line 15, in get
    gql_query = get_gql_query(0, dataset, **kwargs)
  File "/Users/valentin/.pyenv/versions/3.6.3/lib/python3.6/site-packages/san/query.py", line 10, in get_gql_query
    )(idx, slug, **kwargs)
  File "/Users/valentin/.pyenv/versions/3.6.3/lib/python3.6/site-packages/san/sanbase_graphql.py", line 103, in projects
    raise SanError("Unknown project group: {}".format(slug))
NameError: name 'SanError' is not defined

We need to improve the error here.

When upgrading the client from 0.10.2 to 0.11.1 I get an error when calling san.available_metrics

Minimal example

import san
san.available_metrics

The end of the tracelog looks like the following

... in available_metrics()
6 sanbase_graphql_functions = inspect.getmembers(san.sanbase_graphql, inspect.isfunction)
7 all_functions = list(map(lambda x: x[0], sanbase_graphql_functions)) + execute_gql('{query: getAvailableMetrics}')['query']
----> 8 all_functions.remove('get_metric')
9 return all_functions
ValueError: list.remove(x): x not in list

In 0.10.2 there was no issue with the call.

Let's add an MIT license to the repo, before we open source it

Add integration testing to verify whether API ahs changed

Users will be using the library as an external egg. We should change the install instructions to show how you can install the package with pip. Something like ‘pip install https://github.com/santiment/San-Python’. Later we can release it to the official pip repository and start releasing versions.

That should allow easier handling of the time series data.

I was just wondering when it would be possible to support the latest pandas version.

I'm trying to generate a matrix of:
a) metrics supported by santiment (ie. san.available_metrics())
b) slugs available for metrics in (a)

Is there an efficient way to achieve this without having to loop through all slugs running san.available_metrics('slug_name).

Best
Mark

Trying to do this:

san.get("prices/bitcoin", from_date='2018-02-08', to_date=datetime.datetime.now(), interval="1d")

gives an error:

ParseError: Expecting a string datetime.datetime(2018, 7, 5, 17, 23, 46, 788937)

I think we can make the date time parameters accept datetime/date objects directly.

That will allows to see the list of available slug, project names and tickers.

When building a query, how do we set the transform kwarg?

For example, if I want a moving average

I would use this graphql query:

{
  getMetric(metric: "price_usd") {
    timeseriesData(
      slug: "ethereum"
      from: "2021-05-01T00:00:00Z"
      to: "2021-06-01T00:00:00Z"
      includeIncompleteData: false
      interval: "30m"
      aggregation: AVG
      transform: {type: "moving_average", moving_average_base: 100}) {
        datetime
        value
    }
  }
}

However, in sanpy Batch.get doesn't seem to be reading the transform kwarg.

batch = Batch()
batch.get(
    "price_usd/ethereum",
    from_date="2021-05-01T00:00:00Z",
    to_date="2021-06-03T00:00:00Z",
    interval="30m",
    aggregation="AVG",
    transform="{type: \"moving_average\", moving_average_base: 100}"
)
result = batch.execute()
result

Seems like a string formatting issue, because this totally hacky method of building the graphql query works as expected

batch = Batch()
batch.get(
    "price_usd/ethereum",
    from_date="2021-05-01T00:00:00Z",
    to_date="2021-06-03T00:00:00Z",
    interval="30m",
    aggregation="AVG \n  transform: {type: \"moving_average\", moving_average_base: 100}"
)
result = batch.execute()
result

to_date=2018-10-29 defaults to 2018-10-29T00:00:00. It will be better to default to 2018-10-29T23:59:59 so it returns all the data for the day.

san.get('transaction_volume/omisego', from_date='2018-10-28', to_date='2018-10-29')
                           transactionVolume
datetime
2018-10-28 00:00:00+00:00       3.226938e+05
2018-10-29 00:00:00+00:00       1.281221e+03

>>> san.get('transaction_volume/omisego', from_date='2018-10-28', to_date='2018-10-29T00:00:00')
                           transactionVolume
datetime
2018-10-28 00:00:00+00:00      322693.810945
2018-10-29 00:00:00+00:00        1281.220900

san.get('transaction_volume/omisego', from_date='2018-10-28', to_date='2018-10-29T23:59:59')
                           transactionVolume
datetime
2018-10-28 00:00:00+00:00       3.226938e+05
2018-10-29 00:00:00+00:00       9.269391e+05

I have been toying around with SAN API and now I wanted to try some models with up-to-date data. For that, I tried to get a 14-trial Pro account. However, when I try to retrieve data from the last month or older than 2y, I still get the error:

 "parameters are outside the allowed interval you can query exchange_outflow_usd with your current subscription SANAPI free. Upgrade to a higher tier in order to access more data."

Does it mean that the 14-days free trial does not include the API keys usage?

365 days are hardcoded here and here

We need to compile a continue supporting a CHANGELOG file that tracks what's new, fixed, and if there are breaking changes in every version.

Let's start with retrospectively adding what changed in versions 0.9.x, 0.10.x, and 0.11.x

Example changelog format that we can try to replicate: https://github.com/elixir-lang/elixir/blob/v1.13/CHANGELOG.md
As this project is much smaller, we can do this in a single file and not one file per major version.

Thanks for the awesome work on this library. It really makes interacting with santiment easier. I have a suggestion though that I could make the user experience slightly better.

I am getting the following user warning:
/home/ubuntu/.local/lib/python3.10/site-packages/san/__init__.py:31: UserWarning: The package sanpy is out of date. Your version is 0.10.1, the latest is 0.10.2.

I don't think that it's a good idea to warn when a python package is out of date. I use 100 hundreds of python packages. If I would get a warning when the patch version of each package is out of date, then I would be flooded by user warnings. So maybe you can emit a warning there has been a breaking change and the major version is out of date instead?

Add note in the README

I have been using your API for a couple of weeks and I have seen that the call limit is 20K items. I see that this also applies when using a batch call as the total number of elements gets aggregated. That is, it's possible to call 3 separate metrics of 20K items, one at a time but it is not possible to call a batch of the three 20K metrics at the same time:

from san import Batch
import san

# This will work
my_metrics = ['daily_active_addresses', 'active_addresses_24h', 'adjusted_price_daa_divergence']
separate_metrics = []
for i in my_metrics:
    metric_i=san.get(
        i + "/santiment",
        from_date="2018-06-01",        
        to_date="2019-06-05",
        interval="1h"
        )
    separate_metrics.append(metric_i)

batch = Batch()
# This will trigger the limit call
for i in my_metrics:

    batch.get(
        i + "/santiment",
        from_date="2018-06-01",
        to_date="2019-06-05",
        interval="1d"
        )

combined_metrics = batch.execute()

Therefore, I was wondering if each call from the batch counts as 1 call for the limit count or as many as metrics are in the call (in this case 3).

I was wondering if there is any function, even if it is ray graphQL, to call the screener that we can use through the web interface.

errors: [{'locations': [{'column': 9, 'line': 3}], 'message': '[e9204349-22a9-4215-9d5c-a28a1463351a] Can't fetch contract_interacting_addresses_count for project with slug celsius, Reason: "The metric 'contract_interacting_addresses_count' must have at least one of the following fields in the selector: contractAddress"', 'path': ['query_0', 'timeseriesData']}]

san.get("{}/{}".format("contract_interacting_addresses_count", slug),
contractAddress=contract_address,
from_date=from_date,
to_date="utc_now",
interval=interval
)

What is the proper format?

Hello there,
I would like to know how to get intraday metrics for example mvrv_usd_intraday_30d by using your API. I was able to obtain daily based metrics like mvrv_usd_30d by using the code:

result = san.get(metric_name,
from_date=(datetime.datetime.utcnow()-datetime.timedelta(days=30)),
interval=time_interval
)

But I am getting empty data frame instead when trying to receive intraday data.
I have a paid subscription by the way.
What i am doing wrong? How I can get intraday data, for example mvrv_usd_intraday_30d?

Hello,

I was searching for something similar to @phisanti for the same reasons as him.
It appears that your solution (which I find great) seems to not be working for the metric "twitter_followers"
would you have any insight on why ?

{
  getMetric(metric: "twitter_followers") {
    timeseriesDataPerSlug(
      selector: {slugs: [
        "ethereum",
        "bitcoin"
      ]},
      from: "utc_now-7d",
      to: "utc_now"
      interval: "1d"){
        datetime
        data {slug value}
      }
  }
}

Thank you very much :)

Originally posted by @marc-moreaux in #135 (comment)

The actual errors should be returned

PS K:\python bot> pip show sanpy Name: sanpy Version: 0.10.0 Summary: Package for Santiment API access with python Home-page: https://github.com/santiment/sanpy Author: Santiment Author-email: [email protected] License: MIT Location: f:\program file\python\lib\site-packages Requires: iso8601, pandas, requests, setuptools Required-by:
PS K:\python bot> pip show pandas Name: pandas Version: 1.4.2 Summary: Powerful data structures for data analysis, time series, and statistics Home-page: https://pandas.pydata.org Author: The Pandas Development Team Author-email: [email protected] License: BSD-3-Clause Location: f:\program file\python\lib\site-packages Requires: numpy, python-dateutil, pytz Required-by: sanpy
this is pandas and sanpy version in vscode but after test the lib and run program I see this error .
this is my program :
`import san
san.get(
"daily_active_addresses/santiment",
from_date="2018-06-01",
to_date="2018-06-05",
interval="1d"
)

san.get(
"prices/santiment",
from_date="2018-06-01",
to_date="2018-06-05",
interval="1d"
)and this is my error 👍 Exception has occurred: ModuleNotFoundError
No module named 'san'
File "K:\python bot\sanpy.py", line 1, in
import sanalso I see this in terminal 👍:Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "c:\Users\amir.vscode\extensions\ms-python.python-2022.8.0\pythonFiles\lib\python\debugpy_main.py", line 45, in
cli.main()
File "c:\Users\amir.vscode\extensions\ms-python.python-2022.8.0\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444,
in main
in run_file
runpy.run_path(target_as_str, run_name=compat.force_str("main"))
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "k:\python bot\sanpy.py", line 1, in
import san
ModuleNotFoundError: No module named 'san'`
🥇 also see this in problem section of vscode :

[{
"resource": "/k:/python bot/sanpy.py",
"owner": "generated_diagnostic_collection_name#3",
"code": {
"value": "reportMissingImports",
"target": {
"$mid": 1,
"external": "https://github.com/microsoft/pylance-release/blob/main/DIAGNOSTIC_SEVERITY_RULES.md#diagnostic-severity-rules",
"path": "/microsoft/pylance-release/blob/main/DIAGNOSTIC_SEVERITY_RULES.md",
"scheme": "https",
"authority": "github.com",
"fragment": "diagnostic-severity-rules"
}
},
"severity": 4,
"message": "Import "san" could not be resolved",
"source": "Pylance",
"startLineNumber": 1,
"startColumn": 8,
"endLineNumber": 1,
"endColumn": 11
}]

- [ ] how can i solve this and use the lib ?

san.error.SanError: GraphQL error occured running query {
query_0: getMetric(metric: "age_distribution"){
timeseriesData(
slug: "santiment"

        from: "utc_now-30d"
        to: "utc_now"
        interval: "1d"
        aggregation: null
        includeIncompleteData: false
    ){
    datetime value
    }
}
} 

errors: [{'locations': [{'column': 9, 'line': 3}], 'message': '[68531562-8a4e-4147-b5b3-7578a95eb4a7] Can't fetch age_distribution for project with slug santiment, Reason: "The metric 'age_distribution' is not supported or is mistyped. Did you mean the histogram metric 'age_distribution'?"', 'path': ['query_0', 'timeseriesData']}]