Comments (6)
leftys
commented on June 18, 2024
Hi, did you get this email we sent you two hours ago?
did you receive the registration email? The instructions are there together with your credentials and links to a few places to get started (documentation, examples).
The documentation about open interest data are on Data types page of our web.
Also did you get the registration email with subject Welcome to Lake? Everything is described there. Please check your spam folders as well. Also you are using two or three emails/accounts, one on outlook.com, one on ntnu.no domains, so you might want to check both as well.
from lake-api.
dekacypher
commented on June 18, 2024
Hi, yes I've got it on my mail, and have successfully creates the credentials file in the right path. I'm trying to fetch open interest data but when i run this code I, get an error:
lakeapi.use_sample_data(anonymous_access = False)
trades = lakeapi.load_data(
table="origin_time",
start=datetime.datetime(2022, 1, 1),
end=datetime.datetime(2022, 5, 1),
symbols=["BTC-USDT"],
exchanges=['BINANCE'],
)
trades.set_index('received_time', inplace = True)
trades
{
"name": "NoFilesFound",
"message": "No data found for your query",
"stack": "---------------------------------------------------------------------------
NoFilesFound Traceback (most recent call last)
/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb Cell 2 line 3
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=0'>1</a> lakeapi.use_sample_data(anonymous_access = False)
----> <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=2'>3</a> trades = lakeapi.load_data(
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=3'>4</a> table=\"origin_time\",
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=4'>5</a> start=datetime.datetime(2022, 1, 1),
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=5'>6</a> end=datetime.datetime(2022, 5, 1),
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=6'>7</a> symbols=[\"BTC-USDT\"],
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=7'>8</a> exchanges=['BINANCE'],
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=8'>9</a> )
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=9'>10</a> trades.set_index('received_time', inplace = True)
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=10'>11</a> trades
File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:161, in load_data(table, start, end, symbols, exchanges, bucket, boto3_session, use_threads, columns, row_slice, drop_partition_cols, cached)
158 for _ in range(2):
159 try:
160 # TODO: log & skip corrupted files
--> 161 df = lakeapi._read_parquet.read_parquet(
162 path=f\"s3://{bucket}/{table}/\",
163 partition_filter=partition_filter,
164 categories=[\"side\"] if table == \"trades\" else None,
165 dataset=True, # also adds partition columns
166 boto3_session=boto3_session,
167 columns=columns,
168 use_threads=use_threads,
169 ignore_index=True,
170 cached=cached,
171 )
172 break
173 except botocore.exceptions.ClientError as ex:
174 # When 404 file not found error happens, it means the boto cache of available files is wrong and we need
175 # to clear it and try again.
File /opt/homebrew/lib/python3.11/site-packages/lakeapi/_read_parquet.py:576, in read_parquet(path, path_root, path_suffix, path_ignore_suffix, version_id, ignore_empty, ignore_index, partition_filter, columns, chunked, dataset, categories, safe, map_types, use_threads, last_modified_begin, last_modified_end, boto3_session, s3_additional_kwargs, pyarrow_additional_kwargs, cached)
574 paths = _apply_partition_filter(path_root=path_root, paths=paths, filter_func=partition_filter)
575 if len(paths) < 1:
--> 576 raise exceptions.NoFilesFound(\"No data found for your query\")
577 _logger.debug(\"paths:\
%s\", paths)
579 args: Dict[str, Any] = {
580 \"columns\": columns,
581 \"categories\": categories,
(...)
589 \"pyarrow_additional_kwargs\": pyarrow_additional_kwargs,
590 }
NoFilesFound: No data found for your query"
}
from lake-api.
leftys
commented on June 18, 2024
Please delete the lakeapi.use_sample_data call, that switches you to the small free sample dataset. Also table parameter should be trades, book, candles, funding etc (see https://crypto-lake.com/data/). Finally I recommend to update to the newest lakeapi package version. New version should have gave you a better error message in your case iirc.
Have fun exploring the data and we wish you successful research& trading! :)
from lake-api.
dekacypher
commented on June 18, 2024
Thank you so much for your help! I really appreciate it.
I made changes that you proposed, but now i get a different error:
trades = lakeapi.load_data(
table="open_interest",
start=datetime.datetime(2022, 1, 1),
end=datetime.datetime(2022, 5, 1),
symbols=["BTC-USDT"],
exchanges=['BINANCE'],
)
trades.set_index('received_time', inplace = True)
trades
{
"name": "NoFilesFound",
"message": "No files Found on: s3://sample.crypto.lake/open_interest/.",
"stack": "---------------------------------------------------------------------------
NoFilesFound Traceback (most recent call last)
/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb Cell 2 line 3
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=0'>1</a> get_ipython().run_line_magic('pip', 'install --upgrade lakeapi')
----> <a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=2'>3</a> trades = lakeapi.load_data(
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=3'>4</a> table=\"open_interest\",
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=4'>5</a> start=datetime.datetime(2022, 1, 1),
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=5'>6</a> end=datetime.datetime(2022, 5, 1),
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=6'>7</a> symbols=[\"BTC-USDT\"],
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=7'>8</a> exchanges=['BINANCE'],
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=8'>9</a> )
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=9'>10</a> trades.set_index('received_time', inplace = True)
<a href='vscode-notebook-cell:/Users/dekahalane/Price_Prediction_LOB/code/crypto_lake.ipynb#W1sZmlsZQ%3D%3D?line=10'>11</a> trades
File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:144, in load_data(table, start, end, symbols, exchanges, bucket, boto3_session, use_threads, columns, row_slice, drop_partition_cols, cached)
141 assert exchanges[0].upper() == exchanges[0]
143 if method == 'cloudfront':
--> 144 df = _load_data_cloudfront(
145 table = table, start = start, end = end, symbols = symbols, exchanges = exchanges,
146 boto3_session = boto3_session, use_threads = use_threads, username = username, cached = cached,
147 )
148 else:
149 with botocache_context(
150 cache=cache,
151 action_regex_to_cache=[\"List.*\"],
(...)
155 supress_warning_message=is_anonymous_access,
156 ):
File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:275, in _load_data_cloudfront(table, start, end, symbols, exchanges, boto3_session, use_threads, username, cached)
258 boto3_session = boto3.Session(region_name=\"eu-west-1\")
260 # credentials = boto3_session.get_credentials().get_frozen_credentials()
261 # auth = AWSRequestsAuth(
262 # aws_access_key=credentials.access_key,
(...)
273
274 # if not exchanges or not symbols or not start or not end:
--> 275 available_data = list_data(table = table, start = start, end = end, symbols = symbols, exchanges = exchanges, boto3_session = boto3_session)
276 df = pd.DataFrame(available_data)
277 if not exchanges:
File /opt/homebrew/lib/python3.11/site-packages/lakeapi/main.py:416, in list_data(table, start, end, symbols, exchanges, bucket, boto3_session, last_modified_begin, last_modified_end)
414 paths = lakeapi._read_parquet._apply_partition_filter(path_root=path_root, paths=paths, filter_func=partition_filter)
415 if len(paths) < 1:
--> 416 raise lakeapi.exceptions.NoFilesFound(f\"No files Found on: {path}.\")
417 return [_path_to_dict(path) for path in paths]
NoFilesFound: No files Found on: s3://sample.crypto.lake/open_interest/."
}
from lake-api.
leftys
commented on June 18, 2024
Maybe you didn't restart the notebook so that the use_sample_data call is still in effect? Because I still see sample.crypto.lake being accessed.
from lake-api.
leftys
commented on June 18, 2024
I see you already got it working, so closing this.
from lake-api.
Related Issues (6)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lake-api.