Make it possible to interact with external data sources about dask-sql HOT 6 CLOSED

It is now (after #55) possible to create new tables by reading in data, e.g. with

CREATE TABLE
        "nyc"
    WITH (
        format = 'csv',
        location = 'https://support.staffbase.com/hc/en-us/article_attachments/360009197031/username.csv',
        sep = ';'
    )

However, it is not clear so far if this suits all the needs (e.g. S3). Additionally, an interaction with hive would be interesting.

from dask-sql.

BlazingDB/blazingsql#937 may also be relevant

from dask-sql.

Just a small update: interaction with hive is implemented as a experimetal feature in 0.2.0 (#63).
And as fsspec/adlfs#111 was implemented, it should also be possible to use az:// as input format (although untested)

from dask-sql.

it should also be possible to use az:// as input format (although untested)

It'll be worth testing with a public repo

storage_options = {'account_name': 'azureopendatastorage'}
ddf = dd.read_parquet('az://nyctlc/green/puYear=2019/puMonth=*/*.parquet', storage_options=storage_options)

from dask-sql.

see #84

from dask-sql.

Closing this issue now, as we have already integrations included and for the specific new types of integration, be better create a new issue (e.g. like #83)

from dask-sql.