Not able install adlfs from conda channel about adlfs HOT 10 CLOSED

I believe you would need -c conda-forge -c defusco; however, you may want to stick with pip for now until this package is also on conda-forge.

from adlfs.

Hello @martindurant, with pip I am able to install (inside my conda environment):

https://pypi.org/project/adlfs/

However, the documentation (link) is not correct. For DataLakeGen 2, it says:

known_implementations['abfs'] = {'class': 'adlfs.AzureDatalakeFileSystem'}

However, based on the code, it should be:

known_implementations['abfs'] = {'class': 'adlfs.AzureBlobFileSystem'}

Also, there are curly braces at the end of the following lines which needs to be replaced. This is minor (However, copy pasting the code from the documentation doesn't work):

ddf = dd.read_csv('abfs://{CONTAINER}/{FOLDER}/*.csv', storage_options=STORAGE_OPTIONS}
ddf = dd.read_csv('abfs://{CONTAINER}/folder.parquet', storage_options=STORAGE_OPTIONS}

from adlfs.

You no longer need to modify known_implementations at all. Would appreciate a PR updating that and for the syntax you noticed.

from adlfs.

I've added the updated the README in the "blob_not_exist_exception" branch for the syntax, and corrected the {adlfs.AzureBlobFileSystem}

@martindurant -- It looks adlfs is added to the fsspec master branch, but its not currently in PyPi. Should we update the docs in adlfs now, or wait until it the next version of fsspec is released?

from adlfs.

I think it would do now, with a note to fsspec that we would appreciate a release soon

from adlfs.

I've added conditional imports to the README.md docs as a compromise. Appreciate the guidance @martindurant.

from adlfs.

Sorry about late reply. I was on holidays.

To confirm, I am able to read parquet file without the "known_implementations" now with adlfs 0.1.5.

from adlfs.

@martindurant -- I would like to setup up a release pipeline for pypi and conda, but don't have permissions in Azure Pipelines or the Github account to set up secrets. Is there a preferred method you would recommend?

from adlfs.

I've only done this sort of thing for TravisCI. What kind of permissions are you missing? If the conda package is built by conda-forge, then you don't need any further configuration, pushing a release to pypi will trigger the build bot to make and test a PR against the feedstock - but you still need to accept and merge that PR.

I have quite a few repos I control, but I've never released anything often enough for it to seem worthwhile to automate!

from adlfs.

Agree the release process isn't burdensome. I guess I just assumed the releases would be automated. I was thinking about replicating this release example, but would need to store secrets. Not a big deal.

from adlfs.