unable to pass s3 file location in fastdup.run() about fastdup HOT 6 CLOSED

Hi @sameersharma00747 thanks for sharing this bug. We have just released a fix to this bug in v 0.134 let us know if this works for you?

from fastdup.

Still getting the same error. Is passing S3 URI working at your end? Is there any specific format in which I should pass it?

This is how I am running the command:
fastdup.run(input_dir='s3://bucket_name/sameer/', work_dir='out')

from fastdup.

Hi @sameersharma00747 can you make sure you upgraded using
python3.8 -m pip install -U --force-reinstall fastdup ?
Run with verbose=1 and send us the output.

from fastdup.

I am running this in a python notebook on databricks and the kernel is gettin killed after I do fastdup.run() with the s3 folder path.

I don't why but somehow it is killing the kernel midway.

Note: I am running this on python 3.9.

You can have a look at the verbose below in the image:

One more thing apart from this, is it possible to pass image embedding in place of the real images to perform all these computations using Fastdup? I saw something related to this this in the run mode code here: https://github.com/visualdatabase/fastdup/blob/main/fastdup/__init__.py

from fastdup.

HI @sameersharma00747 I think this is a different error than the previous one which is good.
Can you run the command !aws s3 ls --recursive s3://data-science-meesho-dev/sameer on a the jupyter notebook and let me know if this works. If it works, run the rest of the command as printed above until the > out/tmp/files.txt and let me know if it works (run it from the same path you were running before).

It is possible to run fastdup on your embedding as instructed here: https://github.com/visualdatabase/fastdup/blob/main/RUN.md#external

from fastdup.

Fixed on v. 0.137, closing

from fastdup.