The get command doesn't support multiple sources about hdfs HOT 6 CLOSED

That should work - including the tab complete! if not it's a bug.

On Fri, Nov 4, 2016, 06:31 Tim Spriggs [email protected] wrote:

hdfs ls /path/to/file* and hdfs get /path/to/file* /local/dir/ would be
amazing.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#57, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAyKD_RSXr4I9qIvmpQMjeOcaZG_GZAqks5q6sMtgaJpZM4KpNMv
.

from hdfs.

Looks like ls works but get does not.

$ hdfs mkdir /test
$ hdfs touch /test/1
$ hdfs touch /test/2
$ hdfs touch /test/3
$ hdfs get /test/\* test/
stat /test/*: file does not exist

from hdfs.

Hm - that was intended at the time, but it does look like it diverges from the hadoop fs behavior. Thanks for the report!

from hdfs.

hdfs ls /nonexistent* gives me a listing of my hdfs user dir (in other words, it acts like hdfs ls with no args)

from hdfs.

Whoops, didn't intend to close this. c5e93df only fixes ls /nonexistent*.

from hdfs.

So, I did some poking at how this works. The hadoop fs behavior is super obnoxious. hadoop fs allows this:

$ hadoop fs -get
Usage: hadoop fs [generic options] -get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>
$ hadoop fs -get /_test/*.txt

But if I expand the glob myself, it doesn't work:

$ hadoop fs -get /_test/foo.txt /_test/mobydick.txt
get: `/_test/mobydick.txt': No such file or directory

In this case, localdst is optional, and it resolves the ambiguity (is the second text file the destination or another source?) by interpreting my glob as only a single argument.

I'm not willing to do that for this library, because it is confusing in a posix context where globs are expanded before being passed to the program. So the options are to only allow a single source, like I do now, or require dest. But if I require dest, then get *.txt suddenly starts failing with a confusing error message.

I'm going to keep this as-written, but feel free to pipe up if you think I'm wrong.

from hdfs.