Giter VIP home page Giter VIP logo

Comments (6)

colinmarc avatar colinmarc commented on July 28, 2024

That should work - including the tab complete! if not it's a bug.

On Fri, Nov 4, 2016, 06:31 Tim Spriggs [email protected] wrote:

hdfs ls /path/to/file* and hdfs get /path/to/file* /local/dir/ would be
amazing.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#57, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAyKD_RSXr4I9qIvmpQMjeOcaZG_GZAqks5q6sMtgaJpZM4KpNMv
.

from hdfs.

imoverclocked avatar imoverclocked commented on July 28, 2024

Looks like ls works but get does not.

$ hdfs mkdir /test
$ hdfs touch /test/1
$ hdfs touch /test/2
$ hdfs touch /test/3
$ hdfs get /test/\* test/
stat /test/*: file does not exist

from hdfs.

colinmarc avatar colinmarc commented on July 28, 2024

Hm - that was intended at the time, but it does look like it diverges from the hadoop fs behavior. Thanks for the report!

from hdfs.

nlevitt avatar nlevitt commented on July 28, 2024

hdfs ls /nonexistent* gives me a listing of my hdfs user dir (in other words, it acts like hdfs ls with no args)

from hdfs.

colinmarc avatar colinmarc commented on July 28, 2024

Whoops, didn't intend to close this. c5e93df only fixes ls /nonexistent*.

from hdfs.

colinmarc avatar colinmarc commented on July 28, 2024

So, I did some poking at how this works. The hadoop fs behavior is super obnoxious. hadoop fs allows this:

$ hadoop fs -get
Usage: hadoop fs [generic options] -get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>
$ hadoop fs -get /_test/*.txt

But if I expand the glob myself, it doesn't work:

$ hadoop fs -get /_test/foo.txt /_test/mobydick.txt
get: `/_test/mobydick.txt': No such file or directory

In this case, localdst is optional, and it resolves the ambiguity (is the second text file the destination or another source?) by interpreting my glob as only a single argument.

I'm not willing to do that for this library, because it is confusing in a posix context where globs are expanded before being passed to the program. So the options are to only allow a single source, like I do now, or require dest. But if I require dest, then get *.txt suddenly starts failing with a confusing error message.

I'm going to keep this as-written, but feel free to pipe up if you think I'm wrong.

from hdfs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.