Comments (6)
That should work - including the tab complete! if not it's a bug.
On Fri, Nov 4, 2016, 06:31 Tim Spriggs [email protected] wrote:
hdfs ls /path/to/file* and hdfs get /path/to/file* /local/dir/ would be
amazing.—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#57, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAyKD_RSXr4I9qIvmpQMjeOcaZG_GZAqks5q6sMtgaJpZM4KpNMv
.
from hdfs.
Looks like ls works but get does not.
$ hdfs mkdir /test
$ hdfs touch /test/1
$ hdfs touch /test/2
$ hdfs touch /test/3
$ hdfs get /test/\* test/
stat /test/*: file does not exist
from hdfs.
Hm - that was intended at the time, but it does look like it diverges from the hadoop fs
behavior. Thanks for the report!
from hdfs.
hdfs ls /nonexistent*
gives me a listing of my hdfs user dir (in other words, it acts like hdfs ls
with no args)
from hdfs.
Whoops, didn't intend to close this. c5e93df only fixes ls /nonexistent*
.
from hdfs.
So, I did some poking at how this works. The hadoop fs
behavior is super obnoxious. hadoop fs
allows this:
$ hadoop fs -get
Usage: hadoop fs [generic options] -get [-p] [-ignoreCrc] [-crc] <src> ... <localdst>
$ hadoop fs -get /_test/*.txt
But if I expand the glob myself, it doesn't work:
$ hadoop fs -get /_test/foo.txt /_test/mobydick.txt
get: `/_test/mobydick.txt': No such file or directory
In this case, localdst
is optional, and it resolves the ambiguity (is the second text file the destination or another source?) by interpreting my glob as only a single argument.
I'm not willing to do that for this library, because it is confusing in a posix context where globs are expanded before being passed to the program. So the options are to only allow a single source, like I do now, or require dest. But if I require dest, then get *.txt
suddenly starts failing with a confusing error message.
I'm going to keep this as-written, but feel free to pipe up if you think I'm wrong.
from hdfs.
Related Issues (20)
- Can I set more configuration when access hdfs?
- Skip optimization on internal/block_reader.go can lead to incorrect data read from stream HOT 1
- Files written by this library are unreadable by clickhouse HOT 2
- Is it thread-safe that NamenodeConnection?
- could not find "github.com/colinmarc/hdfs/v2/protocol/hadoop_common" "github.com/colinmarc/hdfs/v2/security"
- support UserGroupInformation
- EC not working properly HOT 1
- Allow setting CreateFlag when creating files
- Namenodes not been filtered by fs.defaultFS
- the dns_canonicalize_hostname value not being respected
- panic: mkdir /demo: read tcp 192.168.10.1:57318->192.168.10.102:9870: wsarecv: An established connection was aborted by the software in your host machine.
- how to do setQuota
- Empty file after CopyToRemote HOT 1
- Readme suggestion: also show memory usage advantage HOT 2
- Why the return of func Read() with a FileReader is not consistent?
- Downloading the encrypted file HOT 2
- Libhdfs.so bindings for hdfs-go? HOT 1
- hadoop3 - write: broken pipe
- An error is reported when reading a file in the HAR package
- Support of hadoop delegation tokens with HADOOP_TOKEN_FILE_LOCATION env
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hdfs.