Comments (2)
tyler-sommer
commented on July 28, 2024
Hi there,
I'm not the project maintainer but I had some free time to look into this. @colinmarc feel free to add your thoughts.
This project uses protocol buffers to communicate with HDFS NameNodes and DataNodes. As of Hadoop 2.7, fsck functionality is not exposed via these protocol buffers.
However, you can invoke fsck from the hadoop HTTP api(ref), but I'm not sure if @colinmarc wants to support the HTTP api in addition to protobuffer api.
It might be easier to just use the HTTP api yourself. For example, assuming default HDFS settings with a namenode running at 127.0.0.1, you can invoke fsck like:
curl http://127.0.0.1:50070/fsck
from hdfs.
colinmarc
commented on July 28, 2024
If @tyler-sommer is right and there's no protobuf method (I don't see one either), that'd be add to include in this library.
I'm going to close this, but feel free to chime in with thoughts!
from hdfs.
Related Issues (20)
- Skip optimization on internal/block_reader.go can lead to incorrect data read from stream HOT 1
- Files written by this library are unreadable by clickhouse HOT 2
- Is it thread-safe that NamenodeConnection?
- could not find "github.com/colinmarc/hdfs/v2/protocol/hadoop_common" "github.com/colinmarc/hdfs/v2/security"
- support UserGroupInformation
- EC not working properly HOT 1
- Allow setting CreateFlag when creating files
- Namenodes not been filtered by fs.defaultFS
- the dns_canonicalize_hostname value not being respected
- panic: mkdir /demo: read tcp 192.168.10.1:57318->192.168.10.102:9870: wsarecv: An established connection was aborted by the software in your host machine.
- how to do setQuota
- Empty file after CopyToRemote HOT 1
- Readme suggestion: also show memory usage advantage HOT 2
- Why the return of func Read() with a FileReader is not consistent?
- Downloading the encrypted file HOT 2
- Libhdfs.so bindings for hdfs-go? HOT 1
- hadoop3 - write: broken pipe
- An error is reported when reading a file in the HAR package
- Support of hadoop delegation tokens with HADOOP_TOKEN_FILE_LOCATION env
- no available namenodes: SASL handshake: wrong Token ID. Expected 0504, was 607d HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hdfs.