Comments (4)
@adammoody suggested a possible printf() format string could be passed to dwalk (or dfind) to format text output something like + does for date command. That would be great! Currently, my main use-case requires both atime and mtime for each file's path given in the text output, ideally in %s (epoch seconds) format. File size in bytes would be great too. Thanks!
from mpifileutils.
Until we have the more general solution, which will likely be a while, it's probably easiest to hack the existing format to suit your needs. You'd want to modify the lines in src/common/mfu_flist_io.c
here:
mpifileutils/src/common/mfu_flist_io.c
Lines 1643 to 1646 in 4791815
For example, the following patch:
diff --git a/src/common/mfu_flist_io.c b/src/common/mfu_flist_io.c
index 0b0a2e5..285afd5 100644
--- a/src/common/mfu_flist_io.c
+++ b/src/common/mfu_flist_io.c
@@ -22,6 +22,9 @@
#include <errno.h>
#include <string.h>
+/* define PRI64 */
+#include <inttypes.h>
+
#include "dtcmp.h"
#include "mfu.h"
#include "mfu_flist_internal.h"
@@ -1640,9 +1643,9 @@ static size_t print_file_text(mfu_flist flist, uint64_t idx, char* buffer, size_
const char* size_units;
mfu_format_bytes(size, &size_tmp, &size_units);
- numbytes = snprintf(buffer, bufsize, "%s %s %s %7.3f %3s %s %s\n",
+ numbytes = snprintf(buffer, bufsize, "%s %s %s %7.3f %3s %" PRIu64 " %" PRIu64 " %" PRIu64 " %s\n",
mode_format, username, groupname,
- size_tmp, size_units, modify_s, file
+ size_tmp, size_units, size, acc, mod, file
);
}
else {
changes dwalk --text --output list.txt /path
lines to print file size, atime, mtime as integers immediately following the human readable file size still shown in floating point with units. So rather than the current format of:
drwxrwx--- user1 user1 4.000 KiB Sep 22 2023 17:08 /path
-rw------- user1 user1 854.000 B Sep 22 2023 17:08 /path/CMakeLists.txt
drwx------ user1 user1 4.000 KiB Sep 22 2023 17:08 /path/daos-serialize
it prints as:
drwxrwx--- user1 user1 4.000 KiB 4096 1696015833 1695427689 /path
-rw------- user1 user1 854.000 B 854 1696016217 1695427689 /path/CMakeLists.txt
drwx------ user1 user1 4.000 KiB 4096 1696016421 1695427689 /path/daos-serialize
from mpifileutils.
That worked great! I also took out the human readable size and commented out the lines to format the size and times. Speeds up the text file output time. Final size of text file is about the same. Best part, my post processing now has atime information AND is ~5x faster! Thank you! :)
from mpifileutils.
Great! Wow, it's surprising that the string formatting adds so much overhead, but that's also good to know about. Good idea to try that.
from mpifileutils.
Related Issues (20)
- Feature Request: dchmod: Add ability to set the SETUID/SETGID bits
- fiemap error HOT 1
- RHEL8 build HOT 1
- Export of inconsistent internal headers HOT 1
- dsync generates MPI error when I'm not the owner of the source path
- segfault crash when using --sort opiton for dwalk HOT 20
- [Request] add support for project id (i.e. lsattr -p) HOT 16
- [Request] add access time to dwalk output for text format HOT 1
- ddup warnings are missing new lines HOT 1
- [Request] run ddup without changing atime HOT 13
- New Release HOT 2
- dsync not updating times correctly HOT 87
- [Request] can dwalk "stream" text output as it reads mfu file (to avoid high RAM usge)? HOT 3
- POS36-C. Observe correct revocation order while relinquishing privileges HOT 5
- ddup support for mulitple paths?
- ddup size limit filter?
- ddup output file option
- ddup file size in output
- mknod() failed (errno=2 No such file or directory) HOT 17
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mpifileutils.