`history merge` has mmap issues with certain remote filesystems about fish-shell HOT 8 OPEN

It seems we should do

diff --git a/src/path.rs b/src/path.rs
index 92ecefd1a..97a1b8ff8 100644
--- a/src/path.rs
+++ b/src/path.rs
@@ -660,7 +660,7 @@ fn path_remoteness(path: &wstr) -> DirRemoteness {
                 => DirRemoteness::remote,
             _ => {
                 // Other FSes are assumed local.
-                DirRemoteness::local
+                DirRemoteness::unknown
             }
         }
     }

We'll still use flock for unknown remoteness but won't mmap the file.

We can still return DirRemoteness::local for some most file systems.
Or maybe we can check if mmap fails somehow.. I guess no.

from fish-shell.

Um, I think I know where the problem lies now. This switch-case:

does not correctly handle my storage type, which leads to subsequent use of mmap. Our storage returns a special f_type, which is not included in the enumerated data.

from fish-shell.

great work, can you share the file type magic number? Or the relevant line from /proc/mounts?
Perhaps we can find a better approach.
Does df -l path/to/your/file work? Their heuristic is here: https://github.com/coreutils/gnulib/blob/master/lib/mountlist.c#L232-L236

from fish-shell.

great work, can you share the file type magic number? Or the relevant line from /proc/mounts? Perhaps we can find a better approach. Does df -l path/to/your/file work? Their heuristic is here: https://github.com/coreutils/gnulib/blob/master/lib/mountlist.c#L232-L236

The type of the magic num is fea36969, which is extracted by:

#include <iostream>
#include <sys/statfs.h>

int main() {
    const char* path = "/mnt/hwfile/"; 

    struct statfs fs_info;

    statfs(path, &fs_info);
    unsigned int fs = static_cast<unsigned int>(fs_info.f_type);
    std::cout << std::hex << fs << std::endl;

    return 0;
}

The result of df -lh path/to/your/file is:

Filesystem      Size  Used Avail Use% Mounted on
/hwfile01       850T  755T   96T  89% /mnt/hwfile

Hmm, it seems quite challenging to differentiate remote file systems through simple operations, as various providers' file systems don't adhere to strict standards (perhaps innovation?). I think it would be simpler if Fish could offer a toggle to control whether to enable mmap, or allow users to configure whether it belongs to a remote FS.

from fish-shell.

But still, if there are meaningful difference between remote and local filesystems, I don't think applications should be required to list all possible local filesystems.

I'm not sure if we want to stop using mmap.
(I'm also not sure what's the problem with using it on NFS.)

Maybe we can detect that mmap is failing and then fall back to reading the file?
How does the problem manifest for you? Any reproducer? If mmap returns MAP_FAILED, we could simply do

diff --git a/src/history/file.rs b/src/history/file.rs
index 489bc83cc..f63783277 100644
--- a/src/history/file.rs
+++ b/src/history/file.rs
@@ -121,7 +121,7 @@ impl HistoryFileContents {
         let len: usize = file.seek(SeekFrom::End(0)).ok()?.try_into().ok()?;
         let mmap_file_directly = should_mmap();
         let mut region = if mmap_file_directly {
-            MmapRegion::map_file(file.as_raw_fd(), len)
+            MmapRegion::map_file(file.as_raw_fd(), len).or_else(|| MmapRegion::map_anon(len))
         } else {
             MmapRegion::map_anon(len)
         }?;

I don't know how else it could fail, you don't get garbage data, right?

from fish-shell.