there needs to be at least a single directory traversal of valid handles for nfs to make progress - if every new handle immediately invalidates the previous, then you can't simultaneously talk about both a directory and a file in it.

the caching handler should be very clear it's in an invalid state when it has only 1 or 2 items of cache size.

While the underlying file is safe, the view the server presents has the permissions with a lot of question marks.

I'm trying to use this library, but even though I implemented ReadDir, Stat,Lstat, Open, and Openfile, if I do ls in the mount, I see nothing. I do see ReadDir being called though.

To reproduce:

step 1 - get content

cd /home/git
git clone https://github.com/Tieqiang/dc-heacth
ls dc-heacth/src/main/java/com/dchealth/VO | wc -l
# expected output:  65

step 2 - run osview server

go run ./example/osview /home/git

step 3 - mount nfs and list dir

mkdir /tmp/git
mount -o port=50184,mountport=50184 -t nfs localhost:/ /tmp/git
ls /tmp/git/dc-heacth/src/main/java/com/dchealth/VO | wc -l
# actual output is 27, not the whole 65

After #25 was merged I'm no longer seeing stale cookie errors, but still not all pages are retrieved.

While this issue was discovered when trying to debug #91, this is not related to that issue.

If flock is run on a file in an NFS mount, the program hangs.
(Unlike #91, there's a reproducer)

#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <dlfcn.h>
int main(int argc, char** argv){
	int fd=openat(AT_FDCWD, "test_mount/tmp/gopls-diff-stats-2599980011", O_RDWR);
	printf("%s\n",strerror(errno));
	flock(fd,LOCK_EX);
}

The readme contains this code:

	cacheHelper := nfshelper.NewCachingHandler(handler, 1)

However, this will cause an error like this:

[WARN] Caching handler created with insufficient cache to support directory listing%!(EXTRA string=size, int=1, string=verifiers, int=1)

Ignoring the malformed fmt string, this error makes sense because you need at least two entries for . and .. in a directory. However, if you bump the cache size from 1 to 2, you get empty output for all directories. All other usages of NewCachingHandler appear to specify 1024, and indeed, if you specify that, directory listing works... up to 1024 entries. Pagination still appears broken.

It seems that the started and obj.Cookie handling logic in onReadDirPlus is incorrect, but I haven't fully debugged it yet.

In some cases (eg, when cloning a moderately big repository, like this one), data written to a file can not be read back in a pread call (ie, will return 0 bytes).

When I use the osnfs example and try to chown a file from the client mount, I get an error it's not supported.

expected:

chown works

actual:

client error:

chown root hi.txt
chown: hi.txt: RPC struct is bad

server error:

No file for lookup of ._hi.txt
2021/03/03 13:26:51 call to 0x1125460 failed: No such file or directory
2021/03/03 13:26:51 request: RPC #3742974177 (nfs.SetAttr)
2021/03/03 13:26:51 call to 0x112edc0 failed: lchown /Users/source/repos/github.com/dummy/hi.txt: operation not permitted

I'm trying to write an overlay-fs system using NFS --- while I can list the attributes of device files, trying to read from e.g "/dev/random" gives me "No such device or address".

In 2b12fc2 the go.mod go version was updated to go 1.21. This forces all users of this package to be using go 1.21 too.

So when I do this I get

$ go get github.com/willscott/go-nfs
go: upgraded go 1.19 => 1.21  // this changes the go statement in my go.mod
go: added toolchain go1.21.3
go: upgraded github.com/willscott/go-nfs v0.0.0-20230823072803-2b8e63b4d81f => v0.0.0-20231024182844-ccc7143570a1

This is undesirable for me as I wish to support go1.19, go1.20 and go1.21 as many distributions have older go version.

I will send a PR changing this back to 1.19 in a moment

Thank you

This is because I use string slices, which are not comparable.

I'm running into an issue where if you mount libraries onto the server, you can't load them (get an error about undefined symbols, even though the content is the same with the files I'm passing to the server). After some debugging, I think I figured out the problem --- the server returns the inode number 0 for all files.

server:
go run ./example/helloworld

Server running at [::]:19815

client:
sudo mount -o port=19815,vers=3,tcp,noacl -t nfs localhost:/ /mnt/nfs -v

mount.nfs: timeout set for Fri Jul  2 13:45:04 2021
mount.nfs: trying text-based options 'port=19815,vers=3,tcp,noacl,addr=127.0.0.1'
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: portmap query failed: RPC: Program not registered
mount.nfs: requested NFS version or transport protocol is not supported

generate stable/unique fileids for files for cases where the underlying filesystem info doesn't expose these.

remaining work after #88

hi:
1 start example osview like this:
./osview /public 2045
2 use mount like this:
mount -t nfs -o port=2045,mountport=2045,nfsvers=3,tcp -o nolock 10.243.88.235:/public/ /share/
3 the output at server console:

2021/09/14 20:11:49 request: RPC #4228535329 (nfs.Null)
2021/09/14 20:11:49 request: RPC #4245312545 (nfs.GetAttr)
2021/09/14 20:11:49 call to 0x5dc6a0 failed: Invalid file handle

How can I do above the problem...
help me~~~ thank you
My English is terrible, please forgive me...

Anyone had any success mounting this server on a Windows machine? I'm not sure if the problem is me not mounting in the right way, or some capability missing in the server.
I'm hosting the server on the same machine, tried to mount it using Powershell New-PSDrive and the more native mount.exe, but nothing worked, without meaningful error messages or server logs.

When call onReadDirPlus with empty dir, no errors output on server, but nfs client get error RPC struct is bad.

mount -o port=1234,mountport=1234,nfsvers=3,noacl,tcp -t nfs localhost:/ /mount
cd /mount
mkdir test

f, _ := os.Open("/mount/test")
defer f.Close()
names, err := f.Readdirnames(-1) // error here: RPC struct is bad
if err != nil {
	panic(err)
}
log.Printf("%+v", names)

Easy to reproduce like:

for i in range(1,100):
    os.makedirs(f'broken_catalog/sub_catalog_{i:03}', exist_ok=True)

broken_catalog content will be empty for a NFS client, a similar issue exists for many files in one catalog.

When calling ls on a file in a mounted directory, files that are symlinks show up as their underlying files. This is because ls corresponds to a Lstat call (which doesn't follow symlinks), while the NFS server calls Stat.

I narrowed this down to the size of the pack-files --- a small repo like this will successfully clone, but even a slightly larger repo (like this one) will fail (this repo has packfiles of size at least 4096).

Create socket with nc -lkU test.sock, then mount server on directory and try to connect to it with mount/test.sock --- you get Connection refused.

This is on the same computer.

Hey @willscott,

First of all, thank you for this awesome project.

I'm trying to containerize this project for my demos. I have it working but at this time it doesn't honor file permissions present on the filesystem.

On the readme you have this:

The billy abstraction layer does not extend to exposing uid and gid ownership of files. If ownership is important to your file system, you will need to ensure that the os.FileInfo meets additional constraints. In particular, the Sys() escape hatch is queried by this library, and if your file system populates a syscall.Stat_t concrete struct, the ownership specified in that object will be used.

I don't fully understand what you mean here, I believe my filesystem populates the uid and gid for every object, not sure what I'm doing wrong.

I'm using the example in example/osnfs

Thanks!

When I list a mounted directory that contains symbolic links on macOS, I get Stale NFS file handle errors. Bellow, *.app are symbolic links to directories using their absolute paths.

pedro@Pedros-MacBook-Pro ~ % ls -@Oahl ~/Applications                                                                    
total 2924
dr-xr-xr-x@   5 root   wheel  -  160B 31 Dec  1969 .
	com.apple.FinderInfo	  32B 
drwxr-x---+ 103 pedro  staff  -  3.2K 30 Dec 21:02 ..
-rw-rw-rw-    1 root   wheel  -  1.4M 30 Dec 21:35 .VolumeIcon.icns
-rw-rw-rw-    1 root   wheel  -  406B 30 Dec 21:35 ._.

ls: /Users/pedro/Applications/IntelliJ IDEA CE.app: Stale NFS file handle
lrwxr-xr-x    1 root   wheel  -  116B 31 Dec  1969 IntelliJ IDEA CE.app

ls: /Users/pedro/Applications/Sublime Text.app: Stale NFS file handle
lrwxr-xr-x    1 root   wheel  -   91B 31 Dec  1969 Sublime Text.app

ls: /Users/pedro/Applications/Visual Studio Code.app: Stale NFS file handle
lrwxr-xr-x    1 root   wheel  -  105B 31 Dec  1969 Visual Studio Code.app

pedro@Pedros-MacBook-Pro ~ % ls -@Oahl ~/Applications
ls: IntelliJ IDEA CE.app: Stale NFS file handle
ls: Sublime Text.app: Stale NFS file handle
ls: Visual Studio Code.app: Stale NFS file handle
total 2921
dr-xr-xr-x@   5 root   wheel  -  160B 31 Dec  1969 .
	com.apple.FinderInfo	  32B 
drwxr-x---+ 103 pedro  staff  -  3.2K 30 Dec 21:02 ..
-rw-rw-rw-    1 root   wheel  -  1.4M 30 Dec 21:38 .VolumeIcon.icns
-rw-rw-rw-    1 root   wheel  -  406B 30 Dec 21:38 ._.
ls: fts_read: Stale NFS file handle

I'm also getting this error message from the NFS client: nfs loadattrcache vnode changed type, was 5 now 2. Meaning that the vnode type was VLNK and now is VDIR.

I've added some logging, and my FS implementation is indeed returning different vnode types for those files. But that's because the NFS library is calling billy.Filesystem#ReadDir, that "If the entry denotes a symbolic link, Info reports the information about the link itself, not the link's target.", and then it calls billy.Filesystem#Stat instead of billy.Filesystem#Lstat. So the vnode type will always change.

This patch addressed my problem: t0rr3sp3dr0@7d4f829

But I'm not sure if changing the implementation of tryStat to always use billy.Filesystem#Lstat is something we can do. I think we need a tryLstat and move to this new function on a case-by-case basis.

Hi

I am getting:

go: github.com/willscott/[email protected] requires
	github.com/willscott/[email protected] requires
	go.polydawn.net/[email protected]: unrecognized import path "go.polydawn.net/go-timeless-api": https fetch: Get "https://go.polydawn.net/go-timeless-api?go-get=1": dial tcp 107.170.61.58:443: connect: connection refused

Probably go.polydawn.net/* needs to be replaced with github.com/polydawn/*

Currently, the server only sends a generic I/O Error, which clients (at least the Linux client, anyway) interpret as Permission denied.

While using go-nfs for github.com/u-root/sidecore, I see messages like this:
2023/12/29 20:47:15 [ERROR] No file for lookup of debconf-communicate
20:47:15 [ERROR] call to 0x102a2b490 failed: No such file or directory: file does not exist

I don't think this should be an error, or, at least, more information would be helpful. I'm still trying to see why this happens. Any ideas?

Thanks.

readdirplus correctly sets the fileid, but getattr does not. The result is that on gettattr, linux sees a filehandle of zero, and it all goes downhill from there.

The fix is simple, but it appears I can not create a PR yet. So here it is.
From 60a82cddc7fc8ab58886e67da169958efac210f8 Mon Sep 17 00:00:00 2001
From: "Ronald G. Minnich" [email protected]
Date: Tue, 12 Dec 2023 03:40:17 +0000
Subject: [PATCH] getattr: set the fileid in the returned attrs

Currently, the attr is not set, as it is in readdirplus.
The result is that, after a readdirplus, Linux
detects the fhandle has changed, and interprets that
as an error.

Further, the next use of this file will use an fhandle of all zeros,
causing the server to complain.

Signed-off-by: Ronald G. Minnich [email protected]

nfs_ongetattr.go | 2 ++
1 file changed, 2 insertions(+)

diff --git a/nfs_ongetattr.go b/nfs_ongetattr.go
index bf3d68f..c96cfa9 100644
--- a/nfs_ongetattr.go
+++ b/nfs_ongetattr.go
@@ -3,6 +3,7 @@ package nfs
import (
"bytes"
"context"

• "encoding/binary"
  "os"

  "github.com/willscott/go-nfs-client/nfs/xdr"
  @@ -27,6 +28,7 @@ func onGetAttr(ctx context.Context, w *response, userHandle Handler) error {
  return &NFSStatusError{NFSStatusIO, err}
  }
  attr := ToFileAttribute(info)

• attr.Fileid = binary.BigEndian.Uint64(handle[0:8])

  writer := bytes.NewBuffer([]byte{})
  if err := xdr.Write(writer, uint32(NFSStatusOk)); err != nil {
  --
  2.34.1

I'm writing an overlayfs-like filesystem using NFS, but I've run into this issue. I'm not sure why it occurs.

Hello，I clone go-nfs on windows 10，then execute go run ./example/osview .，but it failed，the errors as follows:

$ go run ./example/osview .
go: downloading github.com/willscott/memphis v0.0.0-20201122065000-f2beb41b6be3
go: downloading github.com/go-git/go-billy/v5 v5.3.1
go: downloading github.com/hashicorp/golang-lru v0.5.4
go: downloading github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93
go: downloading github.com/willscott/go-nfs-client v0.0.0-20200605172546-271fa9065b33
go: downloading github.com/polydawn/rio v0.0.0-20201122020833-6192319df581
go: downloading github.com/warpfork/go-errcat v0.0.0-20180917083543-335044ffc86e
go: downloading github.com/polydawn/go-timeless-api v0.0.0-20201121022836-7399661094a6
go: downloading github.com/polydawn/refmt v0.0.0-20190807091052-3d65705ee9f1
# github.com/willscott/memphis
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:20:31: undefined:
syscall.Stat_t
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:23:21: undefined:
createTime
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:57:30: undefined:
syscall.Stat_t
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:60:18: undefined:
createTime

Running the osnfs example and writing to a file in the mounted directory leads to an error "RPC struct is bad"

this is on macOS big sur. There's many errors in the logs of osnfs but I don't know which are applicable.

BTW, thanks for the awesome lib!

github.com/willscott/go-nfs

.\file.go:114:27: undefined: syscall.Stat_t

I originally thought this was due to a socket (hence #90), but I tested it with unionfs-fuse, which also doesn't work with sockets (presumably for the same reason), and it works fine. Comparing the straces, it seems the problem occurs at a specific munmap, which corresponds to /usr/lib/libresolv.so.2.

$ go run example/osview/main.go .

github.com/willscott/go-nfs/helpers

helpers/cachinghandler.go:112:31: binary.BigEndian.AppendUint64 undefined (type binary.bigEndian has no field or method AppendUint64)

$ go version
go version go1.18.1 linux/amd64

currently it looks for a looked up filename by listing the directory. it would be more efficient to directly stat that file.

When the directory has hundreds of files, we can't get all of those files.

• mount and create files

mount -o port=1234,mountport=1234,nfsvers=3,noacl,tcp -t nfs localhost:/ /mount
cd /mount
for  num in {1..100}
do
    touch file-${num}.txt
done

• read dir files

ls
# just get part of 100 files

Everything works fine except editing Microsoft Office files(e.g. Word, Excel, PowerPoint) from mountpoint.
It can be edited if the file open directly from source folder, but when opening from mountpoint the Microsoft Office shows "readonly".

• NFS Server: macOS, ./example/osnfs
• NFS Client: macOS, mount

Create directory, delete it, then do ls . --- you will get Stale file handle.

I use the osnfs example code to setup a nfs server on windows 10.
On a linux device, i mount the server path. But the bin files in the directory have no execute permission.
ls -l shows -rw-rw-rw- only.
What should i to fix this?

This library currently makes use of the billy v5 filesystem interface as the expected content that the driving program passes it to serve over NFS.

This interface is imperfect, although that is mostly due to rough edges in golang itself - fs.FileInfo's Sys as an escape hatch to system specific stat structs containing owner/group information on unix permissioned files is awkward to say the least, and the billy interface does not integrate the Change interface particularly well, making the requirements to support a read-write filesystem for this library non-obvious.

Golang 1.16 introduces the io/fs interface pattern for thinking about embedded files and filesystem. This uses a similar interface pattern as billy's Change, for interface-based detection of optional filesystem features. The initial 1.16 interfaces do not include any of the write or mutation portion of filesystems, though there is some indication that we may expect that to come eventually. They also do not provide a mechanism for resolving symlinks (no lstat or Readlink).

We should decide and then support either a io/fs.FS -> billy.Filesystem shim or native io/fs.FS.

Apparently there is an incompatibility between the linux kernel as a client and go-nfs as a server:
Steps to reproduce:

Script to reproduce the problem:

if test ! -d testdirs; then
  mkdir -p testdirs
  for ((i=0;i<1024;++i)); do
    mkdir testdirs/glance_$(uuidgen)
  done
fi

go run example/osview/main.go  testdirs/ 12049 &
sudo mount -o port=12049,mountport=12049,nfsvers=3,noacl,tcp,sec=none -t nfs localhost:/  /mnt

ls /mnt

The last command will hang and needs to be interrupted.

Doing the same with ganesha.nfsd seems to work.

Looking with at a tcpdump shows, (now all identifier as tcpdump uses them), that the kernel is always sending with nfs.cookie3=256 a nfs.verifier from a prior readdirplus (the one with entry 122-136). Then checksum then doesn't match, and NFS3_ERR_BAD_COOKIE is returned. And everything starts from the beginning.

I checked the file-handles and file-ids in the reply with entry 122-136, and do not match the one, so I cannot see any sensible reason why the kernel is behaving that way.

It looks to me that the answer from go-nfs is correct, but assuming compatibility is desired I raise the issue here, even if it is presumably a bug in the kernel (hash collision over the cookie id?).

In nfs_oncreate.go, tryStat is called on the appended path and file.Name(). However, unlike with Stat, the Name() is the full path, not just the basename. To fix this, just use newFile. However, I haven't run into any problems stemming from this; but it should probably be fixed.

No metadata update after rename

Problem
When dragging and dropping a file into the mount point in Finder, Finder returns the following error:

The operation can’t be completed because an unexpected error occurred (error code 100072).

Steps to Reproduce

• Run the NFS server using one of the samples, e.g. osnfs
• Mount the share, like the following mount -o port=60718,mountport=60718 -t nfs localhost:/mount Mount
• Open the Finder window view of the mount open Mount
• Drag and drop a File from another Finder window into the Finder view of the NFS share
• Finder produces the error above. File is not copied

Logs
The logs at the time of the error show:

2020/09/29 10:47:15 request: RPC #804778388 (nfs.Create)
2020/09/29 10:47:15 call to 0x1131400 failed: xdr:Unmarshal: can't unmarshal to non-pointer '[8]uint8' - use & operator

Notes

• OS X Catalina 10.15.4 (19E287)

there are currently not tagged releases for this library. it's mature enough at this point that there probably should be

There is a bug on this line: https://github.com/willscott/go-nfs/blob/master/nfs_onreaddirplus.go#L73

handle := userHandle.ToHandle(fs, append(p, c.Name()))

This code is called in a loop, looping over the result from ReadDir(). The problem here is the usage of append()

p is a []string that comes from converting the handle to a previously stored []string.

Each time the loop runs, append(p, ...) is called to either:

1. create a new slice with p copied
2. simply putting ... into existing p slice because it's big enough, return a slice that points to same memory as p but a bigger length

If we happened to run into the second case, what happens is:

1. we create a new handle for a file path, add to handle cache
2. we proceed to create a second handle for a second file path, add to handle cache
3. but the second file path and the first file path points to the same memory, and now you have 2 different file handles (uuid in this implementation) pointing to the same path

When a NFS client list the directory, it gets 2 different handles, but they read the same file.

[ 1811.830000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.830000] fsid 0:13: expected fileid 0x0, got 0x165b98be2e784cc9
[ 1811.870000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.870000] fsid 0:13: expected fileid 0x0, got 0x15e019a7babe4e62
[ 1811.900000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.900000] fsid 0:13: expected fileid 0x0, got 0x8a54dff6858049b8

This was the result of mounting it, doing a tab complete, then an ls.

currently, the repo has only a NullAuthHelper.

To support multi-user user cases, it would be useful to have a credential auth implementation to allow user-views of a mount.

ref: #17