willscott / go-nfs Goto Github PK
View Code? Open in Web Editor NEWgolang NFSv3 server
License: Apache License 2.0
golang NFSv3 server
License: Apache License 2.0
[ 1811.830000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.830000] fsid 0:13: expected fileid 0x0, got 0x165b98be2e784cc9
[ 1811.870000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.870000] fsid 0:13: expected fileid 0x0, got 0x15e019a7babe4e62
[ 1811.900000] NFS: server 192.168.0.10 error: fileid changed
[ 1811.900000] fsid 0:13: expected fileid 0x0, got 0x8a54dff6858049b8
This was the result of mounting it, doing a tab complete, then an ls.
I'm trying to use this library, but even though I implemented ReadDir
, Stat
,Lstat
, Open
, and Openfile
, if I do ls
in the mount, I see nothing. I do see ReadDir
being called though.
Apparently there is an incompatibility between the linux kernel as a client and go-nfs as a server:
Steps to reproduce:
Script to reproduce the problem:
if test ! -d testdirs; then
mkdir -p testdirs
for ((i=0;i<1024;++i)); do
mkdir testdirs/glance_$(uuidgen)
done
fi
go run example/osview/main.go testdirs/ 12049 &
sudo mount -o port=12049,mountport=12049,nfsvers=3,noacl,tcp,sec=none -t nfs localhost:/ /mnt
ls /mnt
The last command will hang and needs to be interrupted.
Doing the same with ganesha.nfsd seems to work.
Looking with at a tcpdump shows, (now all identifier as tcpdump uses them), that the kernel is always sending with nfs.cookie3=256
a nfs.verifier
from a prior readdirplus (the one with entry 122-136). Then checksum then doesn't match, and NFS3_ERR_BAD_COOKIE is returned. And everything starts from the beginning.
I checked the file-handles and file-ids in the reply with entry 122-136, and do not match the one, so I cannot see any sensible reason why the kernel is behaving that way.
It looks to me that the answer from go-nfs is correct, but assuming compatibility is desired I raise the issue here, even if it is presumably a bug in the kernel (hash collision over the cookie id?).
How can I do above the problem...
help me~~~ thank you
My English is terrible, please forgive me...
To reproduce:
step 1 - get content
cd /home/git
git clone https://github.com/Tieqiang/dc-heacth
ls dc-heacth/src/main/java/com/dchealth/VO | wc -l
# expected output: 65
step 2 - run osview server
go run ./example/osview /home/git
step 3 - mount nfs and list dir
mkdir /tmp/git
mount -o port=50184,mountport=50184 -t nfs localhost:/ /tmp/git
ls /tmp/git/dc-heacth/src/main/java/com/dchealth/VO | wc -l
# actual output is 27, not the whole 65
After #25 was merged I'm no longer seeing stale cookie errors, but still not all pages are retrieved.
I narrowed this down to the size of the pack-files --- a small repo like this will successfully clone, but even a slightly larger repo (like this one) will fail (this repo has packfiles of size at least 4096
).
Everything works fine except editing Microsoft Office files(e.g. Word, Excel, PowerPoint) from mountpoint.
It can be edited if the file open directly from source folder, but when opening from mountpoint the Microsoft Office shows "readonly".
./example/osnfs
mount
Create directory, delete it, then do ls .
--- you will get Stale file handle
.
currently, the repo has only a NullAuthHelper
.
To support multi-user user cases, it would be useful to have a credential auth implementation to allow user-views of a mount.
ref: #17
When I use the osnfs example and try to chown a file from the client mount, I get an error it's not supported.
chown works
client error:
chown root hi.txt
chown: hi.txt: RPC struct is bad
server error:
No file for lookup of ._hi.txt
2021/03/03 13:26:51 call to 0x1125460 failed: No such file or directory
2021/03/03 13:26:51 request: RPC #3742974177 (nfs.SetAttr)
2021/03/03 13:26:51 call to 0x112edc0 failed: lchown /Users/source/repos/github.com/dummy/hi.txt: operation not permitted
When I list a mounted directory that contains symbolic links on macOS, I get Stale NFS file handle
errors. Bellow, *.app
are symbolic links to directories using their absolute paths.
pedro@Pedros-MacBook-Pro ~ % ls -@Oahl ~/Applications
total 2924
dr-xr-xr-x@ 5 root wheel - 160B 31 Dec 1969 .
com.apple.FinderInfo 32B
drwxr-x---+ 103 pedro staff - 3.2K 30 Dec 21:02 ..
-rw-rw-rw- 1 root wheel - 1.4M 30 Dec 21:35 .VolumeIcon.icns
-rw-rw-rw- 1 root wheel - 406B 30 Dec 21:35 ._.
ls: /Users/pedro/Applications/IntelliJ IDEA CE.app: Stale NFS file handle
lrwxr-xr-x 1 root wheel - 116B 31 Dec 1969 IntelliJ IDEA CE.app
ls: /Users/pedro/Applications/Sublime Text.app: Stale NFS file handle
lrwxr-xr-x 1 root wheel - 91B 31 Dec 1969 Sublime Text.app
ls: /Users/pedro/Applications/Visual Studio Code.app: Stale NFS file handle
lrwxr-xr-x 1 root wheel - 105B 31 Dec 1969 Visual Studio Code.app
pedro@Pedros-MacBook-Pro ~ % ls -@Oahl ~/Applications
ls: IntelliJ IDEA CE.app: Stale NFS file handle
ls: Sublime Text.app: Stale NFS file handle
ls: Visual Studio Code.app: Stale NFS file handle
total 2921
dr-xr-xr-x@ 5 root wheel - 160B 31 Dec 1969 .
com.apple.FinderInfo 32B
drwxr-x---+ 103 pedro staff - 3.2K 30 Dec 21:02 ..
-rw-rw-rw- 1 root wheel - 1.4M 30 Dec 21:38 .VolumeIcon.icns
-rw-rw-rw- 1 root wheel - 406B 30 Dec 21:38 ._.
ls: fts_read: Stale NFS file handle
I'm also getting this error message from the NFS client: nfs loadattrcache vnode changed type, was 5 now 2
. Meaning that the vnode type was VLNK
and now is VDIR
.
I've added some logging, and my FS implementation is indeed returning different vnode types for those files. But that's because the NFS library is calling billy.Filesystem#ReadDir
, that "If the entry denotes a symbolic link, Info reports the information about the link itself, not the link's target.", and then it calls billy.Filesystem#Stat
instead of billy.Filesystem#Lstat
. So the vnode type will always change.
This patch addressed my problem: t0rr3sp3dr0@7d4f829
But I'm not sure if changing the implementation of tryStat
to always use billy.Filesystem#Lstat
is something we can do. I think we need a tryLstat
and move to this new function on a case-by-case basis.
When call onReadDirPlus
with empty dir, no errors output on server, but nfs client get error RPC struct is bad
.
mount -o port=1234,mountport=1234,nfsvers=3,noacl,tcp -t nfs localhost:/ /mount
cd /mount
mkdir test
f, _ := os.Open("/mount/test")
defer f.Close()
names, err := f.Readdirnames(-1) // error here: RPC struct is bad
if err != nil {
panic(err)
}
log.Printf("%+v", names)
Hello,I clone go-nfs on windows 10,then execute go run ./example/osview .
,but it failed,the errors as follows:
$ go run ./example/osview .
go: downloading github.com/willscott/memphis v0.0.0-20201122065000-f2beb41b6be3
go: downloading github.com/go-git/go-billy/v5 v5.3.1
go: downloading github.com/hashicorp/golang-lru v0.5.4
go: downloading github.com/rasky/go-xdr v0.0.0-20170124162913-1a41d1a06c93
go: downloading github.com/willscott/go-nfs-client v0.0.0-20200605172546-271fa9065b33
go: downloading github.com/polydawn/rio v0.0.0-20201122020833-6192319df581
go: downloading github.com/warpfork/go-errcat v0.0.0-20180917083543-335044ffc86e
go: downloading github.com/polydawn/go-timeless-api v0.0.0-20201121022836-7399661094a6
go: downloading github.com/polydawn/refmt v0.0.0-20190807091052-3d65705ee9f1
# github.com/willscott/memphis
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:20:31: undefined:
syscall.Stat_t
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:23:21: undefined:
createTime
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:57:30: undefined:
syscall.Stat_t
C:\Users\LENOVO\go\pkg\mod\github.com\willscott\[email protected]\deferred.go:60:18: undefined:
createTime
$ go run example/osview/main.go .
helpers/cachinghandler.go:112:31: binary.BigEndian.AppendUint64 undefined (type binary.bigEndian has no field or method AppendUint64)
$ go version
go version go1.18.1 linux/amd64
I'm trying to write an overlay-fs system using NFS --- while I can list the attributes of device files, trying to read from e.g "/dev/random" gives me "No such device or address".
Hi
I am getting:
go: github.com/willscott/[email protected] requires
github.com/willscott/[email protected] requires
go.polydawn.net/[email protected]: unrecognized import path "go.polydawn.net/go-timeless-api": https fetch: Get "https://go.polydawn.net/go-timeless-api?go-get=1": dial tcp 107.170.61.58:443: connect: connection refused
Probably go.polydawn.net/*
needs to be replaced with github.com/polydawn/*
When calling ls
on a file in a mounted directory, files that are symlinks show up as their underlying files. This is because ls
corresponds to a Lstat call (which doesn't follow symlinks), while the NFS server calls Stat.
Running the osnfs example and writing to a file in the mounted directory leads to an error "RPC struct is bad"
this is on macOS big sur. There's many errors in the logs of osnfs but I don't know which are applicable.
BTW, thanks for the awesome lib!
There is a bug on this line: https://github.com/willscott/go-nfs/blob/master/nfs_onreaddirplus.go#L73
handle := userHandle.ToHandle(fs, append(p, c.Name()))
This code is called in a loop, looping over the result from ReadDir()
. The problem here is the usage of append()
p
is a []string
that comes from converting the handle to a previously stored []string
.
Each time the loop runs, append(p, ...)
is called to either:
p
copied...
into existing p
slice because it's big enough, return a slice that points to same memory as p
but a bigger lengthIf we happened to run into the second case, what happens is:
When a NFS client list the directory, it gets 2 different handles, but they read the same file.
I originally thought this was due to a socket (hence #90), but I tested it with unionfs-fuse
, which also doesn't work with sockets (presumably for the same reason), and it works fine. Comparing the straces, it seems the problem occurs at a specific munmap
, which corresponds to /usr/lib/libresolv.so.2
.
This library currently makes use of the billy v5 filesystem interface as the expected content that the driving program passes it to serve over NFS.
This interface is imperfect, although that is mostly due to rough edges in golang itself - fs.FileInfo
's Sys
as an escape hatch to system specific stat structs containing owner/group information on unix permissioned files is awkward to say the least, and the billy interface does not integrate the Change
interface particularly well, making the requirements to support a read-write filesystem for this library non-obvious.
Golang 1.16 introduces the io/fs interface pattern for thinking about embedded files and filesystem. This uses a similar interface pattern as billy's Change, for interface-based detection of optional filesystem features. The initial 1.16 interfaces do not include any of the write or mutation portion of filesystems, though there is some indication that we may expect that to come eventually. They also do not provide a mechanism for resolving symlinks (no lstat
or Readlink
).
We should decide and then support either a io/fs.FS
-> billy.Filesystem
shim or native io/fs.FS
.
server:
go run ./example/helloworld
Server running at [::]:19815
client:
sudo mount -o port=19815,vers=3,tcp,noacl -t nfs localhost:/ /mnt/nfs -v
mount.nfs: timeout set for Fri Jul 2 13:45:04 2021
mount.nfs: trying text-based options 'port=19815,vers=3,tcp,noacl,addr=127.0.0.1'
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: portmap query failed: RPC: Program not registered
mount.nfs: requested NFS version or transport protocol is not supported
generate stable/unique fileids for files for cases where the underlying filesystem info doesn't expose these.
remaining work after #88
readdirplus correctly sets the fileid, but getattr does not. The result is that on gettattr, linux sees a filehandle of zero, and it all goes downhill from there.
The fix is simple, but it appears I can not create a PR yet. So here it is.
From 60a82cddc7fc8ab58886e67da169958efac210f8 Mon Sep 17 00:00:00 2001
From: "Ronald G. Minnich" [email protected]
Date: Tue, 12 Dec 2023 03:40:17 +0000
Subject: [PATCH] getattr: set the fileid in the returned attrs
Currently, the attr is not set, as it is in readdirplus.
The result is that, after a readdirplus, Linux
detects the fhandle has changed, and interprets that
as an error.
Further, the next use of this file will use an fhandle of all zeros,
causing the server to complain.
nfs_ongetattr.go | 2 ++
1 file changed, 2 insertions(+)
diff --git a/nfs_ongetattr.go b/nfs_ongetattr.go
index bf3d68f..c96cfa9 100644
--- a/nfs_ongetattr.go
+++ b/nfs_ongetattr.go
@@ -3,6 +3,7 @@ package nfs
import (
"bytes"
"context"
"encoding/binary"
"os"
"github.com/willscott/go-nfs-client/nfs/xdr"
@@ -27,6 +28,7 @@ func onGetAttr(ctx context.Context, w *response, userHandle Handler) error {
return &NFSStatusError{NFSStatusIO, err}
}
attr := ToFileAttribute(info)
attr.Fileid = binary.BigEndian.Uint64(handle[0:8])
writer := bytes.NewBuffer([]byte{})
if err := xdr.Write(writer, uint32(NFSStatusOk)); err != nil {
--
2.34.1
Easy to reproduce like:
for i in range(1,100):
os.makedirs(f'broken_catalog/sub_catalog_{i:03}', exist_ok=True)
broken_catalog
content will be empty for a NFS client, a similar issue exists for many files in one catalog.
.\file.go:114:27: undefined: syscall.Stat_t
In nfs_oncreate.go
, tryStat
is called on the appended path
and file.Name()
. However, unlike with Stat
, the Name()
is the full path, not just the basename. To fix this, just use newFile
. However, I haven't run into any problems stemming from this; but it should probably be fixed.
Currently, the server only sends a generic I/O Error
, which clients (at least the Linux client, anyway) interpret as Permission denied
.
Anyone had any success mounting this server on a Windows machine? I'm not sure if the problem is me not mounting in the right way, or some capability missing in the server.
I'm hosting the server on the same machine, tried to mount it using Powershell New-PSDrive
and the more native mount.exe
, but nothing worked, without meaningful error messages or server logs.
currently it looks for a looked up filename by listing the directory. it would be more efficient to directly stat that file.
In 2b12fc2 the go.mod
go version was updated to go 1.21
. This forces all users of this package to be using go 1.21
too.
So when I do this I get
$ go get github.com/willscott/go-nfs
go: upgraded go 1.19 => 1.21 // this changes the go statement in my go.mod
go: added toolchain go1.21.3
go: upgraded github.com/willscott/go-nfs v0.0.0-20230823072803-2b8e63b4d81f => v0.0.0-20231024182844-ccc7143570a1
This is undesirable for me as I wish to support go1.19, go1.20 and go1.21 as many distributions have older go version.
I will send a PR changing this back to 1.19 in a moment
Thank you
there needs to be at least a single directory traversal of valid handles for nfs to make progress - if every new handle immediately invalidates the previous, then you can't simultaneously talk about both a directory and a file in it.
the caching handler should be very clear it's in an invalid state when it has only 1 or 2 items of cache size.
In some cases (eg, when cloning a moderately big repository, like this one), data written to a file can not be read back in a pread
call (ie, will return 0 bytes).
I use the osnfs example code to setup a nfs server on windows 10.
On a linux device, i mount the server path. But the bin files in the directory have no execute permission.
ls -l shows -rw-rw-rw- only.
What should i to fix this?
Hey @willscott,
First of all, thank you for this awesome project.
I'm trying to containerize this project for my demos. I have it working but at this time it doesn't honor file permissions present on the filesystem.
On the readme you have this:
The billy abstraction layer does not extend to exposing uid and gid ownership of files. If ownership is important to your file system, you will need to ensure that the os.FileInfo meets additional constraints. In particular, the Sys() escape hatch is queried by this library, and if your file system populates a syscall.Stat_t concrete struct, the ownership specified in that object will be used.
I don't fully understand what you mean here, I believe my filesystem populates the uid and gid for every object, not sure what I'm doing wrong.
I'm using the example in example/osnfs
Thanks!
While this issue was discovered when trying to debug #91, this is not related to that issue.
If flock is run on a file in an NFS mount, the program hangs.
(Unlike #91, there's a reproducer)
#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <dlfcn.h>
int main(int argc, char** argv){
int fd=openat(AT_FDCWD, "test_mount/tmp/gopls-diff-stats-2599980011", O_RDWR);
printf("%s\n",strerror(errno));
flock(fd,LOCK_EX);
}
While the underlying file is safe, the view the server presents has the permissions with a lot of question marks.
While using go-nfs for github.com/u-root/sidecore, I see messages like this:
2023/12/29 20:47:15 [ERROR] No file for lookup of debconf-communicate
20:47:15 [ERROR] call to 0x102a2b490 failed: No such file or directory: file does not exist
I don't think this should be an error, or, at least, more information would be helpful. I'm still trying to see why this happens. Any ideas?
Thanks.
This is because I use string slices, which are not comparable.
I'm writing an overlayfs-like filesystem using NFS, but I've run into this issue. I'm not sure why it occurs.
Create socket with nc -lkU test.sock
, then mount server on directory and try to connect to it with mount/test.sock
--- you get Connection refused
.
This is on the same computer.
there are currently not tagged releases for this library. it's mature enough at this point that there probably should be
Problem
When dragging and dropping a file into the mount point in Finder, Finder returns the following error:
The operation can’t be completed because an unexpected error occurred (error code 100072).
Steps to Reproduce
mount -o port=60718,mountport=60718 -t nfs localhost:/mount Mount
open Mount
Logs
The logs at the time of the error show:
2020/09/29 10:47:15 request: RPC #804778388 (nfs.Create)
2020/09/29 10:47:15 call to 0x1131400 failed: xdr:Unmarshal: can't unmarshal to non-pointer '[8]uint8' - use & operator
Notes
The readme contains this code:
cacheHelper := nfshelper.NewCachingHandler(handler, 1)
However, this will cause an error like this:
[WARN] Caching handler created with insufficient cache to support directory listing%!(EXTRA string=size, int=1, string=verifiers, int=1)
Ignoring the malformed fmt string, this error makes sense because you need at least two entries for .
and ..
in a directory. However, if you bump the cache size from 1 to 2, you get empty output for all directories. All other usages of NewCachingHandler appear to specify 1024, and indeed, if you specify that, directory listing works... up to 1024 entries. Pagination still appears broken.
It seems that the started
and obj.Cookie
handling logic in onReadDirPlus
is incorrect, but I haven't fully debugged it yet.
When the directory has hundreds of files, we can't get all of those files.
mount -o port=1234,mountport=1234,nfsvers=3,noacl,tcp -t nfs localhost:/ /mount
cd /mount
for num in {1..100}
do
touch file-${num}.txt
done
ls
# just get part of 100 files
I'm running into an issue where if you mount libraries onto the server, you can't load them (get an error about undefined symbols, even though the content is the same with the files I'm passing to the server). After some debugging, I think I figured out the problem --- the server returns the inode number 0 for all files.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.