Giter VIP home page Giter VIP logo

p9's Introduction

p9

CircleCI Go Report Card GoDoc

p9 is a Golang 9P2000.L client and server originally written for gVisor. p9 supports Windows, BSD, and Linux on most Go-available architectures.

Server Example

For how to start a server given a p9.Attacher implementation, see cmd/p9ufs.

For how to implement a p9.Attacher and p9.File, see as an example staticfs, a simple static file system. Boilerplate templates for p9.File implementations are in templatefs.

A test suite for server-side p9.Attacher and p9.File implementations is being built at fsimpl/test.

Client Example

import (
    "log"
    "net"

    "github.com/hugelgupf/p9/p9"
)

func main() {
  conn, err := net.Dial("tcp", "localhost:8000")
  if err != nil {
    log.Fatal(err)
  }

  // conn can be any net.Conn.
  client, err := p9.NewClient(conn)
  if err != nil {
    log.Fatal(err)
  }

  // root will be a p9.File and supports all those operations.
  root, err := client.Attach("/")
  if err != nil {
    log.Fatal(err)
  }

  // For example:
  _, _, attrs, err := root.GetAttr(p9.AttrMaskAll)
  if err != nil {
    log.Fatal(err)
  }

  log.Printf("Attrs of /: %v", attrs)
}

p9's People

Contributors

ayushr2 avatar dependabot[bot] avatar djdv avatar fhs avatar hugelgupf avatar lencerf avatar nixprime avatar nlacasse avatar nmeum avatar prattmic avatar rminnich avatar sevki avatar stickler-ci avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

p9's Issues

Make a release 0.2.0 to break u-root dependency

Right now repos that are using this pkg and pull it in via go get github.com/hugelgupf/p9 automatically pull 0.1.0 which still has the u-root dependency. This can be fixed by releasing 0.2.0

gVisor extensions: yay or nay?

p9 code base currently has several gVisor protocol extensions (9P2000.L.Google.N, where N is some monotonically increasing number denoting support for ).

This support is only exercised between the client and server present in this code base. It would likely also have worked with gVisor before gVisor removed 9P support (I think it's all lisafs only now, on the public side).

Is it worth keeping some of the extensions?

  • T/Rwalkgetattr messages -- these were added for performance reasons, one of our first extension. Every Walk was always followed by a GetAttr, saving one roundtrip.
  • Creations with UID in addition to GID specified -- Tucreate, Tusymlink, Tumkdir, Tumknod.

T/Rwalkgetattr is mainly a bit annoying for server implementers at the moment, though p9.DefaultWalkGetAttr provides some relief. I think on the creation side, we currently do not handle UIDs or GIDs correctly for 9P2000.L at all, in any sense of the word. (Though removal of the extension may not necessarily help improve that situation.)

localfs does not implement unlinkat

As the title says, unless I'm reading and using the code wrong, unlinkat is not implemented. Therefore there is no way to remove anything from a server served with p9ufs.

If I'm not mistaken in this, I think I'll take a shot at implementing it, but I wonder why it's not there in the first place, and so second guess myself. Maybe I've misunderstood something.

Walk Server/Client interface inconsistency with 9P spec

There's a problem with how twalk is being handled that forces File implementations to have non-compliant Walk methods.

walk(5) says:

It is legal for nwname to be zero, in which case newfid will represent the same file as fid and the walk will usually succeed; this is equivalent to walking to dot.
The value of nwqid can-not be zero unless nwname is zero. Also, nwqid will always be less than or equal to nwname.

The Server handler for twalk internally requires that len(qids) returned from Walk(nil) is 1, despite the name count being 0.
It then discards those qids, returning nil itself.

p9/p9/handlers.go

Line 1129 in ede3efe

func (t *twalk) handle(cs *connState) message {

calls:

p9/p9/handlers.go

Line 1147 in ede3efe

qids, newRef, _, _, err := doWalk(cs, ref, t.Names, false)

calls:

p9/p9/handlers.go

Line 1041 in ede3efe

qids, sf, valid, attr, err = walkOne(nil, ref.file, nil, getattr)

checks length and only asserts count of 1 as valid (not 0):

p9/p9/handlers.go

Lines 1014 to 1017 in ede3efe

if len(localQIDs) != 1 {
// Expected a single QID.
sf.Close()
return nil, nil, AttrMask{}, Attr{}, linux.EINVAL

despite requiring len(qids) == 1 qids are dropped:

p9/p9/handlers.go

Lines 1072 to 1075 in ede3efe

// Do not return the new QID.
//
// TODO: why?
return nil, newRef, valid, attr, nil

This answers the "TODO: why?", but this requirement causes an inconsistency in returned values from the same File implementation. When called directly, a File will return [1]qid{...}, but when wrapped by Server the same arguments will return nil.
Implying that your implementation can either be compliant with Server's expectations, or compliant when called directly, but not both.
I.e. qids from Client.Walk(nil) and File.Walk(nil) will not be consistent despite being the same underlying implementation.

Support 9P errors with messages

EINVAL is about the most infuriating error message you can get.

In the 9P2000.L case, let's see if Linux will accept Rerror messages instead of Rlerror.

Otherwise, add a verbose error logging mode in which error messages are logged server-side.

internal error packages should be exported (or something)

tl;dr
Need to be able to use errors.Is(clientError, linux.Errno(someVal)) in code that uses Client,


Long version:

internal and internal/linux pkgs can't be used by calling code, which makes handling errors client-side difficult, especially on non-Linux.

Consider a server and client on a Windows host, and a client-side API which wants to wrap Create.

Server-side, Walk and Create methods seem expected to return the os.ErrNotExist and os.ErrExist values,
which get translated by clientFile internally via newErr/internal.ExtractErrno into linux.ENOENT and linux.EEXIST.

Client-side, whether you use Walk to check file existence before sending Create, or send Create alone; there's no easy way to distinguish which specific error you received.

The workaround I'm currently using requires copying the (protocol-level) error values into an accessible pkg, and then using the runtime to extract the underlying decimal values from Client errors for comparison. Which seems less than ideal.


Here's an example from a toy/prototype that might demonstrate this.
(I can bundle this whole thing up if needed, but it's just for experimenting/exercising this library right now so it's gross)

This errors pkg contains direct copies of the internal linux.Errno type and values.
goerrors is the standard errors pkg.
(errors is a bad name on my part and should probably be something like perrors for protocol errors or something)

func makeMessage(dir p9.File, filename string) (p9.File, error) {
	const flags = p9.WriteOnly
	wnames := []string{filename}
	_, messageFile, err := dir.Walk(wnames)
	if err != nil {
		err = runtimeHax(err)
		if !goerrors.Is(err, errors.ENOENT) {
			return nil, err
		}
		_, dirClone, err := dir.Walk(nil)
		if err != nil {
			return nil, err
		}
		messageFile, _, _, err := dirClone.Create(filename, flags,
			p9.ModeRegular|p9.AllPermissions, p9.NoUID, p9.NoGID)
		return messageFile, err
	}

	// TODO: check mode attr isregular
	// TODO: truncate/setattr on open

	if _, _, err := messageFile.Open(flags); err != nil {
		return nil, err
	}
	return messageFile, nil
}

func runtimeHax(err error) errors.Errno {
	// XXX: This works but should basically be considered illegal.
	// type notError struct{ l, h unsafe.Pointer }
	// return *(*errors.Errno)((*notError)(unsafe.Pointer(&err)).h)

	// XXX: Better, not best. Still not safe.
	// We'll panic if `err`'s underlying type changes from `uintptr`, and/or need more complicated handling.
	return errors.Errno(reflect.ValueOf(err).Uint())
}

It'd be ideal if the client could call errors.Is(err, linux.ENOENT) without needing err = runtimeHax(err).


As an aside, there's a subtle discrepancy here which may or may not matter.
Calling server methods directly will return host-native error values (os.ErrExist) while calling the method via Client return the protocol-native values (linux.EEXIST).

This makes sense and should probably be expected, but I'm mentioning it anyway since calling code may have to consider both host-native and protocol-level error values, despite wrapping the same methods.
Although it may be expected to always utilize Client even when in the same process(, using some memcopy-like transport).
And this is kind of out of scope of the problem.

In addition to this, server implementations can't return protocol-level error numbers because linux.Errno is not exported.
ExtractErrno will check for this type and pass it through to the client if err is one, but there's no way for a server implementation to actually return one. As a result returning something like myError(0x11) always becomes linux.EIO in Client methods.

Twalk returns Rlerror{Error: 16} while directory is still open

Hello, i am trying to make use of this library but have an issue getting it to work. Whenever i try to do an "ls" on the filesystem mounted via native v9fs on linux i will get the list of files but not any information on them. I noticed it will call Walk for itself but not for anything inside. The reason is likely because the directory is still opened after Readdir is called and the handler code will refuse before even trying to call Walk in this case. This happens with localfs and staticfs too and therefore i am quite unsure what to do about this.

Invalid message type: 52/53

I've been using the u-root/cpu command. I tried to do a "go build" while "cpu-ed" and get the following error message:

2022/01/30 22:45:26 unknown error: invalid message type: 52
go: RLock /home/xfk/src/cpu/go.mod: input/output error

or sometimes:

2022/01/30 22:45:26 unknown error: invalid message type: 53
go: RLock /home/xfk/src/cpu/go.mod: input/output error

I see those types are not implemented in this library which u-root/cpu uses. (I apologize if I reported the issue in the wrong place)

Handle O_NONBLOCK

O_NONBLOCK can be passed to any filesystem operation. This library should pass this flag through to the underlying implementation (or, at the very least, ignore it). As far as I can tell, it currently just returns an "invalid argument" error.

Note: This breaks ls.

Server prevents walks to ".."

intro(5) says

All directories must support walks to the directory .. (dot–dot) meaning parent directory, although by convention directories contain no explicit entry for .. or . (dot). The parent of the root directory of a server's tree is itself.

And walk(5) says

The name .. (dot–dot) represents the parent directory.
...
A walk of the name .. in the root directory of a server is equivalent to a walk with no name elements.

I don't see any mention of this in diod's protocol document, but I do see it used in their tests:
https://github.com/chaos/diod/blob/9da28f911978957dbec251c653200db7a4dcad6e/tests/user/testopenfid.c#L72

Which leads me to believe the same requirement for 9P, still holds true for 9P2000.L.

However, even if File implementations handle a dot-dot request inside their Walk method, Server explicitly forbids the request from being issued to the File.

p9/p9/handlers.go

Line 1165 in 49c780c

func (t *twalk) handle(cs *connState) message {

calls:

p9/p9/handlers.go

Line 1183 in 49c780c

qids, newRef, _, _, err := doWalk(cs, ref, t.Names, false)

calls:

p9/p9/handlers.go

Lines 1065 to 1070 in 49c780c

for _, name := range names {
err = checkSafeName(name)
if err != nil {
return
}
}

Bypassing the check for .. seems to work as expected (tested against cmd\p9ufs with Client; even trying to escape the root resolves to just . of p9ufs's -root argument).

for _, name := range names {
+	if name == ".." {
+		continue
+	}
	err = checkSafeName(name)
	if err != nil {
		return
	}
}

However, I'm not familiar enough with how the library tracks fid references, so I'm not sure if such a simple exception to the name rules somehow breaks reference handling in some way.
The logic around here is concerning

p9/p9/handlers.go

Line 1146 in 49c780c

walkRef.pathNode.addChild(newRef, names[i])

since newRef should actually be walkRef's parent or walkRef itself if walkRef is the root; but we add newRef as a child of walkRef.

doWalk currently handles the case for clones (nil names), and for steps (some string name).
But if the logic for stepping isn't also valid for backtracking (".." names), support for that may have to be added.
I'm not sure.

Regression in `ReadAt` causes 0'd payloads

78606c4 causes a regression in ReadAt where it will return zeros instead of the actual data that was read.

I have a reproducible here: pipe.go which works on tag v0.2.0 but not on 5cb68b4.
As well as output before and after applying a patch that seems to fix it. (Edit: previous patch was not go fmt'd)

If the patch seems correct, I can submit a PR, but I'm unsure if this is the actually the right approach.
Would appreciate feedback on it.


While debugging, I found this:
A response Message is instantiated here as rread and is passed to a call which will eventually call recv:

p9/p9/client_file.go

Lines 308 to 309 in 5cb68b4

rread := rread{Data: p}
if err := c.client.sendRecv(&tread{fid: c.fid, Offset: uint64(offset), Count: uint32(len(p))}, &rread); err != nil {

The type assertion within recv fails here (and thus the payload is ignored, even if it was read successfully):

if payloader, ok := m.(payloader); ok {

Implementing the payloader methods on rread resolves this, but so does using rreadPayloader inside of the (c *clientFile) readAt call. (However, we can't populate .fullBuffer nor .cs in that scope so that's probably not the right solution.)

This also doesn't seem to happen when using tcp sockets, or at least I didn't witness it. So I'm not really sure what's going on.
The only thing that comes to mind is that they're handled differently in vecnet's ReadFrom, but this doesn't seem like it would influence which message type is used. Not sure why this is the case.

p9/vecnet/vecnet.go

Lines 35 to 36 in 5cb68b4

func (bufs Buffers) ReadFrom(r io.Reader) (int64, error) {
if conn, ok := r.(syscall.Conn); ok && readFromBuffers != nil {


Extra:
I discovered this problem after updating one of my utilities that creates a subprocess and communicates with it over interfaces that wrap stdio (if they're not wrapped, vecnet's ReadFrom tries to use them as syscall.Conn because *os.File implements that, which eventually fails with "socket operation on non-socket"/ENOTSOCK).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.