Giter VIP home page Giter VIP logo

pike's Introduction

pike

A minimal cross-platform high-performance async I/O library written in Zig.

Features

  • Reactor/proactor-based I/O notification support
    • epoll (linux)
    • kqueue (darwin)
    • i/o completion ports (windows)
  • Async POSIX socket support
    • bind, listen, connect, accept
    • read, recv, recvFrom
    • write, send, sendTo
    • get/set socket options
  • Async Windows socket support
    • bind, listen, connect, accept
    • read, recv, recvFrom
    • write, send, sendTo
    • get/set socket options
  • Async signal support
    • signalfd for epoll (linux)
    • EVFILT_SIGNAL for kqueue (darwin)
    • SetConsoleCtrlHandler for i/o completion ports (windows)
  • Async event support
    • sigaction (posix)
    • SetConsoleCtrlHandler (windows)

Design

Notifier

A Notifier notifies of the completion of I/O events, or of the read/write-readiness of registered file descriptors/handles.

Should a Notifier report the completion of I/O events, it is designated to wrap around a proactor-based I/O notification layer in the operating system such as I/O completion ports on Windows.

Should a Notifier report the read/write-readiness of registered file descriptors/handles, it is designated to wrap around a reactor-based I/O notification layer in the operating system such as epoll on Linux, or kqueue on Darwin-based operating systems.

The Notifier's purpose is to drive the execution of asynchronous I/O syscalls upon the notification of a reactor/proactor-based I/O event by dispatching suspended asynchronous function frames to be resumed by a thread pool/scheduler (e.g. kprotty/zap).

Handle

A Handle's implementation is specific to a Notifier implementation, though overall wraps around and represents a file descriptor/handle in a program.

Subject to the Notifier implementation a Handle's implementation falls under, state required to drive asynchronous I/O syscalls through a Handle is kept inside a Handle.

An example would be an intrusive linked list of suspended asynchronous function frames that are to be resumed upon the recipient of a notification that a file descriptor/handle is ready to be written to/read from.

pike's People

Contributors

linuxy avatar lithdew avatar nektro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

pike's Issues

Improve performance for signal handling.

PackedWaker / PackedList in waker.zig was quickly written and could possibly be improved. It takes a Set which is a packed struct of booleans representative of a bitset.

The PackedWaker is used for signal handling, where Set is the set of possible signals that a caller may suspend themselves on.

Upon the notification of a signal being received by a process, should any of the received signals match any callers waited-upon signals, the suspended caller would be resumed.

Once this feature is done, I will move signal handling on linux/darwin to utilize sigaction to be able to catch all signals directed towards a process unlike signalfd or kqueue EVFILT_SIGNAL to keep signal handling behavior consistent across platforms.

c_void is now anyopaque

I'm getting the following errors when compiling a project that uses pike:

/home/duane/Projects/cleartouch/lib/pike/os/windows.zig:35:25: error: use of undeclared identifier 'c_void'
        @ptrCast(*const c_void, &guid),
                        ^
/home/duane/Projects/cleartouch/lib/pike/os/windows/ws2_32.zig:82:21: error: use of undeclared identifier 'c_void'
    lpSendBuffer: ?*c_void,
                    ^

I believe c_void is now anyopaque. I think it may have been backported to zig 0.9.0? Not sure though.

Remove pike.init() / pike.deinit().

pike.init() / pike.deinit() are only used to initialize winsock2 on Windows. A global atomic counter may be used to silently initialize winsock2 upon the very first Socket.init() call, and silently de-initialize winsock2 on the very last Socket.deinit() call.

Cancel all pending asynchronous socket I/O on Windows upon deinit().

On Windows, Socket.call() is responsible for driving the asynchronous execution of a socket I/O syscall.

What needs to be done is that prior to the execution of a syscall, an async frame should be queued to an atomic queue of frames stored in a Socket. Upon executing Socket.deinit(), all frames should be forcibly resumed with a flag that would cause error.OperationCancelled to be returned by Socket.call().

Allow sockets and other handles to be conditionally driven by an I/O notifier.

This is just an idea so far, though the idea is that for each exposed syscall (e.g. read()), there would be two variants: Socket.read(buf: []u8) !usize, and Socket.readAsync(buf: []u8) callconv(.Async) !usize.

Another method would be exposed, which is Socket.wait(.{ .read = ?, .write = ? }) callconv(.Async) void which allows users to manually await until a read/write-readiness notification is provided by an I/O notifier for a given file handle.

Calling Socket.read(...) will return immediately if the syscall cannot immediately be completed, and thus may be driven by Socket.wait(.{ .read = true }). Calling Socket.readAsync(...) will automatically retry the syscall, which is done by awaiting for readiness notifications from an I/O notifier via Socket.wait(.{ .read = true }).

Exposing these two variants also allows for Socket's in pike to be used in blocking mode (e.g. read() / write() may be called directly, blocking the current thread until said read() / write() is completed).

Expose more socket options via getters and setters.

As of right now, Socket.get() and Socket.set() assume that the level parameter which is passed to getsockopt() / setsockopt() os SOL_SOCKET.

However, there exist other options that don't necessarily assume the level parameter is SOL_SOCKET, such as TCP_NODELAY.

To expose these other options, the idea is to replace the tag payload types of the union SocketOption with a tuple of length two comprised of the level and option parameter.

Find alternative means for exposing configurable build flags.

Pike right now has three configurable build flags:

  • pike_task: struct type representing an asynchronous task.
  • pike_batch struct type representing a batch of asynchronous tasks.
  • pike_notifier struct type representing an I/O notifier implementation.

They are overridable via @import("root") at the moment, which is not the best option. Ideally, these flags should be configurable in build.zig for example. Rust exposes configurable build flags via Cargo.toml, in which libraries can configure flags for library dependencies they themselves depend on. Still need to figure out the best option in Zig.

How to listen to two open file descriptors?

I'm struggling to create what seems like should be a very basic use of pike: listen to two file descriptors, and if anything is read, print it out (without blocking).

My use case is I'd like to listen to several linux evdev devices (mouse, touchpad, keyboard for example).

I tried using example_event.zig as a starting off point, but I'm failing to understand how to connect file descriptors to a Notifier (I think?)

I'm also unsure why, when notifier.poll(10_000) is called in the example, it doesn't wait 10 seconds like I expect it should... unless it's actually receiving events, but then, what events is the example receiving?

Anyway, I'm new to Zig but like what I see here and just thought I'd write here in case this use case might help make Pike easier to pick up for others.

Thanks.

Define error sets and create unit tests/CI for sockets.

Error sets should be well-defined with the documentation for methods of Socket across platforms. Unit tests should be written to also demonstrate that behavior across platforms for the intersection set of methods shared across platforms is consistent.

Separate file handle from pike.Handle.

Sockets on Windows are represented by a ws2_32.SOCKET, unlike generic file handles which are represented by windows.HANDLE.

To cleanup all the verbose @ptrCast(ws2_32.SOCKET, self.handle.inner) casts in socket_windows.zig, it would be ideal to separate the file handle from pike.Handle.

Doing so would also allow removing unnecessary fields, such as async types that do not require a file handle (e.g. signal_posix.zig).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.