wahern / cqueues Goto Github PK

e.g. When using UDP, TCP options are attempted:

$ strace -e 'getsockopt' lua <<< 'cs=require"cqueues.socket"; cs.connect{host="localhost", port=2323,type=cs.SOCK_DGRAM}'
getsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], [4]) = 0
getsockopt(4, SOL_SOCKET, SO_REUSEPORT, [0], [4]) = 0
getsockopt(4, SOL_TCP, TCP_NODELAY, 0x7fff86f69ac8, 0x7fff86f69acc) = -1 EOPNOTSUPP (Operation not supported)
getsockopt(4, SOL_TCP, TCP_CORK, 0x7fff86f69ac8, 0x7fff86f69acc) = -1 EOPNOTSUPP (Operation not supported)
getsockopt(4, SOL_IPV6, IPV6_V6ONLY, 0x7fff86f69ac8, 0x7fff86f69acc) = -1 EOPNOTSUPP (Operation not supported)
+++ exited with 0 +++

Don't try and use SOL_TCP options when socket is not AF_INET, SOCK_STREAM, 0

I encountered an issue today, where the descriptor that got dup()d was setting !O_NONBLOCK on close

From here

I'll accept a patch that adds an option to tell .fdopen to take ownership of the existing descriptor number.

First and foremost, this issue is to track this potential patch

However, contrary to what you wrote:

because stale descriptors are perhaps the biggest source of bugs in asynchronous networking software, I prefer that the default behavior is to duplicate the descriptor.

I think it is a more sensible default to have cqueue's fdopen just take over the existing file descriptor; as we can't know what is going to happen to the old one.

My current 'take over from luasocket' code:

local sock = cqueues_socket.fdopen(conn:getfd())
-- cqueues uses dup(), so we close original socket
conn:close()
-- However, luasocket's close() sets it back into 'blocking' mode, as flags are shared between dup()'d fds.
do -- But if we re-dup cqueues' socket, it can re-establish it's flags.
    local newsock = cqueues_socket.fdopen(sock:pollfd())
    sock:close()
    sock = newsock
end

What it could be instead:

local sock = cqueues_socket.fdopen(conn:getfd())
conn:setfd(-1) -- Means invalid; luasocket will not clean up.

From the man page:

The sockfd argument is a file descriptor that refers to a socket of type SOCK_STREAM or SOCK_SEQPACKET.

diff --git a/src/lib/socket.c b/src/lib/socket.c
index e200abc..5e220f7 100644
--- a/src/lib/socket.c
+++ b/src/lib/socket.c
@@ -1242,8 +1242,10 @@ static int so_bind_(struct socket *so) {


 static int so_listen_(struct socket *so) {
-       /* FIXME: Do we support non-SOCK_STREAM sockets? */
-       return (0 == listen(so->fd, SOMAXCONN))? 0 : so_soerr();
+       if (so->host->ai_socktype == SOCK_STREAM || so->host->ai_socktype == SOCK_SEQPACKET)
+               return (0 == listen(so->fd, SOMAXCONN))? 0 : so_soerr();
+       else
+               return 0;
 } /* so_listen_() */

Without this, trying to use cs.listen{type=cs.SOCK_STREAM}:listen() would fail with "Operation not supported"

On my computer, the following has r empty:

local cqueues = require "cqueues"
local select = require "socket".select

local cq = cqueues.new()
cq:wrap(function()
    print("WRAPPED")
end)

local fake_sock = { getfd = function() return cq:pollfd() end }
local r, w = select({fake_sock}, {}, 3)
print("Should have cq's pollfd:", unpack(r))

I expect that :wrap would make the cq ready for reading.

When Integrating with another event loop, this was resulting in a stall of cqueues.

• Some protocols send a secret key over the wire.
• When reading from a socket with cqueues, the result is currently pushed as a string.
• In Lua, strings are interned, which means that depending on if the VM has seen the string before, it will take a varying amount of time to add it to the string pool.

These facts combined make many protocols subject to timing attacks if cqueues is used.

This problem can be mitigated If cqueues sockets could operate on some other non-string buffer (e.g. exposed as userdata) when given certain flags.

On my computer, the following program consistently segfaults:

local socket = require "socket" -- luasocket

local cqueues = require "cqueues"
local cqueues_socket = require "cqueues.socket"


-- Lets use luasocket to get a raw file descriptor
local rawfd do
    local s = socket.tcp()
    s:settimeout(0)
    s:connect("127.0.0.1", 80)

    rawfd = s:getfd()
    s:setfd(-1) -- Invalidate luasocket object; so luasocket doesn't close/clean it up
end
print("Got raw (unconnected) file descriptor: ", rawfd)

local cq = cqueues.new()
cq:wrap(function()
    local sock = cqueues_socket.fdopen(rawfd)
    -- Lets wait for the socket to connect with cqueues
    assert(sock:connect()) -- SEGFAULT!
end)
assert(cq:loop())

I'm wondering what side effects may exist if there are more than KPOLL_MAXWAIT events available.

e.g.
cqueue_step only calls kpoll_wait once, so each :step() may not make it through all pending events.
This means that the .poll() hack (for when not running in a coroutine) that steps twice may not get to the coroutine of interest.

Is it safe to call cqueues.cancel(-1)?
Please consider documenting this; and even special casing it in the code.

Continuing from #6

I have a fd that has had connect(2) called on it; but has returned EINPROGRESS
I want to convert it to a cqueues socket object, and then wait until it is connected.

From #6 (comment)

connect(2) requires passing the connect address, which is unknown when using socket.fdopen. There's no way at the moment to wrap an existing socket descriptor and then call :connect. The way .fdopen is meant to work, the socket should already be in the connected state.
...
You might be able to fake waiting for the connection to finish by doing cqueues.poll{ .pollfd = sock:pollfd(), .events = "w" }

Possible resolutions include:

• an API to add host+port as optional parameters to fdopen
• have sock:connect() poll until writable

I often already have a bitmask ready for poll() from a 3rd party C lib.

Could you make it so the :events() function can (optionally) return a number; which is the bitset of POLLIN, POLLOUT, etc?

The following example only works if you uncomment the lines to place the operations in a cqueues coroutine.

local cqs = require "cqueues"
local cs = require "cqueues.socket"
-- assert(cqs.new():wrap(function()
    local s = cs.connect("google.com", 443)
    s:starttls()
    local ssl = s:checktls()
    print(ssl:getPeerCertificate())
-- end):loop())

https://github.com/wahern/cqueues/blob/master/src/lib/notify.c#L688 is a loop reading from inotify fd multiple times. However, 'msg' is modified on parse, and never reset during the iterations causing out-of-bounds read.

I didn't expect sock:connect() to throw an error on failure
Rather, I expected it to return nil, err_msg similar to :read or :write

cs = require "cqueues.socket"
cq = require "cqueues".new();
s = cs.connect("localhost", 9876); -- port I'm not listening on.
cq:wrap(function()
    print(s:connect(2))
end)
assert(cq:loop())

/usr/local/share/lua/5.1/cqueues/socket.lua:259: socket:connect: Connection refused

If this is intentional, could you please document the behaviour on failure?

Currently only IPv4 or IPv6 is supported. Support address family preferences in DNS code. This includes:

1. Support OpenBSD's /etc/resolv.conf family directive.
2. Support AF_UNSPEC for socket.connect.
   2a) Support AF_UNSPEC in so_connect in lib/socket.c.
   2b) Support AF_UNSPEC family hint and a nil query type in dns_ai_open in lib/dns.c.
   2bi) Support multiple address family lookups in dns_ai_next in lib/dns.c.

The current return protocol is (return-value, error-code) rather than Lua-idiomatic (return-value, error-string, error-code). It was a regrettable choice early in development, and partly based on premature optimization. Unlike with Lua's file operations, many errors, particularly socket errors--EAGAIN, ETIMEDOUT, etc--are checked rather than immediately thrown. Pushing strings onto the Lua stack from C is quite slow, and usually only the integer code would be of value anyhow. This logic is still somewhat defensible, except that error strings can be memoized as upvalues, making them must less costly to push onto the stack. And diverging from the Lua idiom has turned out to be a gigantic headache. So the cost+benefit turned out to be vastly different than I expected.

We need to support the Lua idiom, and preferably make it the default. But the existing behavior should still be supported for the benefit of existing code.

‎[15:27:50] ‎daurnimator‎: lib/kpoll.c has much the same code as cqueues.c
‎[15:27:55] ‎daurnimator‎: intentional duplication?
‎[15:28:14] ‎William Ahern‎: yes. lib/kpoll.c is a refactoring of the code in cqueues.c.
‎[15:28:24] ‎daurnimator‎: which one is in use?
‎[15:29:06] ‎William Ahern‎: just the code in cqueues.c.
‎[15:29:11] ‎William Ahern‎: kpoll.c isn't finished, yet.

Avoid using luaL_check* and luaL_opt* functions on non-arguments, as the error messages use the stack index.

e.g:

cs=require "cqueues.socket";
u=cs.listen({})

lua: stdin:2: bad argument #-1 to 'listen' (string expected, got nil)
stack traceback:
[C]: in function 'listen'
stdin:2: in main chunk
[C]: in ?

The current default behavior for most socket errors is to throw rather than return. Everything except EAGAIN, ETIMEDOUT, and EPIPE is thrown. The original idea was that uncommon errors should be thrown in order to prevent applications from ignoring them and possibly losing data. This is no longer a defensible concern now that socket errors are 1) queryable and 2) repeated until explicitly cleared, just like C stdio FILE operations. Also, the prevalence and recoverability of errors can be very context-dependent; for example ECONNREFUSED may be quite common, either because of the nature of the application (e.g. SMTP callbacks) or because of frequent, transient disruptions of a highly trafficked service

The current behavior has simply become annoying given the way Lua code is typically developed. The old behavior should still be easily available.

NOTE: socket.onerror and socket:onerror can be used to change the default treatment of errors. But developers are understandably reticent to change default behaviors.

I was very confused that I could not socket.listen("0.0.0.0", 8765). kept getting EADDRUSE. Kept playing with the REUSEADDR option; but that wasn't it.
Turns out, because I had another daemon listening on IPv6 with the same port, the bind on IPv4 was failing.
Once I ran sysctl net.ipv6.bindv6only=1, everything was fine.

So, can you please add an option (possibly even default it to 'on') for IPV6_V6ONLY?

Support RFC6724 address sorting algorithm. Would also be nice to add support for the /etc/resolv.conf sortlist directive parsed by glibc and OpenBSD.

See http://pkg-lua.alioth.debian.org/policy.html

Most importantly, the packages need to adhere to the new naming pattern.

Preferably also generate libtoolized modules/libraries. Although this is more involved--excepting GNU Make, the build is intended to work natively and without dependencies on all supported platforms. Need additional build recipes that use libtool.

Currently cqueues has it's own compat52.h, is there a reason that https://github.com/hishamhm/lua-compat-5.2 isn't used?

They're usually pretty open to any adding functions/features if there is something missing.

It seems that currently you need to explicitly register for all filenames you want to track. I would like to track the names of created files automatically instead of rescanning the directory on getting created notification.

cq:loop(0) is a common operation that could deserve an optimised path.

Currently there is useless monotime() and arithmetic on this code path.

I'm encountering some event loops that sort of suck at timeouts; various things include:

• unable to modify timers once started
• un-cancelable timers
• cannot set another timer with the same 'action' as a socket read event
• '0' timeout invalid
• coarse timers (either based on MONOTONIC_COARSE or just second resolution)

To sidestep all this, I'm now thinking I should just use a timerfd and only bother with waiting for read events.
Would you want timerfd use as an option inside of cqueues?

socket:flush([mode][, timeout])
Flushes output buffer. Mode is one of the “nlf” flags described in socket.connect. A nil mode
implies “n”, i.e. no buffering and effecting a full flush. An empty string mode resolves to the
configured output buffering mode

The socket.connect docs do not have this description, socket:setmode() does.
Furthermore, the concept of a 'full flush' or 'fully buffered' are never explained

Lua 5.3 gives us a new function lua_rotate.

This can be used instead of/inside of e.g. luacq_slice.

Would like to be able to install cqueues via luarocks.

Currently the documentation refers the reader to read the source.

DNS resource record objects are implemented within cqueues.dns.record. The global tables and
shared methods are documented below. The type-specific accessory methods are quite numerous.
Until documented please confer with cqueues/src/dns.c. Also, the accessory method names are
usually equivalent to the structure member names in cqueues/src/lib/dns.h, which in return usually
reflect the member names in the relevant RFC

I've been mulling over how hard adding windows support to cqueues would be.
Lower level windows programming is not something I'm familar with, so please correct any incorrect assumptions I've made.

I'd love to have cqueues be a library I can depend on everywhere: if I write a network client in cqueues, even windows developers should still be able to use it (though maybe not in a performant way, which is the approach many applications take, e.g. nginx).

Comments @wahern has made before:

The concept is not portable to Windows because Windows lacks an analog to the pollable event queues of modern Unix systems. Libraries which try to abstract the two approaches—event readiness signaling versus event completion—invariably produce a pallid least common denominator library suffering all the weaknesses and enjoying none of the strengths of each approach.

Microsoft Windows support is basically out of the question¹, for far too many reasons to put here.
Aside from the more technical reasons, Windows I/O and networking programming interfaces have
a fundamentally different character than on Unix. Unix historically relies on readiness polling, while
Windows uses event completion callbacks. There are strengths and weaknesses to each approach.
Trying to paper over the chasm between the two approaches invariably results in a framework with
the strengths of neither and the weaknesses of both. The purpose of cqueues is to leverage the
strengths of polling as well as address the weaknesses.

¹ I have been toying with the idea of using an fd set in-place of a pollable descriptor on Windows, and taking the union of all fd sets when polling.

Solution 1. Add :pollset

One possible route is catering to the lowest common denominator (which helps with other platforms, such as AIX) and use select(). This can be accomplished by iterating over the fileno LLRB and building a pollset; this could be done on demand in a :pollset() function, or inside of fileno_ctl.

Issues

select does not expose a file handle we can wait on itself, and makes things much harder to nest/stack, :pollsets will need to be propogated up to the parent controller.

On windows, select boils down to a WaitForMultipleObjectsEx call, this has a maximum of 64 handles; which could easily be exceeded by a regular application. To get around this, you can spawn extra threads that each wait for events, and wait for them, and now we've recreated stackable event waiting.

Solution 2. Use an IOCP and RegisterWaitForSingleObject

• Create an IOCP port for each cqueues object.
• To 'add' a file descriptor to it, use RegisterWaitForSingleObject where the callback uses PostQueuedCompletionStatus
• To cancel a file descriptor, use UnregisterWaitEx
• kpoll_wait uses GetQueuedCompletionStatusEx
• :pollfd() returns the IOCP handle

Issues

Windows potentially has more than just readable and writable events; you need to use WSACreateEvent + WSAEventSelect to manage a HANDLE that encapsulates the events being waited for.

It is not possible to specify different event objects for different network events. The following code will not work; the second call will cancel the effects of the first, and only the FD_WRITE network event will be associated with hEventObject2:

rc = WSAEventSelect(s, hEventObject1, FD_READ);
rc = WSAEventSelect(s, hEventObject2, FD_WRITE); //bad

As pointed out here: #41 (comment)
Lua 5.1 doesn't support __gc on tables.

However, it does have newproxy.

diff --git a/src/dns.resolvers.lua b/src/dns.resolvers.lua
index a978ec3..b29d3aa 100644
--- a/src/dns.resolvers.lua
+++ b/src/dns.resolvers.lua
@@ -23,6 +23,19 @@ local loader = function(loader, ...)
        -- garbage collected automatically decrement the count and signal
        -- the condition variable.
        --
+       local new_hook
+       if _VERSION == "Lua 5.1" then
+               new_hook = function(mt)
+                       local u = newproxy(false)
+                       debug.setmetatable(u, mt)
+                       return u
+               end
+       else
+               new_hook = function(mt)
+                       return setmetatable({}, mt)
+               end
+       end
+
        local alive = {}

        function alive.new(condvar)
@@ -35,10 +48,6 @@ local loader = function(loader, ...)
                self.hookmt = { __gc = function (hook)
                        self.n = self.n - 1
                        self.condvar:signal()
-
-                       if hook.debug ~= false then
-                               io.stderr:write("reclaiming resolver\n")
-                       end
                end }

                return self
@@ -52,10 +61,9 @@ local loader = function(loader, ...)
                        if hook then
                                self.hooks[#self.hooks] = nil
                        else
-                               hook = setmetatable({}, self.hookmt)
+                               hook = new_hook(self.hookmt)
                        end

-                       hook.debug = debug
                        self.table[x] = hook
                        self.n = self.n + 1
                end
@@ -64,7 +72,6 @@ local loader = function(loader, ...)

        function alive:delete(x)
                if self.table[x] then
-                       self.table[x].debug = false
                        self.hooks[#self.hooks + 1] = self.table[x]
                        self.table[x] = nil
                        self.n = self.n - 1

The hook.debug stuff has to be removed because you can't index a newproxy.

TCP Fast Open (TFO) allows for the reduction of round trips while setting up a TCP connection by including data in the initial SYN packet.

On the server side, this means adding support for the SOL_TCP option TCP_FASTOPEN. This should be quite simple, and no api change around :accept() should be needed.

On the client side, we need to use sendto with MSG_FASTOPEN. Lua API wise, I suggest that if :write is called before :connect, we automatically use TFO when available (this can give a speed up to all applications!), possibly with an explicit option to turn it off.

Support

• Linux: available but turned off for IPv4 by default since 3.6 (clients), 3.7 (servers). On by default since 3.13. IPv6 support added in 3.16.
• Mac: Coming in iOS 9.0 and OSX 10.11

Useful links:

• https://lwn.net/Articles/508865/
• https://tools.ietf.org/html/rfc7413
• https://bradleyf.id.au/nix/shaving-your-rtt-wth-tfo/

Currently, sockets have a single timeout value, set with settimeout() that is shared for both :read and :write operations.

Would be nice to have separate timeouts for each operation. Something like one of:

• :timedread("str", 5) (terrible method name): this matches with other methods like :connect(timeout), :starttls(ssl, timeout), etc
• :settimeout(read, write)
• :setreadtimeout(5), :setwritetimeout()

When you have a :read() going on in one coroutine, and :close() the socket in another, :read() will throw an error:

/usr/local/share/lua/5.1/cqueues/socket.lua:397: calling 'recv' on bad self (socket closed)

The documentation says:

1.2.1 LuaJIT, Lua 5.2, Lua 5.3

cqueues principally targets Lua 5.2 and above. It’s not fully portable
to Lua 5.1 because cqueues relies on ephemeron tables to prevent
coroutine/controller reference cycles, and because Lua 5.1 does not
support yielding from metamethods and iterators. LuaJIT removes the
latter of these handicaps, and so cqueues targets LuaJIT secondarily.

Lua 5.1 with Coco (by the LuaJIT author himself) also removes the yield from metamethod/iterator handicap. Should the documentation mention this?

When an error occurs (e.g. inside a :step()), it would be nice to be able to inspect the failed thread.

One method of doing this would be returning the thread (coroutine) object from :step() on failure (3rd return value).

One of my servers threw at the top level with "internal error in continuation queue: Bad file descriptor"
What does this mean? How can I debug/fix it?

I see two places this could be thrown from:

• cqueues/src/cqueues.c

  Line 1620 in 56fd893

  luaL_error(L, "internal error in continuation queue: %s", strerror(error));

• cqueues/src/cqueues.c

  Line 1670 in 56fd893

  lua_pushfstring(L, "internal error in continuation queue: %s", strerror(error));

From #5

I do need to add proper :recvmsg and :sendmsg calls which allow reading and sending OOB data, support SO_OOBINLINE as a socket option, etc. It's just not a high-priority at the moment. It will require careful testing across all the platforms.

I added the mask because the assignment is a signed, narrowing conversion (.events is a short) and I was just trying to be conscientious about undefined behavior. I only included POLLIN and POLLOUT in the mask because those are the only events currently supported. I could have made the mask 0x7fff but figured it was more prudent to filter out values which previously wouldn't have been seen.

The code has to translate POLLIN and POLLOUT to the subsystem-specific flag. For Linux it's easy because POLLIN, POLLOUT, and POLLPRI are identical to EPOLLIN, EPOLLOUT, and EPOLLPRI. I think Solaris just reuses POLLIN directly.

However, for kqueue I haven't researched the issue. Some quick Googling turned up EV_OOBAND, but it's not documented in the OS X man page, and it doesn't even exist on OpenBSD.

The only application I know of off-hand that uses OOB data is telnet. FWIW, a grep through the PostgreSQL sources, for example, doesn't turn up any matches for "POLLPRI" or "MSG_OOB".

I found myself in need of sending multiple file descriptors in a single packet.
If an array of fds is passed as 2nd arg to :sendfd, it should send all of them in the one packet.

[15:30:18] ‎daurnimator‎: re: eventfd: so at the moment each cqueues object has one epoll fd, and a double ended pipe?
I was thinking about one cqueues object per (e.g.) http request: would this be too much overhead?, eventfd is meant to be lighter weight IIRC
‎[15:31:34] ‎William Ahern‎: yeah, it is kinda wasteful.
[15:31:44] ‎daurnimator‎: I was thinking of writing an eventfd patch for you; but wasn't sure if kpoll.c and cqueues.c should be kept in sync
‎[15:32:50] ‎William Ahern‎: i'll accept a patch for cqueues.c. patch not useful for lib/kpoll.c because i have some unsynchronized code out-of-tree (copied kpoll.c into another project so i could write a benchmarking utility).
[15:33:14] ‎William Ahern‎: just protect the code with #if HAVE_EVENTFD.
[15:33:19] ‎daurnimator‎: yup
[15:33:28] ‎William Ahern‎: that way I can defined HAVE_EVENTFD to 0 in my work projects if necessary.
[15:34:01] ‎William Ahern‎: also, could event delay creation until kpoll_alert is called.
[15:34:05] ‎William Ahern‎: even.

This is a low priority issue

On most operating systems, SCTP options are set with the additional data in the sendmsg call.
libc wraps this up and hides some of the uglyness with sctp_recvmsg and sctp_sendmsg, though then we have a completely seperate codepath (rather than say, coming up with a more generic cmsg api). However, on FreeBSD, they actually have an extra syscall for performance reasons, so the SCTP specific path may be warranted.

Misc note from 40d9933#commitcomment-8268928:

I used to think that the domain and protocol parameters for the socket(2) system call were redundant. But I recently discovered that for SCTP you can do either socket(AF_INET, SOCK_SEQPACKET, IPPROTO_SCTP) or socket(AF_INET, SOCK_STREAM, IPPROTO_SCTP), where the latter doesn't permit multiplexing. Which means to support SCTP I would need to add an additional parameter to the non-argument table versions of socket.connect and socket.listen.

TODO:

• SOCK_SEQPACKET support (#106)
• Allow passing protocol (in progress: https://github.com/daurnimator/cqueues/tree/pass-protocol)

I don't have a particular use case in mind here, but I really do like SCTP protocol.
I ended up doing research on it when learning about WebRTC, and it's quite sensible. But their spec has done some sort of weird SCTP over dTLS over UDP thing (normally dTLS is done on top of SCTP); which makes pure SCTP useless :(

This is currently an unsupported operation, I'm creating this issue to track progress.

If a thread gets killed in certain ways, :join hangs

In the below example I use secure computing mode (via ljsyscall), which will make it so that the thread is killed when it uses a syscall besides read, write, _exit, sigreturn

local ct = require "cqueues.thread"
local thread = ct.start(function(sock)
    local syscall = require "syscall"
    syscall.prctl("SET_SECCOMP", 1)
    syscall.sleep(1) -- something not allowed by seccomp, we'll get killed
end)
print(thread:join()) -- never returns

Sorry for vague title; but I don't know what the cause is.

In background, run: openssl s_server -dtls1, it will start listening on UDP port 4433 for DTLS connections.
To verify correct operation, you can connect with openssl s_client -connect localhost:4433 -dtls1

Now try running:

cs=require"cqueues.socket";
s=cs.connect{
    host="localhost",
    port=4433,
    type=cs.SOCK_DGRAM
};
print(s:starttls(1))

It fails to connect and prints nil, 110.

Investigating with ltrace, I see a different series of calls, 'openssl s_client' uses a BIO:

$ ltrace -e 'SSL*' openssl s_client -connect localhost:4433 -dtls1
openssl->SSLv23_client_method(4, 0x7fffb8c3a5e0, 8, 15)                    = 0x7f01378b4080
openssl->SSL_library_init(0x7fffb8c3c6f0, 0x46f872, 0, 0)                  = 1
openssl->SSL_load_error_strings(0x7f0137635640, 0x7f01376a2c3a, 0, 0)      = 0
openssl->SSL_CTX_new(0x7f01378b4c80, 0, 0, 0x7fffb8c39e10)                 = 0x1fc9610
openssl->SSL_CTX_ctrl(0x1fc9610, 32, 0, 0)                                 = 4
openssl->SSL_CTX_ctrl(0x1fc9610, 41, 1, 0)                                 = 0
openssl->SSL_CTX_set_verify(0x1fc9610, 0, 0x44ce00, 0)                     = 0
openssl->SSL_CTX_load_verify_locations(0x1fc9610, 0, 0, 0)                 = 0
openssl->SSL_new(0x1fc9610, 0, 0, 8)                                       = 0x1fca3f0
CONNECTED(00000003)
openssl->SSL_version(0x1fca3f0, 20, 0, 0)                                  = 0xfeff
openssl->SSL_set_bio(0x1fca3f0, 0x1fcbcd0, 0x1fcbcd0, -1)                  = 0
openssl->SSL_set_connect_state(0x1fca3f0, 0x1fcbcd0, 0x1fcbcd0, -1)        = 0x7f0137689790
openssl->SSL_get_fd(0x1fca3f0, 0x1fcbcd0, 0x1fcbcd0, -1)                   = 3
openssl->SSL_version(0x1fca3f0, 1, 16, 0)                                  = 0xfeff
openssl->SSL_ctrl(0x1fca3f0, 73, 0, 0x7fffb8c39de0)                        = 0
openssl->SSL_state(0x1fca3f0, 0x7fffb8c39de0, 0, 0)                        = 0x5000
openssl->SSL_ctrl(0x1fca3f0, 12, 0, 0)                                     = 0
openssl->SSL_pending(0x1fca3f0, 12, 0xfffffffffffd28d0, 0)                 = 0
openssl->SSL_get_fd(0x1fca3f0, 12, 0x1fca890, 0)                           = 3
openssl->SSL_get_fd(0x1fca3f0, 1, 0, 0x7fffb8c39c74)                       = 3
openssl->SSL_get_fd(0x1fca3f0, 64, 3, 3)                                   = 3
openssl->SSL_get_fd(0x1fca3f0, 64, 3, 3)                                   = 3
openssl->SSL_version(0x1fca3f0, 0x7fffb8c39e30, 0x7fffb8c39eb0, -1)        = 0xfeff
openssl->SSL_ctrl(0x1fca3f0, 74, 0, 0)                                     = 0
openssl->SSL_get_fd(0x1fca3f0, 0x7fffb8c39c40, 0, 0)                       = 3
openssl->SSL_write(0x1fca3f0, 0x1fbd230, 0, 8^C <no return ...>

While cqueues just uses SSL_set_fd():

_cqueues.so->SSL_load_error_strings(2, 0x7fff25916fd8, 0x7fff25916ff0, 0x7fff25916ff0) = 0
_cqueues.so->SSL_library_init(10, 1, 0x7f3712f378d3, 408)                  = 1
_cqueues.so->SSLv23_method(0, 1, 0, 469)                                   = 0x7f3713410e80
_cqueues.so->SSL_CTX_new(0x7f3713410e80, 1, 0, 469)                        = 0x6c5d50
_cqueues.so->SSL_new(0x6c5d50, 0, 0x6bbde0, 0x7f3713c62cc0)                = 0x6c8390
_cqueues.so->SSL_ctrl(0x6c8390, 33, 2, 0)                                  = 2
_cqueues.so->SSL_ctrl(0x6c8390, 33, 1, 0)                                  = 3
_cqueues.so->SSL_get_ssl_method(0x6c8390, 33, 1, 0)                        = 0x7f3713410e80
_cqueues.so->SSLv3_server_method(0x6c8390, 33, 1, 0)                       = 0x7f3713410c80
_cqueues.so->SSL_ctrl(0x6c8390, 55, 0, 0x68ef50)                           = 1
_cqueues.so->SSL_CTX_free(0x6c5d50, 0x68ef59, 0x7fff, 3929)                = 2
_cqueues.so->SSL_set_fd(0x6c8390, 7, 0, 469)                               = 1
_cqueues.so->SSL_set_connect_state(0x6c8390, 0x6b2b10, 0x6b2b10, 0x7fff25916554) = 0x7f37131db750
_cqueues.so->SSL_do_handshake(0x6c8390, 0x6b2b10, 0x6b2b10, 0x7fff25916554) = 0xffffffff
_cqueues.so->SSL_get_error(0x6c8390, 0xffffffff, 0x6b2ee8, 0)              = 2
_cqueues.so->SSL_do_handshake(0x6c8390, 1, 0, 469)                         = 0xffffffff
_cqueues.so->SSL_get_error(0x6c8390, 0xffffffff, 0x6b2ee8, 0)              = 2

Trying to build on OSX, where CC=export MACOSX_DEPLOYMENT_TARGET=10.5; gcc results in the error:

mk/vendor.cc: line 7: export: `-E': not a valid identifier
mk/vendor.cc: line 7: export: `-': not a valid identifier

Running manually with bash -x:

$ env CC='export MACOSX_DEPLOYMENT_TARGET=10.5; gcc' bash -x mk/vendor.cc
+ set -e
+ : export 'MACOSX_DEPLOYMENT_TARGET=10.5;' gcc
+ export 'MACOSX_DEPLOYMENT_TARGET=10.5;' gcc -E -
+ awk '/sunpro/||/clang/||/gcc/||/other/{ print $1; exit; }'
+ MACOSX_DEPLOYMENT_TARGET='10.5;'
mk/vendor.cc: line 7: export: `-E': not a valid identifier
mk/vendor.cc: line 7: export: `-': not a valid identifier

Add Lua bindings for .sa_bind socket option member so Lua code can choose egress interface.

cs = require "cqueues.socket"
s = cs.connect("localhost", 22);
s:close()
print(s:stat())

calling 'stat' on bad self (socket closed)

Which I expected to be able callable on a closed socket

There are a few of things missing before it's possible to write a UDP server with cqueues.

There is currently no binding to recvfrom(2) or recvmsg(2) to allow receiving messages from an un-connected peer. If you call udp_sock:recv() you don't know who the packet came from.

Also see:

• #20 (.listen() with type DGRAM returns not supported)
• #22 DTLS over UDP doesn't work

There's a couple of times I've needed some generic pooling code with low and high water marks; condition signalling, etc.

Would you consider splitting out the code in dns.resolvers.lua to be usable for any type of pool?