Hi,

experimenting with SSL (luasec) + copas. The server part seems to freeze when it receives more than one connection. When using pure TCP (by disabling/commenting out the SSL code), everything works as expected.

Not sure if this is a copas or luasec issue, but trying here first. I am currently using Copas 1.2.1 + LuaSec 0.6, so I'm not sure how this works on the latest version.

Here is the code I'm using:

local copas = require("copas")
local socket = require("socket")
local ssl = require("ssl")


sslParams = {
  mode = "server",
  protocol = "tlsv1_2",
   key = "./cert/clientkey.pem",
   certificate  = "./cert/client.pem",
   cafile = "./cert/root.pem",
  verify = {"peer", "fail_if_no_peer_cert"},
  options = "all"
}

local function requestHandler(sock)
  print("New connection")
  sock:setoption('keepalive', true)
  local ctx, err = ssl.newcontext(sslParams)
  if ctx == nil then
    print (err)
  end
  print("Handshaking..")
  sock = assert( ssl.wrap(sock, ctx))
  assert(sock:dohandshake(sslParams))
  print("Handshake OK")
  while true do
    print("receiving..")
    local data, err = copas.receive(sock)
    if not err then
      print(data)
      copas.send(sock, "Pong!\n")
    else 
      break
    end
  end
end


local server = socket.bind("*", 2000)
copas.addserver(server, requestHandler)
copas.loop()

Client code:

local socket = require("socket")
local ssl    = require("ssl")

local params = {
   mode = "client",
   protocol = "tlsv1_2",
   key = "../cert/clientkey.pem",
   certificate  = "../cert/client.pem",
   cafile = "../cert/root.pem",
    verify = {"peer", "fail_if_no_peer_cert"},
   options = "all",
}

local function createClient()
  local peer = socket.tcp()
  peer:connect("127.0.0.1", 2000)
  peer = assert( ssl.wrap(peer, params) )
  assert(peer:dohandshake())
  return peer
end


local peer = createClient()
peer:send("Ping!\n")
print(peer:receive("*l"))
-- freeze
local peer2 = createClient()
peer2:send("Ping!\n")
print(peer2:receive("*l"))
peer:close()

There is a

copas.addthread(func [, ...])

function but no way to remove said thread from the dispatcher. I know you can have the coroutine itself just return, but it would be nice to be able to remove the thread form the dispatcher externally.

Can the handlers communicate via channels as in libmill (http://libmill.org/tutorial.html#step6) or Haskell (http://hackage.haskell.org/package/base-4.8.1.0/docs/Control-Concurrent-Chan.html)?

If not, how should they avoid race conditions?

Any blockers for this? I'm trying to run the LuaRocks test suite with Lua 5.4 and I hit the 3.0.0-1 issue.

I try to query https://www.zpool.ca/api/status or https://poll.gmod.app, but get a copas.lua:688: TLS/SSL handshake failed: sslv3 alert handshake failure

Copas version is 3.0.0

When I try to make the request directly through luasec 1.0.2 it will succeed

The sites is behind CloudFlare and on the Internet it is often referred to.

I found this issues, but I don't know about SNI and all these protocols: lunarmodules/luasec#111, #109

I also tried to set some params as in the luasec by default, but it doesn't work for me

reqt.protocol = "any"
reqt.options = {"all", "no_sslv2", "no_sslv3", "no_tlsv1"}
reqt.verify = "none"

Perhaps a related discussion, though not related to copas and luasec: https://forum.defold.com/t/ssl-handshake-failed-1-for-certain-https-hosts-def-2633-solved/7067/25

Could these test pem files be added to the repository so that the test runs to completion? Thanks!

I'm currently getting this:

starting loop
Writing... 49501... Done!   0.11228013038635    nil 10485760
Reading... 49501... Done!   0.15213513374329    nil 10485760
Writing... 49500... Done!   0.15223217010498    nil 10485760
Reading... 49500... Done!   0.19177603721619    closed  10485760
1   minutes:    60.054605960846
Loop done
starting loop
/Users/hisham/.luarocks/share/lua/5.1/copas.lua:369: error loading private key (No such file or directory)  thread: 0x9690af8   tcp{client}: 0x9851fec
/Users/hisham/.luarocks/share/lua/5.1/copas.lua:369: error loading private key (system lib) thread: 0x96877c8   tcp{client}: 0x984ff84
/Users/hisham/.luarocks/share/lua/5.1/copas.lua:369: error loading private key (system lib) thread: 0x986d290   tcp{client}: 0x986b23c
/Users/hisham/.luarocks/share/lua/5.1/copas.lua:369: error loading private key (No such file or directory)  thread: 0x96812a0   tcp{client}: 0x9868c64
1   minutes:    60.061335086823
2   minutes:    120.12024402618
3   minutes:    180.17772102356
^C^C

When I modify the system time, I change it to a future time. For example, the current time is 2020.07.07, and it is modified to 2020.07.09. When I send an http request, the request will be block and will not time out.

I'd like to be able to just run make test (or some other single command) to execute all tests. That'll make testing easier, both manual and automatic (running tests on travis-ci would be nice).

• Ensure that failing tests exit with non-zero instead of just printing an error message
• Ensure that tests can be run from root project directory
• Ensure that tests use local copas sources
• (optional) Move supporting .pem files into a subdirectory
• Make largetransfer test run faster than 2 minutes
• Add test target to Makefile or some other way to run all tests
• Ensure that tests exercise all copas.* modules
• Add a separate target for running tests with coverage

coxpcall is mandatory with PUC Lua 5.1.
But with LuaJIT, there is a mismatch between these 2 files:
http://github.com/keplerproject/copas/blob/master/src/copas.lua#L28-L30 --> not required
http://github.com/keplerproject/copas/blob/master/src/copas/limit.lua#L13-L15 --> required

Ex. I have my client on Lua using copas:

client.lua

-- code using copas.addserver()

Myscript.lua

-- here im using my server
-- connection here, here im using send
x:send("string") -- i want, if my server returns something, i get it here, how?

Lua 5.1

The place where the error occurred:

Copas is used exclusively for asynchronous https requests. This is the first time I've noticed this error, and it caused copas.loop to stop working

Below is the code I used to run the script

local copas = require("copas")

copas.setErrorHandler(function(err, thread)
	local tr = debug.traceback()
	PRINT("copas.setErrorHandler GLOBAL", err, thread, tr) -- the error didn't appear here
end, true)

local res,err = pcall(copas.loop, function()
	print("Started")
end)

if not res then
	-- Stop working. Reason: ~/.luarocks/share/lua/5.1/copas.lua:904: attempt to index local 'new_q' (a number value)
	-- Note: My line number is 904 instead of 963 as pointed in issue. I think It's because luarocks downloads a bit older version of copas for lua 5.1
	print("Stop working. Reason: " .. tostring(err))
end

You can see my http request function here: https://github.com/TRIGONIM/ggram/blob/061387978c8c73ff2437749119e34752fbbcbc2e/lua/ggram/glua/http_async.lua#L61-L83

A second before the script crashed, there was a failed callback on line 80 in the link above

Hi,

I am just trying to make multiple GET requests to some urls and collect responses to parse the results. The example shown in High level requests matches what I need but it prints the result. Is there any way to collect the results of all the coroutines into a table. I tried doing this-

all_data={}
for _, host in ipairs(list) do
results=copas.addthread(handler, host)
table.insert(all_data,results)
end
copas.loop()

But all_data ends up being a collection of threadlists. I am not sure how to get the results from this thread. I see in general coroutines, result is retrieved by coroutine.resume but I don't see any copas.resume() method?

Also, for my purpose, i don't have any server implementation and just require an asynchronous client requesting different URLs. I feel for my application, I can just implement a simple coroutine example given in LUA coroutine tutorial instead of copas library. I am just curious if using copas for my application would add enough robustness/error handling so that I can overlook the overhead added by copas library or is copas library primarily used for server handling applications. It would be great if overhead can be estimated.

Thanks,
Aditya

Is coxpcall needed in LuaJIT? When running the tests in Travis, I forgot to add the coxpcall dependency and noticed that the tests passed in Lua 5.2, 5.3 and failed in 5.1 and LuaJIT.

Maybe the test can be made

if _VERSION == "Lua 5.1" and not jit then

Is there a reason why the rockspec constrains LuaSocket to = 3.0rc1-2 (as of 4.6.0)? This is an old version which does not work well with Lua 5.4.3+ because of lunarmodules/luasocket#334

I have written an express.js port based on copas. While testing my application via loader.io, I send thousands of requests to my application and after some time I encounter the mentioned error.

I have a theory that it might be related to the locking mechanism in copas in some way. Below I provide an example of how to reproduce the error. In addition to express, the application uses my implementation of mysql connection pool, which executes queries using luasocket + copas exclusively. If you set the number of connections and workers to low in the pool settings, the pool will not have time to execute requests and soon there will be 1000 unprocessed requests in copas.queue (pool.queries) and the application will crash.

I think that this problem should be tried to solve in copas but if it is impossible (I have tried some variants), are there any ways to handle this error?

Used libraries:

• lua-express: https://github.com/TRIGONIM/lua-express/tree/main
• luasocket-mysql with connection pool: https://github.com/TRIGONIM/luasocket-mysql/

Reproduce example:

local host, port = os.getenv("MYSQL_HOST"):match("([^:]+):?(%d*)")
local mysql_pool = require("mysql.pool").new({
	pool_size  = 1,
	workers    = 1,
	mysql_opts = {
		host      = host,
		port      = port,
		user      = os.getenv("MYSQL_USER"),
		password  = os.getenv("MYSQL_PASS"),
		database  = os.getenv("MYSQL_NAME"),
		charset   = "utf8mb4",
	},
})

local express = require("express")
local app = express()

local request_num = 0
app:use(function(_, _, next)
	request_num = request_num + 1
	print("✅ request_num: " .. request_num)
	next()
end)

app:post("/", function(req, res, next)
	mysql_pool:query("SELECT 1 + 2 as res", function(q_res, q_err, q_errcode, q_sqlstate)
		res:send("res: " .. q_res[1].res)
	end, 50)

	print("current queries pool size", mysql_pool.queries:get_size())
end)

app:get("loaderio-hash.txt", function(req, res)
	res:send("loaderio-hash")
end)

app:listen(3000)

loader.io settings

I chose the POST method out of habit, but it probably doesn't matter.

From: Jerzy Labocha
Sent: dinsdag 7 april 2015 0:57
To: Thijs Schreijer
Subject: Re: Possible bug in Copas ssl_implemented?

Hello Thijs,

something is wrong, at least for me. While the "body" string is 10MB long, everything works rather well. After reducing the body size to 1MB (local body = ("A"):rep(1024_1024_1)), test (LuaJIT 2.1) runs like this:

z:\>luajit largetransfer.lua
starting loop
Writing... 49501... Done!       1.282660484314  nil     1048576
Reading... 49501... Done!       1.2983026504517 nil     1048576
Writing... 49500... Done!       1.3452293872833 nil     1048576
Reading... 49500... Done!       1.3452293872833 nil     1048576
1       minutes:        60.081690311432
Loop done
starting loop
Writing... 49500... Done!       0.67261457443237        nil     1048576

After the last message, I see is a zero CPU load, the counter does not work anymore (does not display anything).

The same test with Lua 5.1 (PUC Rio):

z:\>lua5.1.exe -e "package.path=[[.\?.lua;c:\LUA\bin\lua\?.lua;c:\LUA\bin\lua\?\
init.lua;]]..package.path; package.cpath=[[.\?.dll;c:\LUA\bin\?.dll;c:\LUA\bin\l
oadall.dll]]..package.cpath" largetransfer.lua
starting loop
Writing... 49500... Done!       1.5014305114746 nil     1048576
Reading... 49500... Done!       1.5327110290527 nil     1048576
Writing... 49501... Done!       1.5639896392822 nil     1048576
Reading... 49501... Done!       1.5796298980713 closed  1048576
1       minutes:        60.088495254517
Loop done
starting loop
Writing... 49500... Done!       1.1886329650879 nil     1048576

The symptoms are the same as for LuaJIT, it looks like some sort of deadlock.
I downloaded the sources from github hour ago (dated 2015-04-06 04:46)

Tomorrow will check on another machine.

Jurek.

I ran all the tests on lua 5.3 and they seem to work fine

Can I ask you to update the luarockt for this project, please?

I tried running the testlimit.lua test with both Copas 2.0.0 and the current Git HEAD (bd3de0b), using both Lua 5.1 and 5.3, and got the same output:

lua-5.1 ./testlimit.lua                         
Starting: http://www.google.com...
...Users/hisham/.luarocks/share/lua/5.1/copas/limit.lua:21: attempt to yield across metamethod/C-call boundary  thread: 0x8148dc0   nil
Starting: http://www.microsoft.com...
...Users/hisham/.luarocks/share/lua/5.1/copas/limit.lua:21: attempt to yield across metamethod/C-call boundary  thread: 0x8149258   nil
Starting: http://www.apple.com...
...Users/hisham/.luarocks/share/lua/5.1/copas/limit.lua:21: attempt to yield across metamethod/C-call boundary  thread: 0x8149730   nil
Starting: http://www.facebook.com...
...Users/hisham/.luarocks/share/lua/5.1/copas/limit.lua:21: attempt to yield across metamethod/C-call boundary  thread: 0x8149c50   nil
Starting: http://www.yahoo.com...
...Users/hisham/.luarocks/share/lua/5.1/copas/limit.lua:21: attempt to yield across metamethod/C-call boundary  thread: 0x814a0a0   nil
All tasks finished
I was also waiting...

These are the rocks I have installed:

Installed rocks:
----------------

alt-getopt
   0.7.0-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

copas
   2.0.0-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

coxpcall
   1.15.0-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

dkjson
   2.5-2 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

lpeg
   1.0.0-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

luafilesystem
   1.6.3-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

luarocks
   2.2.3rc1-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

luasec
   scm-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

luasocket
   3.0rc1-2 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

moonscript
   0.3.2-1 (installed) - /Users/hisham/.luarocks/lib/luarocks/rocks-5.1

Any idea what's going on?

copas.http with a https URL that requires SNI will not work.

I tried to define the new (copas 3) wrap&sni parameters as defined in #105, but it looks like http.lua is not supporting SNI, and it will force the default protocol & mode triggering the backward compatibility mode.

In my understanding, copas.http should detect the https protocol and in that case, parse the domain and automatically define the SNI. It should also honor the user request and just pass whatever the user define inside SNI.

If you agree, I can send a pull request with the proposed change.

I'm curious about your thoughts on adding copas.select() call that would function like luasocket's socket.select() but allow a copas thread to block on multiple sockets' readyness. This seems like this would be beneficial both for porting code from blocking sockets to copas & generally for people who are used to non-blocking networking with select directly to develop new code on copas.

I wanted to get an opinion early because it seems like copas wasn't really written with something like this in mind. Right now it's only possible to yield a single socket (with a single readyness queue), so it would probably require quite a bit of low-level change. Is that an idea you'd even be willing to entertain? Or is it something you've thought about the best way to accomplish before?

#6 adds some functions that use os.time() for timing. This only has second precision. As Copas already relies on LuaSocket, it should use socket.gettime() which has higher precision.

Hello, your library was perfect for my project.
But I ran into a problem. I can’t understand how to pass the following header: “Content-Type: application/json” and I also want to switch between the POST and GET methods, how can this be done?

My code:

function:

function httpRequest(request, body, handler) 
        if not copas.running then
            copas.running = true
            lua_thread.create(function()
                wait(0)
                while not copas.finished() do
                    local ok, err = copas.step(0)
                    if ok == nil then error(err) end
                    wait(0)
                end
                copas.running = false
            end)
        end
        -- do request
        if handler then
            return copas.addthread(function(r, b, h)
                copas.setErrorHandler(function(err) h(nil, err) end)
                h(http.request(r, b))
            end, request, body, handler)
        else
            local results
            local thread = copas.addthread(function(r, b)
                copas.setErrorHandler(function(err) results = {nil, err} end)
                results = table.pack(http.request(r, b))
            end, request, body)
            while coroutine.status(thread) ~= 'dead' do wait(0) end
            return table.unpack(results)
        end
    end

Usage example:

httpRequest("https://sampmultichat.azurewebsites.net/get/last_message", nil, function(response, code, headers, status)
        if response then
            local data = json.decode(response)
            last_message= data["last_message"]
            print("last_message: ", last_message)
        else
            print('Error', code)
        end
    end)

How can I do the same with just the headers and the method POST or GET?

This error:

 ....3.2/1.19.3.2/luarocks/share/lua/5.1/copas/semaphore.lua:113: attempt to index local 'self' (a nil value) (coroutine: nil, socket: nil)
 stack traceback:
         ....3.2/1.19.3.2/luarocks/share/lua/5.1/copas/semaphore.lua:113: in function <....3.2/1.19.3.2/luarocks/share/lua/5.1/copas/semaphore.lua:109>
         [C]: in function 'xpcall'
         ....3.2/1.19.3.2/luarocks/share/lua/5.1/timerwheel/init.lua:136: in function 'step'
         [email protected]/1.19.3.2/luarocks/share/lua/5.1/copas.lua:1103: in function <[email protected]/1.19.3.2/luarocks/share/lua/5.1/copas.lua:1100>

When a timeout gets executed on the semaphore, it will look up the semaphore instance in the registry, belonging to the coroutine on which the timeout occured.

Sometimes this returns nil (eg. the semaphore is gone, garbage collected)

Only happens on master.

Copas 4.2.0

My script should perform HTTP requests to API 24/7 (about 1000 requests per minute), but after some time (sometimes after an hour, sometimes after several days) in the global error handler appears error {"timeout"}. Not as a string, but as a table.

Timeout error location

The code from the file launcher.lua, through which my scripts (Telegram bots) are launched:

local res,err = pcall(copas.loop, function()
	print("Started")

	copas.setErrorHandler(function(msg, co)
		-- msg = {"timeout"}
		-- tostring(co) = "(thread) thread: 0x55ada1cb5e0"
		logger(msg, co)
	end, true)
end)

...

Executing HTTP request

Simplified view of the function

local copas_request = require("copas.http").request

function http_request(tParams, fSuccess, fError)
	copas.addthread(function()
		copas.setErrorHandler(function(err, thread)
			-- handles errors like TLS/SSL handshake failed: timeout
			-- Everything works fine here
			timer_Simple(60, function() http_request(tParams, fSuccess, fError) end)
		end)

		local res, code, headers = copas_request(tParams)
		if res then fSuccess(code, res, headers) else fError(code) end
	end)
end

In the http_request() function, the errors are handled normally. Nowhere else is copas.addthread used, but an error {"timeout"} appears in the global handler. After this error appears, the execution of http requests starts to fall apart.

Examples of tracebacks:

From the global error handler.

2022-09-12 12:25:46 /home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:79: attempt to call upvalue 'f' (a nil value)
stack traceback:
        /home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:79: in function </home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:79>
stack traceback:
        launcher.lua:109: in function <launcher.lua:108>
        (tail call): ?
        /home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:47: in function </home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:45>
        (tail call): ?
        (tail call): ?
        ...e/ubuntu/.luarocks/share/lua/5.1/timerwheel/init.lua:136: in function 'step'
        /home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1211: in function </home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1208>
        (tail call): ?

2022-09-12 12:27:10 ["timeout"]
stack traceback:
        launcher.lua:109: in function <launcher.lua:108>
        [C]: in function 'oldpcall'
        /home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:68: in function </home/ubuntu/.luarocks/share/lua/5.1/coxpcall.lua:64>
        (tail call): ?
        /home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1041: in function '_doTick'
        /home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1332: in function 'step'
        /home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1424: in function 'step'
        /home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1460: in function </home/ubuntu/.luarocks/share/lua/5.1/copas.lua:1452>
        [C]: in function 'pcall'
        launcher.lua:102: in main chunk
        [C]: ?

The question is, how to make the script fail-safe to such problems? 🤔

P.S. Thank you very much for your work on copas. I'm a bit of a cheapskate, but if there was a "Donate" button here, I'd use it ❤️

The autoclose feature stems from a distant Copas past. Where it would operate solely as a TCP server. In those circumstances each incoming connection would get its own client-socket and a handler coroutine. So when the handler coroutine would exit, the connection was considered to be complete and it would be closed automatically.

These days however it is also used for outgoing connections, and additional threads, where the different threads share a single socket (for example the luamqtt MQTT client). If in those circumstances a single thread exits, we shouldn't be closing the socket automatically, since it would render the socket useless for the other threads.

see xHasKx/luamqtt#41

Destroying the queue on stop or finish happens either when the items in the queue itself becomes 0, or after a timeout. But originally I thought that queue destroying occurs when the queue becomes empty and all workers have finished their work. So the queue is destroyed before the worker with the last element finishes its job. This may be only my problem, but if this is the expected behavior, it would be nice to clarify it in the documentation, as there are a number of cases where we want to expect not only the queue to be empty, but all its workers to complete. E. g.:

local copas = require("copas")

local function main()
    local queue = copas.queue.new()

    for i = 1, 3 do
        queue:push(i)
    end

    queue:add_worker(function (number)
        copas.pause(2)
        print("Queue handler #" .. number .. " done")
    end)
    queue:finish()
    print("Exit from parent thread")
end

copas.addthread(main)
copas()

Result is:

> luajit queue_test.lua
Queue handler #1 done
Queue handler #2 done
Exit from parent thread
Queue handler #3 done

I use the following code to guarantee the behavior when the queue ends with all the workers:

local copas = require("copas")

local function main()
    local queue = copas.queue.new()
    local tasks = 0

    for i = 1, 3 do
        tasks = tasks + 1
        queue:push(i)
    end

    queue:add_worker(function (number)
        copas.pause(2)
        print("Queue handler #" .. number .. " done")
        tasks = tasks - 1
    end)
    queue:finish()

    while tasks > 0 do
        print("Waiting for the handlers to complete...")
        copas.pause(1)
    end
    print("Exit from parent thread")
end

copas.addthread(main)
copas()

Result is:

Queue handler #1 done
Queue handler #2 done
Waiting for the handlers to complete...
Waiting for the handlers to complete...
Queue handler #3 done
Exit from parent thread

I also tried to use queue:get_workers() to check which workers were still working, but that produced the same result as the first one. But maybe you have a better solution. Thanks.

Is it possible to switch to Xx.Yy.Zz format? I ask as I've been asked to do this for OpenWrt where most versions of packages follow that format. Changing it upstream would simplify the process.

This code hangs forever if the remote side does not exist and does not accept any connections (i.e., nothing is listening on the port):

local socket = require'socket'
local copas = require'copas'

copas.addthread(function()

    sock = socket.tcp()
    sock:settimeout(10)

    print('go on...')

    local _,err = copas.connect(sock,"10.211.55.2","8080")

    print('done')
    print(err)
end)

copas.loop()

Expected behavior: fail with error immediately ("connection refused" is returned immediately under the hood) or at least after 10 seconds.

The reason is this line of code in copas.lua:

function copas.connect(skt, host, port)
  skt:settimeout(0)
  ...

It just resets the timeout on the socket. The reason and purpose of this call is unclear to me, so I cannot suggest a solution. Removing this line fixes the bug, though ("connection refused" error returned immediately).

PS: Windows 7, Lua 5.1.4

The errorhandlers are called after the coroutine returns, this means that they will not have a stacktrace, which makes it very cumbersome to track down errors in handlers/coroutines.

Revising them by wrapping the handler function in an xpcall which attaches the stacktrace to the error before reporting.

First of all, thanks for the great work. I am able to convert my single connection server into multi-connection in no time.

My server is kind of reverse of traditional servers. This is a simulator to send hardware messages to the connected clients where server sends messages and occasionally receives messages the clients.

In my Handler, I have SendMessage() and ReceiveMessage() with timeout 0.1 sec just to make sure I read pending messages if any.

If I don't wrap the socket in the copas, time out on client:receive() works fine but if I wrap the socket in copas, timeout doesn't work. It waits there indefinably. I even tried swapping the socket wrap and socket:settimeout function but didn't help.

Any help would be appreciated.

-------------------------------------------------------------------------
-- ReceiveMessage : Receive the message from client if available
-- @param client: socket
-------------------------------------------------------------------------
function ReceiveMessage(client)

     --set the timeout 
     client:settimeout(0.1, 't')

    -- wraps socket into copas
     client = copas.wrap(client)

   --read the header of 8 bytes
    local header, status = client:receive(8)

     if(header== nil) then
       return
    else
      -- get the payload size from the header
    local length = tonumber(string.byte(header,3))

      -- read the payload
    local payload, status, partial = client:receive(length)
    end
end

As in keplerproject/xavante#19, would it please be possible to update the luasocket dependency to 3.0rc1 ?

I have a scenario where I need to occasionally check for a socket, so I add it with addthread when I need to and then I remove it with removeserver when I'm done, but I don't want to close and reopen the socket each time; I want to just leave it open (so that no-one else takes over the port number while the application runs).

I wrote this hack in my code (highly dependent on the Copas implementation) to stop it from closing:

-- HACK to stop copas.removeserver(wskt) from calling wskt:close()                                                                   
local proxy = setmetatable({ socket = wskt, close = function() end }, getmetatable(copas.wrap(wskt)))                            
copas.removeserver(proxy)                                                                                                        

Would it be possible to add an extra flag argument to copas.removeserver to stop it from closing the socket automatically? I can send a PR if the idea is accepted.

Snippet from the test results:

lua -e "print(([[=]]):rep(70))" -e "package.path='src/?.lua;'..package.path" tests/timer.lua
======================================================================
tests/timer.lua:34: didn't honour initial delay, or recurred (coroutine: thread: 0x01fc9c58, socket: nil)
stack traceback:
        [C]: in function 'assert'
        tests/timer.lua:34: in function 'callback'
        src/copas/timer.lua:15: in function <src/copas/timer.lua:11>
hello world 1
hello world 2
hello world 3
hello world 4
hello world 5
hello world 6
tests/timer.lua:44: expected t2 to already be stopped (coroutine: thread: 0x01fca1a0, socket: nil)
stack traceback:
        [C]: in function 'assert'
        tests/timer.lua:44: in function 'callback'
        src/copas/timer.lua:15: in function <src/copas/timer.lua:11>
test success!

Hey,

Copas currently doesn't support HTTPS servers that need SNI support. This breaks websites behind Cloudflare or shared hosting services.

LuaSec already has support for it, so it would be a matter of adding the relevant lines to dohandshake.

In the copas.lua ... copas socket wrapper... I think these two are wrong:

               setoption = function(self, ...) return self.setoption:accept(...) end,
               shutdown = function(self, ...) return self.shutdown:accept(...) end,

Shouldn't they be:

               setoption = function(self, ...) return self.socket:setoption(...) end,
               shutdown = function(self, ...) return self.socket:shutdown(...) end,

According to its documentation in the reference manual, copas.step() should return nil plus error message in case of errors. However, if _select(), which is called by step(), returns anything (else than "timeout") copas.step() calls error() which stops the whle program, as it is difficult to catch this one.
Is this intentional?
Note: errors in the threads itself are handled.

This code:

is intended to yield to prevent starvation. It should be replace by a simple copas.sleep() to achieve the same thing, but simpler. Also raises the question on starvation when reading data. Should read have a similar check?

For the last week I've been trying to find the causes of memory leaks in my application code from morning till night. After about 24 hours of running my bot in Lua, collectgarbage("count") reaches more than a gigabyte (lots of http requests running). I still don't really understand how garbage collection works in Lua, but maybe some coroutines aren't unloaded from memory or whatever. (Of course, I'm not excluding that the problem is in my code, but I've already searched everything)

Below is what the leak looks like on different versions of Lua. It always happens about the same way, in steps ( strange )

I came to standalone Lua after Garry's Mod. It has a function timer.Simple(iDelay, fCallback) for "one-time" timers and timer.Create(sName, iDelay, iReps, fCallback) for timers with repetition.

Below is code that in Garry's Mod runs very fast and leaves almost no garbage behind. More below is similar code for copas (Lua 5.3.6), which runs longer and leaves behind ~128 MB of garbage on Copas version 4.0.0 and ~96 MB on version 3.0.0

local function print_garbage()
	collectgarbage("collect")
	print("count", collectgarbage("count") / 1024)
end

timer.Create("garbage_timer", 0.5, 0, print_garbage)

print_garbage()

for i = 1,1000000 do
	timer.Simple(0.1, function() end)
end

tests/timers_leak.lua

package.path = string.format("../src/?.lua;%s", package.path)

local function print_garbage()
	collectgarbage("collect")
	print("count", collectgarbage("count") / 1024)
end

local co_timer = require("copas.timer")

co_timer.new({
	name      = "garbage_timer",
	delay     = 0.5,
	recurring = true,
	callback  = print_garbage,
})

print_garbage()

for i = 1,1000000 do
	-- timer.Simple(0.1, function() end)
	co_timer.new({
		name      = "name" .. i,
		delay     = 0.1,
		recurring = false,
		callback  = function(self)

		end,
	})
end

require("copas").loop()

I gave an example with timers, but I assume that it's not so much about them as it is about coroutines or that somewhere they are not unloaded from memory. Or maybe it's not a copas problem at all, but a specific feature of lua

Steps to reproduce

git clone https://github.com/lunarmodules/copas.git copas_leak_test
cd copas_leak_test/tests
curl -o timer_leak.lua https://pastebin.com/raw/b81qvt3S
lua timer_leak.lua

First of all, thanks for the great library.

Copas has a timeout functionality already in step() and loop() and it works fine. This timeout applies to waiting for a new connection.

I wonder if it would be possible to implement a different kind of timeout: a timeout for an already open connection so that a client cannot keep it open forever.

Does that make sense? See http://libmill.org/tutorial.html#step5.

I'm trying to make an asynchronous wrapper for lua-redis and faced a problem that if I execute any redis command in several threads but with one connection, for example INCR, then only one thread will receive the response and the script will terminate. However, all commands in REDIS will be done.

If you make a separate connection for each request, there is no problem, but it will create a huge number of connections. If you work with one connection and execute REDIS commands using copas.queue, they are executed very slowly (or you need to create workers, and with them new connections to be fast). This breaks the essence of asynchrony. I want to have one connection, but execute requests to it from different threads at once. Can I do this?

For the sake of interest, I ran a test with npm ioredis. In redis-cli INFO | grep connected_clients it creates a single connection, but it executes 100k INCR commands in about 3 seconds. In a single copas thread it would take 20 minutes. In 100 connections and 100 threads, it takes 12 seconds.

P.S. redis library I use: https://github.com/nrk/redis-lua

Hello,
the cpas.loop() function never stop. Here is the whole code:

local socket = require("socket")
local copas = require("copas")

local host = "127.0.0.1"
local port = 9982
local server, err = socket.bind(host, port)

function connectionHandler(socket)
  while true do
    local data, err = copas.receive(socket)
    if data == nil then
      print("Client socket: " .. err)
      break
    else
      print("Message: " .. data)
    end
  end
  print("Client disconnected")
end

function errorHandler(error, coroutine, socket)
  print("Error handler: " .. error)
end

copas.autoclose = true
copas.addserver(server, connectionHandler)
copas.setErrorHandler(errorHandler)

print("Server started")
copas.loop()
print("Server stopped")

In this code everything works, but when a client disconnect from the server, the connectionHandler break the loop and then display the "Client disconnected" message but the server still run, the "Server stopped" message is never displayed. Why?

Thanks for help.

This test is failing, because google started to redirect to https.

The expectation is that http://goo.gl/tBfqNu redirects to http://www.thijsschreijer.nl/blog/, but what actually happens is that http://goo.gl/tBfqNu redirects to https://goo.gl/tBfqNu which then redirects to http://www.thijsschreijer.nl/blog/.

You now get the same redirect as the next test is expecting to fail, while this test expects the request to succeed.

The only way I think this could be fixed is by using some other url which redirects from http -> http.

Add a LICENSE text file to the repository.