Hi all,

Just wondering what community members think of changing the signature of HttpHandler.handle from f(req, res) to f(req)?

Currently a default response, namely Response(), is passed to the handler, which is then modified. Wouldn't it make more sense to simply pass in the request and return the response? From a user's point of view I think this makes the functionality of an application more obvious.

For reference, I currently use the following:

function app(req::Request)
    ...    # do stuff here
    res    # Response object
end

server = Server((req, res) -> app(req))    # Boilerplate
run(server, 8000)

Thoughts?

Currently all sockets for clients that have the Keep-Alive header set are not closed when close(Server) is called. This can lead to very unintuitive behaviour e.g. a server on the same ip and port as a previous instance will serve requests with the old callback and not the new one (took me a while to figure this out...). Additionally I assume that this will also prevent the Garbage Collection from doing its job.

As such I propose that all connections of keep-alive clients are closed whenever the corresponding server is closed. To do so we need to store all of these connections somewhere. Any suggestions on how to accomplish this efficiently?

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.4

• On 2014-08-15 the testing status was Tests pass.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("HttpServer")' log
 - Library "libhttp_parser"

INFO: Installing BinDeps v0.3.0
INFO: Installing Calendar v0.4.2
INFO: Installing FactCheck v0.1.2
INFO: Installing GnuTLS v0.0.1
INFO: Installing HttpCommon v0.0.5
INFO: Installing HttpParser v0.0.9
INFO: Installing HttpServer v0.0.7
INFO: Installing ICU v0.4.1
INFO: Installing Nettle v0.1.4
INFO: Installing SHA v0.0.2
INFO: Installing URIParser v0.0.2
INFO: Building ICU
=================================[ ERROR: ICU ]=================================

None of the selected providers can install dependency icu.
Use BinDeps.debug(package_name) to see available providers

while loading /home/idunning/pkgtest/.julia/v0.4/ICU/deps/build.jl, in expression starting on line 28

================================================================================
INFO: Building Nettle
INFO: Building GnuTLS
INFO: Building HttpParser

================================[ BUILD ERRORS ]================================

WARNING: ICU had build errors.

 - packages with build errors remain installed in /home/idunning/pkgtest/.julia/v0.4
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

================================================================================
INFO: Package database updated

>>> 'using HttpServer' log
ERROR: could not open file /home/idunning/pkgtest/.julia/v0.4/ICU/src/../deps/deps.jl
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_3500 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/ICU/src/ICU.jl, in expression starting on line 54
while loading /home/idunning/pkgtest/.julia/v0.4/Calendar/src/Calendar.jl, in expression starting on line 1
while loading /home/idunning/pkgtest/.julia/v0.4/HttpCommon/src/HttpCommon.jl, in expression starting on line 3
while loading /home/idunning/pkgtest/.julia/v0.4/HttpServer/src/HttpServer.jl, in expression starting on line 7
while loading /home/idunning/pkgtest/.julia/v0.4/HttpServer/testusing.jl, in expression starting on line 1


>>> test log
no tests to run
>>> end of log

Hi,

I am in the process of migrating a web app from v0.4.6 to v0.5.0-rc0.

Things are proceeding fine as I wade through deprecation warnings. I am incrementally getting on master for all the JuliaWeb packages, but I have one deprecation warning I can't figure out. This might be related to other issues/PRs, but I'm having trouble deciphering things, so I'm not sure.

Basically, my server launches without any obvious problems, i.e. no errors/warnings. but when I access a web page, I get the following deprecation warning:

WARNING: String(s::Cstring) is deprecated, use unsafe_string(s) instead.
 in depwarn(::String, ::Symbol) at .\deprecated.jl:64
 in String(::Cstring) at .\deprecated.jl:50
 in on_headers_complete(::Ptr{HttpParser.Parser}) at C:\Users\Eric Forgy\.julia\v0.5\HttpServer\src\RequestParser.jl:73
 in http_parser_execute(::HttpParser.Parser, ::HttpParser.ParserSettings, ::Array{UInt8,1}) at C:\Users\Eric Forgy\.julia\v0.5\HttpParser\src\HttpParser.jl:115
 in process_client(::HttpServer.Server, ::HttpServer.Client{TCPSocket}, ::Bool) at C:\Users\Eric Forgy\.julia\v0.5\HttpServer\src\HttpServer.jl:388
 in (::HttpServer.##8#10{HttpServer.Server,Bool})() at .\task.jl:309
while loading no file, in expression starting on line 0

Looking at https://github.com/JuliaWeb/HttpServer.jl/blob/master/src/RequestParser.jl#L73 , I'm guessing it is the convert, but I'm not sure.

Any ideas?

Thanks

julia> versioninfo()
Julia Version 0.5.0-rc0+0
Commit 0030eec (2016-07-26 20:22 UTC)
Platform Info:
  System: NT (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

Hi there,

I'm having trouble running examples/https.jl.
After creating a self-signed certificate as per the example, and executing the last line run(server, port=8000, ssl=cert_store) things seem OK. No errors yet.

When I navigate to https://0.0.0.0:8000 I get the error:
Error establishing SSL connection: ErrorException("stream is closed or unusable")

I'm using Debian 8, Julia 0.3.9.

Any ideas?

Jock

About a year ago, the Http* stuff was under hackerschool/. We moved it to JuliaLang at the time because there was bit rot and all the people who had made it had moved on from Hacker School.

I think that it'd mayyybe a good time now to split it out into its own organization, with this possible list of packages:

• GeoIP http://github.com/johnmyleswhite/GeoIP.jl @johnmyleswhite (abandonware right now)
• GnuTLS http://github.com/Keno/GnuTLS.jl @Keno
• HTTPClient http://github.com/amitmurthy/HTTPClient.jl @amitmurthy
• HttpCommon
• HttpParser
• HttpServer
• LibCURL http://github.com/amitmurthy/LibCURL.jl
• Meddle http://github.com/JuliaLang/Meddle.jl
• Morsel http://github.com/JuliaLang/Morsel.jl
• Requests http://github.com/Keno/Requests.jl
• SMTP http://github.com/aviks/SMTPClient.jl @aviks
• UAParser http://github.com/randyzwitch/UAParser.jl @randyzwitch
• URIParser http://github.com/Keno/URIParser.jl
• URITemplate http://github.com/Keno/URITemplate.jl
• WebSockets http://github.com/JuliaLang/WebSockets.jl

Deprecate

• Curl http://github.com/forio/Curl.jl @forio
• URLParse http://github.com/tanmaykm/URLParse.jl @tanmaykm

Thoughts?
(This may just be shuffling papers around on my virtual Github desk for little gain)

where is root dir on my hard disk ?
I have install julia on win 7 on D disk. and some path of julia are on C/users/.../.Julia...
where is root dir on my hard disk ?
Wher I must put the file index.html ?
Paul

@unix_only error at line 221 httpserver.jl

I understand why but given I am trying to use Juno/Atom as my IDE and I'm blocked by this I think that it would be good to flag that this is a dependency issue at the front of the project so that other projects don't pick it up...

Hi. I am using HttpServer on Mac 10.10
I am having trouble with single line under julia.app :

using HttpServer

It output :
ERROR: could not open file /Users/myusername/.julia/v0.3/GnuTLS/src/../deps/deps.jl
in include at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in reload_path at loading.jl:152
in _require at loading.jl:67
in require at loading.jl:54
in include at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.3.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in reload_path at loading.jl:152
in _require at loading.jl:67
in require at loading.jl:51
while loading /Users/myusername/.julia/v0.3/GnuTLS/src/GnuTLS.jl, in expression starting on line 6
while loading /Users/myusername/.julia/v0.3/HttpServer/src/HttpServer.jl, in expression starting on line 190

Even when I rm & reinstall pkg, or run it under terminal

thanks

Hello,

Seems like Base no longer exports the bind function so bind is no longer defined here unless you do

import Base: bind 

at the top.

Also, can someone point me to the discussion link where the decision was made to make certain Base functions not exported? I'm sure there are good reasons but it's kind of annoying to have to namespace functions that seem very std-lib.

I'll handle this soon unless someone beats me to it. Goal is to tidy up like HttpCommon was tidied recently.

HttpServer's request parser is adding extraneous characters to the end of req.data, e.g. 0x9c + 6 characters of '\0'.

For cases where the request header contains 'Content-Length' but not 'Transfer-Encoding' and where the HttpParser has returned extra bytes could we just return the number of bytes specified by Content-Length? (If that seems acceptable, I can put together a pull request)

Also perhaps relevant is that the subproject link from HttpParser.jl to http-parser appears to be dated.

Details

In the test code below, we send a 129 character JSON string using Requests.jl
HttpServer ends up with a req.data with length()=130 and endof()=137:

Listening on 8001...
{"requestId":"1411409233","dataset":"nxms",
"textFields":["xm_report"],
"idList":["wrtwertwer_A5016_14_1", "ttee_trwE_03_436_1"]}
length of jreq:129

in ismatch
length req.data:130
endof req.data:137
Response(200 OK, 9 Headers, 5 Bytes in Body)

Requests sent using a Python client lead to the same problem, so it doesn't appear to be on the client side.

When I attempted to echo back the request using return Response(req.data), it threw:

   ERROR: HTTP Parser Exception: HPE_INVALID_CONSTANT(25):invalid constant string
   in http_parser_execute at /home/keithc/.julia/v0.3/HttpParser/src/HttpParser.jl:110

using HttpCommon
using HttpServer
using Requests
using Debug

http = HttpHandler() do req::Request, res::Response
            if ismatch(r"^/echo/",req.resource)
                println("in ismatch")
                #return Response("hey")
                println("length req.data:", length(req.data))
                println("endof req.data:", endof(req.data))

                return Response("howdy")
                #=
                #return Response(req.data) throws:
                ERROR: HTTP Parser Exception: HPE_INVALID_CONSTANT(25):invalid constant string
                in http_parser_execute at /home/keithc/.julia/v0.3/HttpParser/src/HttpParser.jl:110
                =#
            else
                println("in 404")
                return Response( 404 )
            end
        end

http.events["error"]  = (client, err) -> println(err)
http.events["listen"] = (port )       -> println("Listening on $port...")

server = Server(http)
@async run(server, 8001)

sleep(1.0) 

jreq="""{"requestId":"1411409233","dataset":"nxms",
         "textFields":["xm_report"],
         "idList":["wrtwertwer_A5016_14_1", "ttee_trwE_03_436_1"]}"""

println(jreq)
println("length of jreq:", length(jreq))

function post_check(jreq)
    response=Requests.post("http://localhost:8001/echo/"; data = jreq)
    return response
end

response = post_check(jreq)
println(response)

The Julia version is:

Version 0.3.1-pre+48 (2014-09-08 12:46 UTC)
release-0.3/8c1b105* (fork: 74 commits, 44 days)
x86_64-redhat-linux

Hello,

I am using JuliaLang PPA nightly build (julia version 0.2.0-prerelease+3443) on Ubuntu 12.10 Quantal.

    Pkg2.add("BinDeps")
Pkg2.add("Calendar")
Pkg2.add("GnuTLS")
Pkg2.add("HttpServer")

But I failed to run the readme example

ERROR: access to undefined reference
 in ClientParser at /home/user/.julia/HttpServer/src/RequestParser.jl:143
 in run at /home/user/.julia/HttpServer/src/HttpServer.jl:174
 in include at boot.jl:238
 in include_from_node1 at loading.jl:96
 in process_options at client.jl:274
 in _start at client.jl:349
at /home/user/2.jl:11

Can you please help?

Just executing the run function would save us from a lot of deprecation/unexport/rename bugs.
Is there a way we can test running that function, even though it never returns?

Running the example:

using HttpServer

http = HttpHandler() do req::Request, res::Response
    Response( ismatch(r"^/hello/",req.resource) ? string("Hello ", split(req.resource,'/')[3], "!") : 404 )
end

http.events["error"]  = ( client, err ) -> println( err )
http.events["listen"] = ( port )        -> println("Listening on $port...")

server = Server( http )
run( server, 8000 )

I get the following error:

ERROR: TcpServer not defined
 in HttpHandler at _____/.julia/v0.4/HttpServer/src/HttpServer.jl:83
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:293
 in _start at ./client.jl:362
 in _start at _______/julia/usr/lib/julia/sys.dylib
while loading _____/main.jl, in expression starting on line 3

In my build, the Base.TcpServer type seems to be undefined. I am trying to find out if this is an isolated problem. I have built Julia manually on OSX 10.10.

METADATA.jl currently references tag 0.1.2, which is missing the changes to REQUIRE. Please tag and publish a later version so using HttpServer doesn't break on not finding MbedTLS.

Thanks!

Make naming of 'HTTP' packages (HTTPClient.jl, HttpServer.jl, HttpParser.jl, and HttpCommon.jl) consistent. Timing to mirror timing of rename of Color.jl to Colors.jl.

How to make a response with multiple Set-Cookie header? Dictionary only allowed to use one key for one value.

Is this possible?

If I am working on some code that involves launching an HttpServer, it would be nice if I could close the socket (if that is the right way to say that) without having to kill Julia.

If it is easy to implement, I could have a go at a PR, but some hints would be welcome as I'm just learning this stuff :)

Edit: Starting to dig into this a little...

When you "run", you need to provide a "Server" object. A Server object has a HttpHandler member object, which in turn has a Base.TCPServer object. If there is an easy way to stop that TCPServer, then a "close" method should be possible (?)

Edit^2: I found that TCPServer inherits from LibuvServer and in Base.streams.jl, we can find this encouraging method:

function close(stream::Union{LibuvStream, LibuvServer})
    if isopen(stream) && stream.status != StatusClosing
        ccall(:jl_close_uv,Void, (Ptr{Void},), stream.handle)
        stream.status = StatusClosing
    end
    nothing
end

Edit^3: I was able to close the HttpServer, but it did not die gracefully. Any ideas?

Listening on 0.0.0.0:8000...

julia> http
HttpServer.HttpHandler((anonymous function),Base.TCPServer(active),Dict("error"=>(anonymous function),"listen"=>(anonymous function)))

julia> tcp = http.sock
Base.TCPServer(active)

julia> close(tcp)

julia> ERROR (unhandled task failure): accept: software caused connection abort (ECONNABORTED)
 in accept at stream.jl:1063
 in handle_http_request at C:\Users\Eric Forgy\.julia\v0.4\HttpServer\src\HttpServer.jl:232
 in run at C:\Users\Eric Forgy\.julia\v0.4\HttpServer\src\HttpServer.jl:339
 in run at C:\Users\Eric Forgy\.julia\v0.4\HttpServer\src\HttpServer.jl:343
 in anonymous at task.jl:447
julia> tcp
Base.TCPServer(closed)

Hi,

I'm just running the basic example from the GitHub page

using HttpServer

http = HttpHandler() do req::Request, res::Response
Response( ismatch(r"^/hello/",req.resource) ? string("Hello ", split(req.resource,'/')[3], "!") : 404 )
end

server = Server( http )
run( server, 8000 )

It launches the server instance but immediately throws the following error:

Listening on 0.0.0.0:8000...
ERROR: type cannot be constructed
in ClientParser at /Users/marcstein/.julia/v0.3/HttpServer/src/RequestParser.jl:133
in handle_http_request at /Users/marcstein/.julia/v0.3/HttpServer/src/HttpServer.jl:224
in run at /Users/marcstein/.julia/v0.3/HttpServer/src/HttpServer.jl:305
in run at /Users/marcstein/.julia/v0.3/HttpServer/src/HttpServer.jl:309

I'm running Julia 0.3.11 on OSX El Capitan Beta 7. Has anyone gotten this to work on El Capitan?

ArgumentError: cannot convert NULL to string
 in on_header_value at /home/tom/.julia/v0.4/HttpServer/src/RequestParser.jl:60

Ref: JuliaGizmos/Blink.jl#22

I'm on master of HttpServer and HttpParser and it seems like the new updates to HttpParser breaks HttpServer.

MethodError: `convert` has no method matching convert(::Type{HttpParser.ParserSettings}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void})
This may have arisen from a call to the constructor HttpParser.ParserSettings(...),
since type constructors fall back to convert methods.WARNING: both Patchwork and Base export "Text"; uses of it in module Main must be qualified

Closest candidates are:
  HttpParser.ParserSettings(::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, ::Ptr{Void}, !Matched::Ptr{Void}, !Matched::Ptr{Void})
  HttpParser.ParserSettings(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, !Matched::Any, !Matched::Any)
  call{T}(::Type{T}, ::Any)
  ...
 in call at essentials.jl:57
 in call at /Users/rohitvarkey/.julia/v0.4/HttpServer/src/RequestParser.jl:133
 in handle_http_request at /Users/rohitvarkey/.julia/v0.4/HttpServer/src/HttpServer.jl:243
 in run at /Users/rohitvarkey/.julia/v0.4/HttpServer/src/HttpServer.jl:358
 in run at /Users/rohitvarkey/.julia/v0.4/HttpServer/src/HttpServer.jl:362
 [inlined code] from /Users/rohitvarkey/.julia/v0.4/Lazy/src/macros.jl:274
 in anonymous at task.jl:447

This is the traceback I obtained. After digging around I found that ParserSettings constructor call in HttpServer requires 2 new extra arguments as added in JuliaWeb/HttpParser.jl#36. These haven't been defined in HttpServer yet.

I would have made a PR but I wasn't sure what functions to create for these callbacks!

Regards,
Rohit

This error shows up when I try to run the Morsel demo app:

ERROR: listen! not defined
 in run at /Users/james/.julia/HttpServer/src/HttpServer.jl:169
 in start at /Users/james/.julia/Morsel/src/Morsel.jl:184

The relevant code in run is this. It appears that listen! no longer exists. What did it do / is there a clear drop in replacement?

Hi,
I couldn´t find any TLS HTTPServer example, nor couldn´t get it running by myself via "function run_https(server::Server, auth::GnuTLS.CertificateStore, args...)"
Could you please add an example?

Somewhat related to #13, HttpServer currently ignores Connection: close headers.

GnuTLS was pretty easy to set up. MbedTLS is not.

I am trying to update examples/https.jl to work properly with the new code, and having no luck. Can anyone shed light?

Probably need to write to a buffer/string and then check that.

Julia Version 0.4.0-dev+4504 (2015-04-26 02:17 UTC) (the newest version of Julia ...)

julia> module PackageName

       using Docile
       @doc """
       ...
       """
       end
Warning: both Docile and Base export "@doc"; uses of it in module PackageName must be qualified
ERROR: UndefVarError: @doc not defined

more

This warning happened after JuliaLang/julia#12739

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+7032 (2015-08-27 04:01 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 5d3ccd6 (0 days old master)
|__/                   |  x86_64-linux-gnu

julia> using HttpServer
WARNING: New definition
    write(#T<:Base.IO, HttpCommon.Response) at /home/sachs/.julia/v0.4/HttpServer/src/HttpServer.jl:178
is ambiguous with:
    write(#T<:Base.AbstractPipe, Any...) at stream.jl:547.
To fix, define
    write(#T<:Base.AbstractPipe, HttpCommon.Response)
before the new definition.
WARNING: New definition
    write(#T<:Base.IO, HttpCommon.Response) at /home/sachs/.julia/v0.4/HttpServer/src/HttpServer.jl:178
is ambiguous with:
    write(#T<:Base.DevNullStream, Any...) at process.jl:95.
To fix, define
    write(#T<:Base.DevNullStream, HttpCommon.Response)
before the new definition.

Cc: @vtjnash

Hi,

I first reported this on Morsel before I realized it traced back to HttpServer.

With this simple HTTP server taken from README:

using HttpServer

http = HttpHandler() do req::Request, res::Response
    Response("Hello World")
end

http.events["error"]  = (client, err) -> println(err)
http.events["listen"] = (port) -> println("Listening on $port...")

server = Server(http)
run(server, 8000)

I noticed there's an increase in memory usage each time it received new requests (monitored with htop). Apache Bench showed it clearly:

ab -n 10000 -c 5 http://0:8000/

On my machine (Debian Jessie 64-bit, Julia 0.3.0+6 built from source), it reached 164.9 MB from 115.4 MB after the first Apache Bench run and kept increasing. Following @IainNZ's advice, I added a route to trigger GC like so:

    if ismatch(r"^/gc/", req.resource)
        gc()
        Response("GC-ed")
    else
        Response("Hello World")
    end

But memory usage stayed the same even when the GC has finished. Any idea?

Duong

Maybe it is time to tag a new version so that new users on Julia 0.4 have better out-of-box experience? Current version you get from Pkg.add("HttpServer") produces some warnings on load, but all them are already fixed on master.

Trying to run a simple httpServer

using HttpServer

http = HttpHandler() do req::Request, res::Response
    Response("Hello World")
end

http.events["error"]  = (client, err) -> println(err)
http.events["listen"] = (port) -> println("Listening on $port...")

server = Server(http)
run(server, 8000)

I receive the following error

apr_poll: The timeout specified has expired (70007)

Loading the port from a browser and curl seems to work fine.

Listening on 8000...
ERROR: assertion failed: :((err[1]==0))
 in ICUDateFormat at C:\Users\Iain\.julia\ICU\src\ICU.jl:505
 in _get_format at C:\Users\Iain\.julia\Calendar\src\Calendar.jl:100
 in format at C:\Users\Iain\.julia\Calendar\src\Calendar.jl:174
 in CalendarTime at no file
WARNING: backtraces on your platform are often misleading or partially incorrect

related to JuliaStrings/ICU.jl#7 I think

https://groups.google.com/forum/?fromgroups=#!topic/julia-users/2a26Fwu3Fwc

Hangs indefinitely in tests
@malmaud what have you wrought?!

I'm not pressing for it, it's more of a questions, wouldn't it belong here (and elsewhere?)?

[SPDY is "dead", so do not worry about that.]

In case it's already supported, it should be in NEWS file, otherwise could just keep this open in case someone wants to implement later..

Not sure why this is only coming up now, but:

julia> using HttpServer
Warning: using HttpServer.run in module Main conflicts with an existing identifi
er.

...

julia> run( server, 8000 )
ERROR: no method run(Server,Int64)
WARNING: backtraces on your platform are often misleading or partially incorrect

Win64 Julia 0.2

:/$ julia ~/.julia/HttpServer/examples/hello.jl
WARNING: add_library_mapping is deprecated, use push!(DL_LOAD_PATH,"/path/to/search") instead.
ERROR: no method bind(TcpSocket,IPv4,Uint16)
in run at ~/.julia/HttpServer/src/HttpServer.jl:167
in include_from_node1 at loading.jl:91
in process_options at client.jl:274
in _start at client.jl:347
at ~/.julia/HttpServer/examples/hello.jl:11

This is in Julia:
Version 0.2.0-2661.rf19700d3c
Commit f19700d3c 2013-07-16 13:20:23

With the HttpServer in METADATA

The new HttpCommon release of v0.2.1 expects FileResponse and mimetypes to be moved to HttpServer in JuliaWeb/HttpCommon.jl@b6821d8.
But HttpServer does this only in 8005fe6, which is not included in the v0.1.0 release. (https://github.com/JuliaWeb/HttpServer.jl/tree/v0.1.0)

So Pkg.update() breaks stuff currently.

On 64-bit red hat linux with newly updated packages,
julia> VERSION
v"0.4.0-dev+3163"

The tests hang up at the line shown here:

julia> include("test/runtests.jl")
HttpServer utility functions
     - HttpServer.write does sensible things
5 facts verified.
HttpServer run
     - HttpServer can run the example

Also

julia> include("examples/hello.jl")
Listening on 8000...

does not respond to http://localhost:8000/hello/name

I have written a service that uses HttpServer.jl, but have an issue where I have to kill julia from another terminal because Ctrl-C doesn't work after handling a request (although I can kill it before any requests have been made). My code is at https://github.com/IainNZ/SudokuService, I haven't managed to make a minimal reproducible example though. The only noteworthy thing I can think of about this service that is different from the simple examples is that it calls an external library.

PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.

On Julia 0.3

• On 2014-08-15 the testing status was Tests pass.
• On 2014-08-16 the testing status changed to Package doesn't load.

Tests pass. means that PackageEvaluator found the tests for your package, executed them, and they all passed.

Package doesn't load. means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using failed.

This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.

Test log:

>>> 'Pkg.add("HttpServer")' log
 - Library "libhttp_parser"

INFO: Cloning cache of HttpServer from git://github.com/JuliaLang/HttpServer.jl.git
INFO: Installing BinDeps v0.3.0
INFO: Installing Calendar v0.4.2
INFO: Installing FactCheck v0.1.2
INFO: Installing GnuTLS v0.0.1
INFO: Installing HttpCommon v0.0.5
INFO: Installing HttpParser v0.0.9
INFO: Installing HttpServer v0.0.7
INFO: Installing ICU v0.4.1
INFO: Installing Nettle v0.1.4
INFO: Installing SHA v0.0.2
INFO: Installing URIParser v0.0.2
INFO: Building ICU
=================================[ ERROR: ICU ]=================================

None of the selected providers can install dependency icu.
Use BinDeps.debug(package_name) to see available providers

while loading /home/idunning/pkgtest/.julia/v0.3/ICU/deps/build.jl, in expression starting on line 28

================================================================================
INFO: Building Nettle
INFO: Building GnuTLS
INFO: Building HttpParser

================================[ BUILD ERRORS ]================================

WARNING: ICU had build errors.

 - packages with build errors remain installed in /home/idunning/pkgtest/.julia/v0.3
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

================================================================================
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of HttpServer
INFO: Use `Pkg.update()` to get the latest versions of your packages

>>> 'using HttpServer' log

ERROR: could not open file /home/idunning/pkgtest/.julia/v0.3/ICU/src/../deps/deps.jl
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:54
 in include at ./boot.jl:245
 in include_from_node1 at ./loading.jl:128
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
 in include at ./boot.jl:245
 in include_from_node1 at loading.jl:128
 in process_options at ./client.jl:285
 in _start at ./client.jl:354
 in _start_3B_1699 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.3/ICU/src/ICU.jl, in expression starting on line 54
while loading /home/idunning/pkgtest/.julia/v0.3/Calendar/src/Calendar.jl, in expression starting on line 1
while loading /home/idunning/pkgtest/.julia/v0.3/HttpCommon/src/HttpCommon.jl, in expression starting on line 3
while loading /home/idunning/pkgtest/.julia/v0.3/HttpServer/src/HttpServer.jl, in expression starting on line 7
while loading /home/idunning/pkgtest/.julia/v0.3/HttpServer/testusing.jl, in expression starting on line 1

>>> test log
no tests to run
>>> end of log

I've made several performance tests to compare HttpServer.jl with other frameworks, and here are some results:

Flask (Python), 1 thread

• latency: 1 ms
• throughput: 1200 rps

Flask (Python), 1000 threads

• latency: 102 ms
• throughput: 1200 rps

Spray (Scala), 1 thread

• latency: < 1 ms
• throughput: 5500 rps

Spray (Scala), 1000 threads

• latency: 12 ms
• throughput: 18700 rps

HttpServer.jl, 1 thread

• latency: 39 ms
• throughput: 24 rps

HttpServer.jl, 1000 threads

• latency: 160 ms
• throughput: 6200

All tests done with Gatling on my laptop with Intel Core i7 (2 physical / 4 virtual cores) using Julia v0.3.5. Number of threads refers to a number of simultaneous requests on a client side, latency and throughput - to a 50th percentile of corresponding values.

Results for Flask are pretty predictable: Python's GIL doesn't allow to use full power of multithreading, showing almost no difference between 1 and 1000 simultaneous requests from clients. Spray is also no surprise: being based on actor model (via Akka framework), it utilizes both - concurrency and asynchronous IO to gain topnotch results.

I also like how HttpServer.jl handles multiple simultaneous requests (probably thanks to libuv, since as far as I know Julia's @async macro, used to handle separate users, still uses only single system thread). But I'm really surprised and dissapointed by its latencies: compared to average of 1ms for other servers 39ms for HttpServer.jl sounds totally unreasonable.

Is it a known issue? Is it solvable within current architecture? Or HttpServer.jl just isn't meant to provide low latency?

Here is a simple server

using HttpServer

gc_enable(false)

const Z = "HELLO WORLD"

http = HttpHandler() do req::Request, res::Response
    r = Response(Z)
    r
end

server = Server( http )
@profile run( server, 8000 )

And here are three ab tests:
NO KEEPALIVE 15k:

$ ab -n15000  'http://localhost:8000/'
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1500 requests
Completed 3000 requests
Completed 4500 requests
Completed 6000 requests
Completed 7500 requests
Completed 9000 requests
Completed 10500 requests
Completed 12000 requests
Completed 13500 requests
Completed 15000 requests
Finished 15000 requests


Server Software:        Julia/0.4.0
Server Hostname:        localhost
Server Port:            8000

Document Path:          /
Document Length:        11 bytes

Concurrency Level:      1
Time taken for tests:   9.492 seconds
Complete requests:      15000
Failed requests:        0
Write errors:           0
Total transferred:      2505000 bytes
HTML transferred:       165000 bytes
Requests per second:    1580.27 [#/sec] (mean)
Time per request:       0.633 [ms] (mean)
Time per request:       0.633 [ms] (mean, across all concurrent requests)
Transfer rate:          257.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     0    1   2.3      0     276
Waiting:        0    0   0.2      0      23
Total:          0    1   2.3      1     276

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%    276 (longest request)

NO KEEPALIVE 20k:

$ ab -n20000  'http://localhost:8000/'
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 2000 requests
Completed 4000 requests
Completed 6000 requests
Completed 8000 requests
Completed 10000 requests
Completed 12000 requests
Completed 14000 requests
Completed 16000 requests
apr_socket_recv: Operation timed out (60)
Total of 16351 requests completed

WITH KEEPALIVE 20k:

$ ab -n20000 -k 'http://localhost:8000/'
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 2000 requests
Completed 4000 requests
Completed 6000 requests
Completed 8000 requests
Completed 10000 requests
Completed 12000 requests
Completed 14000 requests
Completed 16000 requests
Completed 18000 requests
Completed 20000 requests
Finished 20000 requests


Server Software:        Julia/0.4.0
Server Hostname:        localhost
Server Port:            8000

Document Path:          /
Document Length:        11 bytes

Concurrency Level:      1
Time taken for tests:   8.167 seconds
Complete requests:      20000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    20000
Total transferred:      3820000 bytes
HTML transferred:       220000 bytes
Requests per second:    2448.94 [#/sec] (mean)
Time per request:       0.408 [ms] (mean)
Time per request:       0.408 [ms] (mean, across all concurrent requests)
Transfer rate:          456.78 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0       3
Waiting:        0    0   0.1      0       3
Total:          0    0   0.1      0       3

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      1
  95%      1
  98%      1
  99%      1
 100%      3 (longest request)

As can be seen, HttpServer has a good performance, however after some number of requests it temporarily hangs somewhere. It is still operational after the hang has passed.

Tests were done with Julia 0.4.0 on MacOS 10.9.5 with standard ab

HTTP/1.1 has persistent connections by default, while HTTP/1.0 does not. Accordingly if the latter is specified connections should be terminated after every request unless the keep alive header is specified. Luckily http-parser already has a convenient method to query whether or not to terminate the connection.

It's important for me to be able to access the client's IP, but I don't see a way to do get it. Maybe this could be integrated into Request somehow?

Even the simplest example, hello.jl, is broken because creating a HttpServer and then calling run on it ends up being equivalent to the following interaction in the REPL:

julia> serv = Base.TcpServer()
TcpServer(not listening)

julia> bind(serv, Base.IPv4(uint32(0)), uint16(8000))
true

julia> serv
TcpServer(not listening)

julia> accept(serv)
ERROR: accept: Server not connected. Did you `listen`?
 in accept at stream.jl:547
 in accept at socket.jl:319

julia> listen(serv)
ERROR: no method bind(TcpServer,TcpServer)
 in listen at socket.jl:467

The result of running hello.jl:

$ julia examples/hello.jl 
ERROR: no method bind(TcpServer,TcpServer)
 in listen at socket.jl:467
 in run at /home/leah/.julia/HttpServer/src/HttpServer.jl:168
 in include_from_node1 at loading.jl:91
 in process_options at client.jl:274
 in _start at client.jl:347
at /home/leah/.julia/HttpServer/examples/hello.jl:11

What are we supposed to do to go from a "not listening" TcpServer to one that we can call accept on?

Line 381 in Websocket.jl:

sock = WebSocket(client.id, client.sock)

Receives a MbedTLS SSLContext in the client.sock value as per HttpServer.jl:
JuliaWeb/WebSockets.jl#51

  sess = MbedTLS.SSLContext()
        MbedTLS.setup!(sess, ssl_config)
        # set_priority_string!(sess)
        # set_credentials!(sess, cert_store)
        client = accept(server.http.sock)
        try
            MbedTLS.set_bio!(sess, client)
            MbedTLS.handshake(sess)
            # associate_stream(sess, client)
            # handshake!(sess)
        catch e
            println("Error establishing SSL connection: ", e)
            close(client)
            continue
        end
        client = Client(id_pool += 1, sess)
        client.parser = ClientParser(message_handler(server, client, websockets_enabled))
        @async process_client(server, client, websockets_enabled)`

As I'm not sure which project is better to help massage the socket into, I'm looking for help on this one. Please let me know what to do! What else might be needed to make websockets on TLS work? Thanks

PS Here's more interesting debug info:

HttpServer.Client{MbedTLS.SSLContext}(14,MbedTLS.SSLContext(Ptr{Void} @0x0000000003245910,MbedTLS.SSLConfig(Ptr{Void} @0x0000000002dff880,MbedTLS.CtrDrbg(Ptr{Void} @0x0000000002ca2f00,Nullable(MbedTLS.Entropy(Ptr{Void} @0x0000000002e54c90,Any[]))),#undef,(anonymous function),cert. version : 3
serial number : DB:7E:D8:55:3E:F1:AB:AC
issuer name : C=US, ST=CA, L=San Francisco, O=BXI, CN=bxi.io, emailAddress=[email protected]
subject name : C=US, ST=CA, L=San Francisco, O=BXI, CN=bxi.io, emailAddress=[email protected]
issued on : 2016-02-21 07:47:43
expires on : 2017-02-20 07:47:43
signed using : RSA with SHA-256
RSA key size : 2048 bits
basic constraints : CA=true
,MbedTLS.PKContext(Ptr{Void} @0x0000000002cf1160)),true,TCPSocket(open, 0 bytes waiting)),HttpServer.ClientParser(HttpParser,HttpParser.ParserSettings(Ptr{Void} @0x00007fcb39e9ffc0,Ptr{Void} @0x00007fcb39ea35d0,Ptr{Void} @0x00007fcb39ea3810,Ptr{Void} @0x00007fcb39ea3850,Ptr{Void} @0x00007fcb39ea3a80,Ptr{Void} @0x00007fcb39ea4c80,Ptr{Void} @0x00007fcb39ea50a0,Ptr{Void} @0x00007fcb39ea5210)))WARNING: both Patchwork and Base export "Text"; uses of it in module Main must be qualified
ERROR (unhandled task failure): MethodError: convert has no method matching convert(::Type{TCPSocket}, ::MbedTLS.SSLContext)
This may have arisen from a call to the constructor TCPSocket(...),
since type constructors fall back to convert methods.
Closest candidates are:
TCPSocket(::Any)
call{T}(::Type{T}, ::Any)
convert{T}(::Type{T}, ::T)
...
in call at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:57
in handle at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:387
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/HttpServer.jl:393
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/RequestParser.jl:104

PPS This is what I tried (using client.sock.bio):
dioptre/WebSockets.jl@3bb92d7

...and new error

handling websocket connection:
write: broken pipe (EPIPE)
in yieldto at ./task.jl:71
in wait at ./task.jl:371
in stream_wait at ./stream.jl:60
in uv_write at stream.jl:944
in flush at stream.jl:979
in locked_write at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:154
in write at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:168
in start_updates at /home/a/projects/bxi/server/www.jl:213
in anonymous at /home/a/projects/bxi/server/www.jl:274
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:15
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:8
in splitquery at /home/a/.julia/v0.4/Mux/src/basics.jl:28
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:8
in wcatch at /home/a/.julia/v0.4/Mux/src/websockets_integration.jl:12
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:8
in todict at /home/a/.julia/v0.4/Mux/src/basics.jl:21
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:12 (repeats 2 times)
in anonymous at /home/a/.julia/v0.4/Mux/src/Mux.jl:8
in anonymous at /home/a/.julia/v0.4/Mux/src/server.jl:38
in handle at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:394
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/HttpServer.jl:393
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/RequestParser.jl:104ERROR (unhandled task failure): write: broken pipe (EPIPE)
in yieldto at ./task.jl:71
in wait at ./task.jl:371
in stream_wait at ./stream.jl:60
in uv_write at stream.jl:944
in flush at stream.jl:979
in locked_write at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:154
in close at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:197
in handle at /home/a/.julia/v0.4/WebSockets/src/WebSockets.jl:395
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/HttpServer.jl:393
in on_message_complete at /home/a/.julia/v0.4/HttpServer/src/RequestParser.jl:1

Hi,

A page retrieved with curl gets truncated if there are letters like ä and ö in the content (utf-8).

Should the line:

response.headers["Content-Length"] = string(length(response.data))

use sizeof (length in bytes) instead of length (length in characters)?