juliaweb / juliawebapi.jl Goto Github PK

I'm trying to process POST requests with JSON body. For now I don't see a way how to do it.
parsepostdata doesn't recognize the header Content-Type: application/json and interprets the data as parameters of a query only.

Any suggestions why it is not implemented? Should I use other libraries for JSON processing like Mux.jl or is it just non implemented feature?

POST /document HTTP/1.1
Host: localhost
Content-Type: application/json
cache-control: no-cache
Postman-Token: 6761fa70-a0d8-413a-a31c-79e882cc17d2

{
	"key":"document",
	"value":{"text":"some text"},
	"description":"",
	"type":"text",
	"enabled":true
}------WebKitFormBoundary7MA4YWxkTrZu0gW--

Hi, I am using Julia Version 0.4.3 with the Red Hat Linux distribution. I get the error below when testing JuliaWebAPI v0.0.2. Whether I add or checkout/build the package doesn't seem to make much difference. Any advice would be appreciated, thanks! If it's any help, HttpParser builds & passes tests only after fixing the issue with libhttp-parser.so described here.

julia> Pkg.add("JuliaWebAPI")
INFO: Building HttpParser
INFO: Building MbedTLS
INFO: Building ZMQ

julia> Pkg.test("JuliaWebAPI")
INFO: Recompiling stale cache file /home/james/.julia/lib/v0.4/ZMQ.ji for module ZMQ.
WARNING: Method definition info(Any...) in module Base at util.jl:334 overwritten in module Logging at /home/james/.julia/v0.4/Logging/src/Logging.jl:61.
WARNING: Method definition warn(Any...) in module Base at util.jl:364 overwritten in module Logging at /home/james/.julia/v0.4/Logging/src/Logging.jl:61.
INFO: Recompiling stale cache file /home/james/.julia/lib/v0.4/HttpParser.ji for module HttpParser.
spawining `/usr/bin/julia /home/james/.julia/v0.4/JuliaWebAPI/test/srvr.jl`
testing httpresponse...
        testfn1: Response(500 Internal Server Error, 4 headers, 38 bytes in body)
                data: UInt8[0x7b,0x22,0x64,0x61,0x74,0x61,0x22,0x3a,0x22,0x69,0x6e,0x76,0x61,0x6c,0x69,0x64,0x20,0x64,0x61,0x74,0x61,0x20,0x3a,0x20,0x2d,0x33,0x22,0x2c,0x22,0x63,0x6f,0x64,0x65,0x22,0x3a,0x2d,0x33,0x7d]
                hdrs: Dict{AbstractString,AbstractString}("Content-Type"=>"text/html; charset=utf-8","Date"=>"Sat, 20 Feb 2016 02:54:41","Content-Language"=>"en","Server"=>"Julia/0.4.3")
        testfn1: Response(500 Internal Server Error, 4 headers, 17 bytes in body)
                data: UInt8[0x69,0x6e,0x76,0x61,0x6c,0x69,0x64,0x20,0x64,0x61,0x74,0x61,0x20,0x3a,0x20,0x2d,0x33]
                hdrs: Dict{AbstractString,AbstractString}("Content-Type"=>"text/html; charset=utf-8","Date"=>"Sat, 20 Feb 2016 02:54:42","Content-Language"=>"en","Server"=>"Julia/0.4.3")
        testbinary: Response(404 Not Found, 4 headers, 16 bytes in body)
                data: UInt8[0x69,0x6e,0x76,0x61,0x6c,0x69,0x64,0x20,0x61,0x70,0x69,0x20,0x3a,0x20,0x2d,0x31]
                hdrs: Dict{AbstractString,AbstractString}("Content-Type"=>"text/html; charset=utf-8","Date"=>"Sat, 20 Feb 2016 02:54:42","Content-Language"=>"en","Server"=>"Julia/0.4.3")
ERROR: LoadError: LoadError: test error in expression: (fnresponse(resp))["data"] == arg1 * narg1 + arg2 * narg2
MethodError: `*` has no method matching *(::ASCIIString, ::Dict{AbstractString,Any})
Closest candidates are:
  *(::Any, ::Any, ::Any, ::Any...)
  *(::AbstractString, ::AbstractString...)
 in fnresponse at /home/james/.julia/v0.4/JuliaWebAPI/src/APIInvoker.jl:60
 in anonymous at test.jl:90
 in do_test at test.jl:50
 [inlined code] from /home/james/.julia/v0.4/JuliaWebAPI/test/clnt.jl:37
 in anonymous at no file:0
 in fnresponse at /home/james/.julia/v0.4/JuliaWebAPI/src/APIInvoker.jl:60
 in anonymous at test.jl:90
 in do_test at test.jl:50
 [inlined code] from /home/james/.julia/v0.4/JuliaWebAPI/test/clnt.jl:37
 in anonymous at no file:0
while loading /home/james/.julia/v0.4/JuliaWebAPI/test/clnt.jl, in expression starting on line 34
while loading /home/james/.julia/v0.4/JuliaWebAPI/test/runtests.jl, in expression starting on line 11
===============================================[ ERROR: JuliaWebAPI ]===============================================

failed process: Process(`/usr/bin/julia --check-bounds=yes --code-coverage=none --color=yes /home/james/.julia/v0.4/JuliaWebAPI/test/runtests.jl`, ProcessExited(1)) [1]

====================================================================================================================
ERROR: JuliaWebAPI had test errors
 in test at pkg/entry.jl:803
 in anonymous at pkg/dir.jl:31
 in cd at file.jl:22
 in cd at pkg/dir.jl:31
 in test at pkg.jl:71

JuliaBox.jl is a very confusing name for this package, and prevents discovery. We should rename it to something more relevant.

Some suggestions include:

APIBox.jl
Serve.jl
APIServe.jl

cc: @ViralBShah

I'm exposing a function which accepts a (possibly long) string as input. Where can I change the configuration to avoid error 413? Thank you.

Not an issue... just thought you would like to know that https://chloe.plantenergy.edu.au is using
JuliaWebAPI in anger. I guess not the web part but the ZMQ transport part. Working well 👍

Can i expose an entire module via a ZMQ listener?

Ex.:

module exa
  function hi()
    ...
  end
  export hi
end
process([exa], "tcp://127.0.0.1:9999"; bind=true)

I saw that the command to terminate the server is :terminate but this is not a valid command.

function isvalidcmd(cmd)
    isempty(cmd) && return false
    Base.is_id_start_char(cmd[1]) || return false
    for c in cmd
        Base.is_id_char(c) || return false
    end
    true
end

because this function returns false

Base.is_id_start_char(':')

is there another way?

In Julia 0.7 on MacOS, I get:

============================================================
Running tests from test_httprpc.jl
============================================================
starting httpserver in async mode...
starting client...
starting http rpc tests.
Test Failed at /Users/stevenj/.julia/dev/JuliaWebAPI/test/clnt.jl:95
  Expression: resp.status == 404
   Evaluated: 200 == 404
ERROR: LoadError: There was an error during testing
in expression starting at /Users/stevenj/.julia/dev/JuliaWebAPI/test/test_httprpc.jl:17
ERROR: LoadError: failed process: Process(`/Applications/Julia-0.7.app/Contents/Resources/julia/bin/julia test_httprpc.jl --runhttptests`, ProcessExited(1)) [1]
Stacktrace:
 [1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:42
 [2] pipeline_error at ./process.jl:712 [inlined]
 [3] #run#509(::Bool, ::Function, ::Cmd) at ./process.jl:670
 [4] run at ./process.jl:668 [inlined]
 [5] run_test(::String, ::String) at /Users/stevenj/.julia/dev/JuliaWebAPI/test/runtests.jl:16
 [6] top-level scope at none:0
 [7] include at ./boot.jl:317 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1038
 [9] include(::Module, ::String) at ./sysimg.jl:29
 [10] include(::String) at ./client.jl:398
 [11] top-level scope at none:0
in expression starting at /Users/stevenj/.julia/dev/JuliaWebAPI/test/runtests.jl:26
ERROR: Package JuliaWebAPI errored during testing

This happens both with ZMQ 1.0 and with the previous version (0.6.4).

My number one wishlist for this is the ability to have a single ZMQ listener, dispatching the method call accross multiple julia processes. So maybe round robin, but dropping processes that hang.

I've used this for so long that I assumed this was in METADATA. Apparently it isn't. Lets add it?

cc: @tanmaykm @bmharsha @ViralBShah @mdpradeep

There's a deprecation warning... not sure if it's related.

I wanted to redefine the function but when I hit Ctrl-C, the Julia runtime also ends.

julia> process([(testfn1, true), (testfn2, false)], "tcp://127.0.0.1:9999"; bind=true)
WARNING: processs(apispecs::Array,...) is deprecated, use process(conn::APIResponder; async::Bool=false) instead
Stacktrace:
 [1] depwarn(::String, ::Symbol) at ./deprecated.jl:70
 [2] #process#15(::Logging.LogLevel, ::Bool, ::String, ::Bool, ::Bool, ::Function, ::Array{Tuple{Function,Bool},1}, ::String) at /Users/tomkwong/.julia/v0.6/JuliaWebAPI/src/APIResponder.jl:189
 [3] (::JuliaWebAPI.#kw##process)(::Array{Any,1}, ::JuliaWebAPI.#process, ::Array{Tuple{Function,Bool},1}, ::String) at ./<missing>:0
 [4] eval(::Module, ::Any) at ./boot.jl:235
 [5] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:66
 [6] macro expansion at ./REPL.jl:97 [inlined]
 [7] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
while loading no file, in expression starting on line 0

^Cfatal: error thrown and no exception handler available.
InterruptException()
jl_run_once at /Users/osx/buildbot/slave/package_osx64/build/src/jl_uv.c:132
process_events at ./libuv.jl:82 [inlined]
wait at ./event.jl:216
task_done_hook at ./task.jl:256
jlcall_task_done_hook_21957 at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply at /Users/osx/buildbot/slave/package_osx64/build/src/./julia.h:1424 [inlined]
finish_task at /Users/osx/buildbot/slave/package_osx64/build/src/task.c:232
start_task at /Users/osx/buildbot/slave/package_osx64/build/src/task.c:275

My configuration:

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

This line of code
if isempty(args) || !isalnum(args[1]) || !isalpha(args[1][1])
means that valid julia function names with underscores returns a 404 .

Hi, I re-run your example on a CentOS 7 with Julia v0.4.3, but encounter this error "cannot convert NULL to string". It seems that it is caused by a dependency HttpParser which can't parse integer.

Error Message
julia> run_rest(apiclnt, 8000)
ERROR (unhandled task failure): ArgumentError: cannot convert NULL to string
in on_header_value at /root/.julia/v0.4/HttpServer/src/RequestParser.jl:60
^Cfatal: error thrown and no exception handler available.

Troubleshooting observation

1. Based on JuliaWeb/Requests.jl#75, it suggests the problem is with older version of HttpParser. One developer reports success after installing HttpParser v2.1. I don't know how to replace with v2.1, though I tried something like Pkg.add("HttpParser"), then Pkg.checkout, etc without lucks. I checkout HttpParser from github, and build. It generates libhttpparser.so but i dont know where to copy to. CentOS7 Julia does not currently use libhttparser.so but it uses libhttpaser.so.2.
2. I test Pkg.test("HttpParser") with errors.
3. I installed Julia with cmake but it does not help resolving any issue.
4. I tried repeating your sample codes to create REST API by using Julia v0.5. same errors.

any pointer?
thanks

Hi @tanmaykm and rest of contributors,

Julia noob here! Trying to create a prototype based on your example. Only difference is that I'd like to have a function that adds to an existing array InfoncastSoundFiles.

I get a "function does not accept keyword arguments" error from APIResponder if I'm understanding the error stack correctly. I believe I'm not configuring the create_responder function properly, but it could just be a syntax error elsewhere. Please advise.

Thanks,
Jose

Take a look at my srvr.jl and httpsrvr.jl:

using Dates
using JuliaWebAPI

mutable struct SoundFile
    title::String
    creator::String
    date::String
    tags::Array
    URL::String
end

#Let's see how this wwould work if this is not a const
#const InfoncastSoundFiles = SoundFile[
const InfoncastSoundFiles = SoundFile[
  SoundFile("Fur Elise", "Ludwig Beethoven","12202019",["mp3","song", "music"],"https://URL Here"),
  SoundFile("Fidelio", "Ludwig Beethoven","11032011",["song", "dance"],"https://URL Here"),
  SoundFile("Missa Solemnis", "Ludwig Beethoven","03102019",["mp3", "song"], "https://URL Here")
]


function listSoundFiles()
  return InfoncastSoundFiles
end

function updateSoundFiles(InfoncastSoundFiles, title1, author, date1, tag1, url1)
    a = SoundFile(title1,author,date1,tag1,url1);
    #push!(InfoncastSoundFiles, a);
    append!(InfoncastSoundFiles, a);
    #print(InfoncastSoundFiles);
end


process(
    JuliaWebAPI.create_responder([
        (listSoundFiles, true),
        (updateSoundFiles, false)
    ], "tcp://127.0.0.1:9999", true, "")
)

httpsrvr.jl looks like this:

using JuliaWebAPI   #Load package

#Create the ZMQ client that talks to the ZMQ listener above
const apiclnt = APIInvoker("tcp://127.0.0.1:9999");

#Starts the HTTP server in current process
run_http(apiclnt, 8888)

As you can see, there's really not much that's different from the original example. Unfortunately, my lack of experience reading Julia code didn't allow me to decipher the options for JuliaWebAPI.create_responder to be able to configure it properly, which I think is the problem.

I would very much appreciate any help on this. As a matter of fact, I'll make a pull request to add it as one of the examples in the repo if you think it's worthwhile.

Thanks again.

https://github.com/tanmaykm/JuliaWebAPI.jl/blob/master/src/APIResponder.jl#L62

It is assumed here that all the APIs take only Dict type as an argument. This will force the APIs to take DICT type and convert the same to a string.

Any objections @tanmaykm ?

Hi guys,

I mentioned in google groups here, after Viral flagged this package, that I needed similar functionality and ended up building something myself using Morsel (before seeing JuliaBox.jl).

I am no expert for sure, but so far am happy with my Morsel server and Viral has been encouraging me to go with JuliaBox.jl instead.

Does the JuliaBox RESTServer support POST requests? I am building a web app and the Julia input data coming from the interface is too complex for parameters set in a GET request URL so I managed to get POST requests working (miraculously) with Morsel.

Also, what are some downsides you may see with Morsel as opposed to JuliaBox?

Like I mentioned to Viral recently, I am open to using this package and scrapping my own stuff if I can figure out how this works and will likely have more questions :)

The tag name "0.3.0" is not of the appropriate SemVer form (vX.Y.Z).
cc: @tanmaykm

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Julia v1.9.2, Windows 10
Add JuliaWebAPI v0.6.3 to a temp project with no other packages.

Tests run up to

============================================================
Running tests from test_httprpc.jl
============================================================
starting httpserver in async mode...

then stall.

For what it's worth, I was trying to update the package to the latest version of HTTP.jl and found this. The problem, is already there on the latest release, and I can't get past this.

First, I ran the process async. Then, I want to either a new function but it came back with the world age issue.

julia> apiresponder = APIResponder("tcp://0.0.0.0:8890")
JuliaWebAPI.APIResponder with endpoints:

julia> function testfn1(arg1, arg2; narg1="1", narg2="2")
           return (parse(Int, arg1) * parse(Int, narg1)) + (parse(Int, arg2) * parse(Int, narg2))
       end
testfn1 (generic function with 1 method)

julia> register(apiresponder, testfn1)
JuliaWebAPI.APIResponder with endpoints:
"testfn1"

julia> process(apiresponder; async=true)
JuliaWebAPI.APIResponder with endpoints:
"testfn1"

Now, add and register a new function:

julia> testfn4(arg1, arg2) = test1(args1, args2)
testfn4 (generic function with 1 method)

julia> register(apiresponder, testfn4)
JuliaWebAPI.APIResponder with endpoints:
"testfn1", "testfn4"

Hit the testfn4 url and got this exception:

julia> 29-Mar 23:40:30:ERROR:root:api_exception: MethodError: no method matching testfn4(::String, ::String)
The applicable method may be too new: running in world age 21847, while current world is 21848.
Closest candidates are:
  testfn4(::Any, ::Any) at REPL[7]:1 (method too new to be called from this world context.)

If I try to run the example from the readme, I get

julia srvr.jl &

ERROR: LoadError: ArgumentError: invalid value for Enum LogLevel: 1
 in enum_argument_error(::Symbol, ::Int32) at ./Enums.jl:27
 in convert(::Type{Logging.LogLevel}, ::Int32) at ./Enums.jl:79
 in (::JuliaWebAPI.#kw##process)(::Array{Any,1}, ::JuliaWebAPI.#process, ::Array{Tuple{Function,Bool},1}, ::String) at ./<missing>:0
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318

In the example we get 2 servers up, how can we kill the client side server? The one with the message [ Info: running HTTP RPC server...

In the example, the line process([(testfn1, true), (testfn2, false)], "tcp://127.0.0.1:9999"; bind=true) produces an error. Where is process described? That line of code is take from the example code on this project.

Hey @tanmaykm, @viralshah

I'm a little slammed at the moment, but wanted to share some feedback and thoughts on taking this for a test drive for the last week:

• For some reason, the logging functionality didn't work for me; I just ended up with empty .log files. I ended up putting println()s everywhere there were Logging actions and that worked really well to spin up two terminals and run the server in one and the invoker in another
• It'd be nice to be able to Ctrl+C the Invoker (doing it to the server worked just fine). Just more convenient to Ctrl+C than have to kill the process
• I had an issuer where I wasn't able to return non-JSON, the diff below fixed it for me

@@ -61,8 +64,12 @@ function get_resp(api::Nullable{APISpec}, status::Symbol, resp::Any=nothing)
     stcode = st[2]
     stresp = ((stcode != 0) && (resp === nothing)) ? "$(st[3]) : $(st[2])" : resp

-    if !isnull(api) && get(api).resp_json
-        return @compat Dict{String, Any}("code" => stcode, "data" => stresp)
+    if !isnull(api)
+        if get(api).resp_json
+            return JSON.json(stresp)
+        else
+            return stresp
+        end
     else
         return stresp

• I had to change all the info calls to Logging.info because of the recent change to Base that kills both names when they clash
• Doing some long-running connections, I often ran into the error below; not sure if there's a nicer way to handle this or how "bad" this is to encounter

ERROR (unhandled task failure): ArgumentError: stream is closed or unusable
 in check_open at stream.jl:301

A few questions:

• How do request headers come through? Is there a way to access them from a function call? I ask because I'm interested in doing some basic authentication through header values (I guess I could do it through query parameters instead...)
• How can you tell what HTTP method was used in invoking a call? I was only trying GET, but may want to shift to POST

Again, sorry for the mess here and not doing proper PRs, but wanted to share some feedback. Hopefully I'll be able to contribute more soon.

Currently, JuliaWebAPI works for functions that are exported via process and take String parameters.

It may be useful to have a mechanism where arbitrary functions (maybe limited to those with String and int/float params) could be executed via JSON and the APIInvoker.

cc: @tanmaykm @ViralBShah Thoughts?