A simple framework/library for creating a web API powered by kdb+/q. Provided
with a trivial example API application (see example.q
)
Read the blog post: https://jonathonmcmurray.github.io/kdb/q/rest/api/2018/05/22/rest-api-in-kdb.html
The library is developed with qutil in mind.
To use with qutil, install package to location of QPATH
env var & within q
call
.utl.require"webapi"
For simplest installation, install via conda (from Anaconda/Miniconda installation) e.g.
$ conda install -c jmcmurray qwebapi
This will ensure all dependencies (incl. qutil
) are installed.
To use without qutil, download this repo & reQ
and load req.q
first followed by webapi/api.q
from this repo.
Usage of the library is fairly simple. Create a standard q function as usual,
and then use .api.define
to add it to the public API, while defining
defaults (and therefore datatypes). Examples are seen in example.q
Loading the example.q
script and setting a port will create an instance of
the example API:
jonny@grizzly ~ $ q example.q -p 8104
KDB+ 3.5 2017.10.11 Copyright (C) 1993-2017 Kx Systems
l32/ 2()core 1945MB jonny grizzly 127.0.1.1 NONEXPIRE
q)
This example can be accessed via browser, wget, another q session etc.
jonny@grizzly ~ $ wget -O - -q http://localhost:8104/gettime
{"time":"2018-05-08D21:36:32.125999000"}
jonny@grizzly ~ $ wget -O - -q 'http://localhost:8104/example?user=jonny&num=3'
{"user":"jonny","response":[{"col1":56701412,"col2":2125076200,"col3":1574348340},
{"col1":607727242,"col2":1753511110,"col3":2059906728},
{"col1":293447866,"col2":424844191,"col3":209668615}]}
There is inbuilt support for HTTP basic authorization, making use of an
authorization file similar to kdb+ itself uses with the -u
/-U
flag. This
file contains pairs of user:pass
on each line, where pass
can be plaintext
or the md5 hash of the password. For example, using the following file:
$ more auth.txt
user:5f4dcc3b5aa765d61d8327deb882cf99
anon:example
It is possible to login with user:password
or anon:example
:
q).Q.hg`$":http://user:notpassword@localhost:1234/gettime" //failed login example
""
q).Q.hg`$":http://user:password@localhost:1234/gettime"
"{\"time\":\"2018-05-23D13:16:26.227135000\"}"
q).Q.hg`$":http://anon:example@localhost:1234/gettime"
"{\"time\":\"2018-05-23D13:16:30.522253000\"}"
Note that this does enforce one limitation on passwords; they cannot be 32 digit hexdecimal numbers, as these would be indistinguishable from an md5 hash and therefore would allow logging in using md5 hash, thus defeating the point of not storing passwords in plaintext.
Adding support for other types of HTTP authentication is simply a case of
adding a .api.verify.{type}
function where {type}
is the authentication
type to be added. For example, Basic authentication is handled by
.api.verify.basic
. Credentials will automatically be passed to the relevant
function based on the type in the request Authorization HTTP header.
Things that need to be done:
- Handle POST requests (inc. JSON body)
- Required & optional parameters
- Replace urlencode definition with reQ (?)
- Error handling of function execution
- Generalise
.z.pp
and.z.ph
- refactor shared code into other functions - Support for authentication (?)
- Proper documentation