xoraes / devicereg Goto Github PK

Go, REST API interface to get the information of devices and reserve/release the device. There will be 1000’s of devices and multiple
customers (in 100s) who can reserve a device. I utilized golang channels and locks for this implementation. See TradeOffs below. 
 
Each device table will contain
a) GPS co-ordinate information
b) Serial number
c) IP address


To Execute via Make:
    * make run (starts https server with info logging on port 8081)
    * make debug (starts http server with debug logging on port 8080)

Running the executable directly:
./devicereg [-nossl] [-debug]

IMPORTANT NOTE: the program is preloaded with 20 devices with id "dev-1" to "dev-20" (inclusive).
In lieu for auth service/db, as long as "test" is the password, any username (customer) can be used with
Basic Auth.

Following rest APIs are supported using BASIC AUTH (see curl examples, please use password: test)
    GET /ping
    - HTTP 200: returns "Pong"

    GET /debug/vars
    - HTTP 200: returns memstats and gc info

    GET /devices/:id
    - HTTP 200: Json Object with serial, geo location and ip is returned
    - HTTP: 404: Device not found
    - HTTP: 403: Device is reserved and not owned by the customer

    GET /devices/claimed
    - HTTP 200: returns json array of all claimed devices. Note that only serial/id's are returned

    GET /devices/unclaimed
    - HTTP 200: returns json array of all unclaimed devices. Note that only serial/id's are returned

    POST /devices/:id/reserve
    - HTTP 200: successfully reserved the given device
    - HTTP 409: Unable to reserve because the device is already reserved
    - HTTP 404: Device not found

    POST /devices/:id/release
    - HTTP 200: successfully release the given device with given id
    - HTTP 403: cannot release device not owned by the customer
    - HTTP 404: Device not found


================
Sample Session Calls:
================
    curl -k -u C1:test -X GET https://localhost:8081/ping
    curl -k -u C1:test -X GET -u C1:test https://localhost:8081/devices/unclaimed       [200 OK:array of device ids]
    curl -k -u C1:test -X POST https://0.0.0.0:8081/devices/dev-1/reserve [200 OK:reserve dev-1 for customer C1]
    curl -k -u C2:test -X POST https://0.0.0.0:8081/devices/dev-1/reserve [409 Conflict]
    curl -k -u C1:test -X POST https://0.0.0.0:8081/devices/dev-1/release [200 OK]
    curl -k -u C1:test -X GET https://localhost:8081/devices/claimed      [200 OK: array of device ids]


Internal Structures:
NetworkDevices: Struct that holds the main device table and all other substructures.
	* (Un)Claimed Set: A "set" of (Un)claimed devices updated via a single channel consumer.
	* (Un)ClaimedBuffer Array: Acts as a buffer which serves the (Un)Claimed devices requests.
	   Updated periodically (1sec) via a worker goroutine.
	* Channel: A single buffered channel that is consumed by a single worker. The worker adds or removes devices from
	    Un(Claimed) device set.
	* DeviceTable: Map of device id => pointer to a device
	* Device: Struct containing device data, a mutex and a reservation flag.
	  A Mutex help serialize changes to the reservation flag.

Tradeoffs:

Release/Reserve are strictly consistent and synchronous requests. When a reserve or release requests are issued,
we do NOT lock the DeviceTable (see above). We lock the "Device" long enough to change the reservation flag in it.
Because the DeviceTable is not locked, concurrent reserve and release are fast and scalable.

Requests for Claimed/Unclaimed devices served via a buffer/cache. The buffer is periodically updated via a worker
goroutine. This makes the response eventually consistent but fast.

How is the (Un)ClaimedBuffer updated ?
When a reservation/release is made, an async message is fired to a single go channel. The buffer capacity for the
channel  is set to 1K. A worker goroutine processes the channel messages and serially (using mutex lock) updates
two sets - for unclaimed/claimed devices. NOTE: We are using both a channel and a mutex. The channel provides order
for operations on the set, while the mutex is used because there are concurrent reads(iteration)
and writes over Un(Claimed) sets. A goroutine runs periodically (every second) and iterates over the two
sets (by locking a set level mutex) and updates the (un)claimed buffer. Because the updates completely are
asynchronous, locking does not affect latency, however, the consistency is somewhat sacrificed.
This is generally ok, as the API calls for separate process for release/reserve requests in any case.

Tuning and scale:
Memory needed to run this program is mostly constrained by the number of network connections a machine can handle
not the size of data as data requirement per device is insignificant for a modern server.

The channel buffer size is set to 1000 by default. This can be increased to allow for higher concurrency.
As the number of devices increase we can also tune how often the worker process runs to update the buffer/cache. T
his allows us to make the tradeoff between cpu usage(latency) and consistency.

Utilizing Multi Cores/Parallelism: It is possible to optimize the program such that we shard the channel. This will
allow us to make better use of multi-core cpu by adding mutiple worker go-routines.