This proposal contains breaking change of the 0MQ API.
Current 0MQ API of RE Manager contain a number of inconsistencies related to representation of queue items. The differences in representation of queue items accumulated over time as functionality of the Queue Server was extended from 'plans' to generalized 'items' and have no justification. The inconsistencies don't limit functionality of the Queue Server, but can make application development more complicated. The proposed changes will require very limited changes in RE Manager code, significant changes in unit tests. The existing applications communicating with RE Manager will also need to be changed, but those are going to be superficial changes (mostly renaming of parameters).
Following is the example of currently accepted representation of a plan and an instructions as queue items (the format of the items as they are added to the queue and returned by 0MQ API functions queue_item_get, queue_item_remove and queue_item_move):
# Current representation of a plan as a queue item
{
"name": "count", # Required
"args": [["det1", "det1"]], # Optional arguments (list)
"kwargs": {"num": 5, "delay": 1}, # Optional kwargs (dict)
"meta": {}, # Optional dict or list(dict)
"item_type": "plan", # Currently set by the server when the item is added to the queue
"item_uid": "...", # Set by the server
"user": "User Name", # Name of the user who added the plan, set by the server
"user_group: "Group", # User group to which the user belongs
}
# Current representation of an instruction as a queue item
{
"action": "queue_stop", # Required
"args": [], # Optional arguments (list)
"kwargs": {}, # Optional kwargs (dict)
"meta": {}, # Optional dict or list(dict) - not used for instructions
"item_type": "instruction", # Currently set by the server when the item is added to the queue
"item_uid": "...", # Set by the server
"user": "User Name", # Name of the user who added the plan, set by the server
"user_group: "Group", # User group to which the user belongs
}
The representation of plans and instructions is almost identical except that instruction 'name' is represented as 'action'. Change 1: replace key 'action' with 'name' (an instruction can be distinguished from a plan by looking at 'item_type' key). Using common schema for representation of all existing items will simplify processing of lists of items on all stages of development and simplify extension of API to different types of items if needed.
# Item representation using common schema. Optional arguments may be omitted
# when item is submitted in the request (`queue_item_add` or `queue_item_update`).
# If an items is set by the server, it will be overwritten by the server even if it is
# set in the request.
{
"item_type": "plan", # Required (should be 'plan' or 'instruction')
"name": "count", # Required
"args": [["det1", "det1"]], # Optional arguments (list)
"kwargs": {"num": 5, "delay": 1}, # Optional kwargs (dict)
"meta": {}, # Optional dict or list(dict)
"item_uid": "...", # Optional, set by the server
"user": "User Name", # Optional, set by the server
"user_group: "Group", # Optional, set by the server
}
In current implementation, the input and output parameters of API functions queue_item_add, queue_item_update and queue_item_add_batch are using different way to represent items (purely for historic reason):
# Current implementation: example plan representation for `queue_item_add` function
plan_payload = {
"plan": {
"name": "<plan-name>",
"args": [...], # Optional args
"kwargs": {...}, # Optional kwargs
"meta": {}, # Optional dict or list(dict)
},
"user": <user_name>,
"user_group": <user_group>,
}
# Current implementation: example instruction representation for `queue_item_add` function
instruction_payload = {
"instruction": {
"action": "<instruction-name>";
"args": [...]; # Optional args
"kwargs": {...}; # Optional kwargs
"meta": {}, # Optional dict or list(dict)
},
"user": <user_name>,
"user_group": <user_group>,
}
Change 2: functions queue_item_add, queue_item_update and queue_item_add_batch should accept and return items represented using standard schema (set 'item_type' in request, replace 'action' with 'name' for instructions):
# Proposed implementation: example plan representation for `queue_item_add` function
plan_payload = {
"item": {
"item_type": "plan", # Now it is required (currently it is set by the server)
"name": "<plan-name>", # Required
"args": [...], # Optional args
"kwargs": {...}, # Optional kwargs
"meta": {}, # Optional dict or list(dict)
# Optionally 'item_uid', 'user', 'user_group' may be submitted, but the values
# will be replaced by the server.
},
"user": <user_name>,
"user_group": <user_group>,
}
# Proposed implementation: example instruction representation for `queue_item_add` function
instruction_payload = {
"item": {
"item_type": "instruction", # Now it is required (currently it is set by the server)
"name": "<instruction-name>"; # Required
"args": [...]; # Optional args
"kwargs": {...}; # Optional kwargs
"meta": {}, # Optional dict or list(dict)
# Optionally 'item_uid', 'user', 'user_group' may be submitted, but the values
# will be replaced by the server.
},
"user": <user_name>,
"user_group": <user_group>,
}
Change 3: rename return parameters in queue_item_add and queue_item_update. In the current implementation, 'plan' is used as a return parameter name if plan is added and 'instruction' is used if an instruction is added. This is consistent with current implementation of input parameters. It is proposed change the parameter name from 'plan'/'instruction' to 'item' for all types of items.
Change 4: input/output parameters for queue_items_add_batch. Change the representation of the list of items from
[ { "plan": { ... plan1 ... },
{ "plan": { ... plan 2 ... },
{ "instruction": { ... instruction 1 ... } ]
to
[ { ... plan1 ... },
{ ... plan2 ... },
{ ... instruction1 ... } ]
The change is possible, since in the new standard each item is expected to contain item_type parameter which tells if the item is a plan or instruction. The structure of output parameters should also be changed:
# Current structure of output parameters of 'queue_items_add_batch'
{ "success": True, # Success of batch processing (tells if the batch was accepted)
"msg": "", # Error message for the whole batch
"item_list": [
{ "plan": { ... plan1 ...},
"success": True, # Tells if the plan was validated successfully
"msg": "", # Validation error message for the plan
},
{ "instruction": { ... instruction1 ...},
"success": True, # Tells if the instruction was validated successfully
"msg": "", # Validation error message for the instruction
},
]}
# Proposed structure of output parameters of 'queue_items_add_batch'
{ "success": True, # Success of batch processing (tells if the batch was accepted)
"msg": "", # Error message for the whole batch
"items": [ # Regular list of items that could be handled as any other list of items
{ ... plan1 ...},
{ ... instruction1 ...},
]
"results": [ # Item validation results (list of the same size as "items")
{"success": True, "msg": ""},
{"success": True, "msg": ""},
]
In the new set of returned parameters, the item parameters are put in the separate list from validation results. The item list can now be treated as any other item list and the item that failed validation could be easily found by using item index. The only downside of this approach is that the results could be difficult to visually interpret for long lists if the returned results are printed during debugging, but this is not the major concern, since logging output may always be generated in the convenient form.
Change 5: rename return parameter of queue_get request from queue to items for consistency with other API functions.
Change 6: rename return parameter of history_get request from history to items for consistency with other API functions.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent