- 0.2.1
- Add field not set hooks
- 0.2.0
- Add before/instead/after field set hooks
- 0.1.1
- Add Validators
- 0.1.0
- Initial version
QuickScheme is a quick and easy to map arbitrary data in any python dict to an object without forcing markup of the data file.
This allows easy data definition in YAML or JSON files including references, defaults, etc
- Easily load standard python data (from python dict, or json or yaml)
- Allow use of the data directly in Python code
- Produce "inflated" data with default values and references populated
- Do all of this without relying on custom markup in the source data
NOTE: This example exists in examples directory as "basic.py" |
---|
To get the most out of QuickScheme you need to define the objects your schema consists of.
for example if we want to process a yaml file like this:
version: 1
updates:
# this is our update log to demonstrate lists
- 2019-08-14: initial version
- 2019-08-15: added user3
- 2019-08-15: added project2
users:
user1:
first_name: User
last_name: One
user2:
first_name: Another
last_name: User
email: [email protected]
desc: Another User
user3:
first_name: Another
last_name: User
email: [email protected]
desc: Another User
groups:
users:
desc: Regular Users
admins: Admins
projects:
project1:
desc: My First Project
users:
- user1
groups:
- admins
project2:
desc: My Other Project
groups:
- users
So, what we have here is a version of the file, a list of users and groups and list of projects. A few interesting things:
- admin group is defined by a string instead of a mapping
- Users contain references to groups by a key
- Similarly projects contain references to both users and groups
So, to be easily this we define our objects that represent the entities:
Let us define Group first:
class Group(SchemeNode):
FIELDS = [
Field('groupname', identity=True),
Field('desc', brief=True, default='No Description')
]
What is happening here:
- We define an object called Group which extends our SchemeNode(a field defined mapping) with two
fields:
- groupname - which is an identity field, meaning it is populated from the id of the field. If this object is in a mapping, this would be populated from the item's key, if it is in a sequence, it will be populated with sequence index number
- desc - our description. We specify a default value. We also mark it as our brief field, meaning if the item is specified as a string instead of a mapping, we will populate this field
This takes care of how we defined admin group and allows us to know the groupname if we are working with this object directly in python.
Next lets define User
class User(SchemeNode):
FIELDS = [
Field('username', identity=True),
Filed('userid', ftype=int, required=True),
Field('first_name', default="", required=True),
Field('last_name', default="", required=True),
Field('email', default="", required=False),
Field('desc', default="No Description", required=False),
Field('groups', ftype=ListOfReferences(Group, ".groups", False),
default=[], required=False),
]
What is happening here:
- Similar to
Group
object we extendedSchemeNode
to create a field based mapping in which:- We defined identity field
username
- We defined an integer field
userid
- and made it a required field - We added two more required fields
first_name
andlast_name
- which are both strings because ftype is omitted - We added optional
email
anddesc
fields - with latter having a default value - And we added a
groups
field, wich is a list of references to Group objects which are to be resolved against 'groups' subkey of the root document.
- We defined identity field
Lastly We need a Project object:
class Project(SchemeNode):
FIELDS = [
Field('projectname', identity=True),
Field('order', ftype=int, required=True),
Field('users', ftype=ListOfReferences(User, ".users"),
default="", required=True),
Field('groups', ftype=ListOfReferences(Group, ".groups"),
default=[], required=False),
]
This is similar to previous objects
Now to put it all together, we create a root Object that represents the document:
class Data(SchemeNode):
FIELDS = [
Field('version', ftype=str, default='1'),
Field('updates', ftype=ListOfNodes(str)),
Field('groups', ftype=KeyBasedList(Group)),
Field('users', ftype=KeyBasedList(User)),
Field('projects', ftype=KeyBasedList(Project))
]
PRESERVE_ORDER = True
ALLOW_UNDEFINED = True
Here we have the same thing describing the root document:
version
field is a simple stringupdates
field is a List of Nodes - where each node is a simple string.groups
,users
, andprojects
field are each KeyBasedList - or list of objects mapped by their identity. The difference between KeyBasedList and SchemeNode is that all child nodes are of the same type and all keys are identities of those nodes.- In addition, we define a few more properties for this node:
PRESERVE_ORDER
suggests to attempt to preserve the order of the keys in this objectALLOW_UNDEFINED
tells the parser that if it encounters a key that in not a defined field, store it as a plain value. If not set to True, QuickScheme will throw an exception
Currently implemented Node Types:
-
SchemaNode - A basic mapping with pre-defined fields.
- Options
FIELDS
- a list of Fields (see Fields bellow)ALLOW_UNDEFINED
(Default: False) - if true, store fields that are not definedMAP_CLASS
(Default: None) if set, force use of this dict subclass.PRESERVE_ORDER
(Default: False) if set to true, attempt to preserve order of fieldsBRIEF_OUT
(Default: False) - if set to true, attempt to use brief format if brief field exists and is the only field that is set
- Options
-
KeyBasedListNode - A list of nodes presented as a mapping, where all nodes are of the same type, and the key in the mapping is the identity field for each node.
- Options
TYPE
- type of nodes contained in this list
- Helper Functions
- KeyBasedList(item_type) generates a KeyBasedListNode class with TYPE set to
item_type
. For example, to create a list of Group nodes keyed by their id, useKeyBasedList(Group)
- KeyBasedList(item_type) generates a KeyBasedListNode class with TYPE set to
- Options
-
ListOfNodesNode - Basic sequence of items (list or array) where each of the items is of same type
- Options
TYPE
- type of nodes contained in this List
- Helper Functions
- ListOfNodes(item_type) generates a ListOfNodesNode with TYPE set to
item_type
. For example, to create a list of Group nodes useListOfNodes(Group)
- ListOfNodes(item_type) generates a ListOfNodesNode with TYPE set to
- Options
-
ReferenceNode - A Node that stores an ID that references another node stored in KeyBasedListNode object somewhere in document
- Options
PATH
- Path to the KeyBasedListNode in the document. Path uses '.' as separator.DEREFERENCE
- (Default: True) If set to true, dereference the item on conversion to data. If False, then when converting to data, return only the id field.
- Helper Functions
- ListOfNodes(item_type) generates a ListOfNodesNode with TYPE set to
item_type
. For example, to create a list of Group nodes useListOfNodes(Group)
- ListOfNodes(item_type) generates a ListOfNodesNode with TYPE set to
- Options
Definitions of key-based fields
- Field(name, [options]) Basic field definition
- Options:
name
- (string) name of the field key in the parentftype
- (Class) - Field Type. Default is string May be any class that can be initialized with the data or a derivative ofSchemeBaseNode
required
- (bool) - If true, this field must be specified explicitly.)default
- (string or callable) - If specified, value to use if field is not specified. If value is a callable, it is executed with the object as argumentidentity
- (bool) - if true, this fields is set when identity is set. By default, only one identity field is allowed to be present.brief
- (bool) - if True, this field is set when "brief" format (i.e. specifying value) as a value instead of a mapping) - By default only one brief field is allowed. If more complicated processing is required, do not set this field and instead implement_brief_set(data)
method in your class deriving fromSchemeNode
always
- (bool) - If true(default) show this field when asking for data even if not set. If false only show if explicitly setvalidator
- (callable) - If provided, called to validate this field. Must takeFieldValue
object as first parameter
- Options:
Things that may be documented but not yet implemented or are in works:
(right now this is a very open list, many things are not yet done)
-
Node
specifications- ???
-
Field
specifications- ???