Opinions and ideas about XML <=> JSON transformations about specifications-its-rest HOT 20 CLOSED

1. @xsi:type vs @Class - i think others mentioned they would prefer 'type' - my suggestion would be @type to mark it as meta-data rather than an attribute of the model.
2. I would prefer not to use @ for what are attributes in xml. archetype_node_id is just a field (trying to avoid using term attribute not to mix it up with xml attr) in the RM just like 'name' for example. I would rather model JSON based on RM itself instead of using XML representation of RM. Why would one have field 'name' as regular JSON attribute and 'archetype_node_id' prefixed with a '@'.

from specifications-its-rest.

The XML->JSON mapping convention described at https://github.com/beckchr/staxon/wiki/Mapping-Convention might be good for representaing XML faithfully, but it is not neccesarily the most obvious (or most compact or readable) way to represent the openEHR object structure. But I guess things can be configured to make things look better.

@ppazos do you have link to an example of openEHR json instances created using staxon?

from specifications-its-rest.

@ErikSundvall no, we took particular care on that one (actually I did in the early 2000s, because I put everything into an Eiffel project and Eiffel was (is) dumb w.r.t. namespacing, so I had to do the old-school thing and make every class globally unique in the entire model.)

from specifications-its-rest.

@bostjanl your are right, I was thinking only on LOCATABLE and biased by the XML representation. Thinking of single value values on DATA_VALUE I agree those shouldn't have names starting in @, so to be consistent, also other single value attributes, including archetype_node_id, shouldn't have the @ prefix.

from specifications-its-rest.

I think discussions and principles describe here should be kept independent of the programming language or platforms - we need to focus specifications, not implementation details.

from specifications-its-rest.

@sebastian-iancu +1 this is about the transformation rules. I mentioned the tool because it has it's own rules we can borrow for the specs so we don't need to reinvent the wheel.

from specifications-its-rest.

Yes, let's not forget we're not doing XML as JSON - we're doing JSON natively. I would for sure advise against doing all the namespaces, etc. Let's have a typical Javascript front-end developer in mind as our customer.

from specifications-its-rest.

I think one thing is clear re: JSON - we need an openEHR-wide standardised approach to JSON, which just means stating a few very simple rules - how type names are included. Various approaches:

  // Type name as a property, same as above
  {
    "@type" : "Employee",
     ...
  }

  // Wrapping class name as pseudo-property (similar to JAXB)
  {
    "com.fasterxml.beans.EmployeeImpl" : {
       ... // actual instance data without any metadata properties
    }
  }

  // Wrapping class name as first element of wrapper array:
  [
    "com.fasterxml.beans.EmployeeImpl",
    {
       ... // actual instance data without any metadata properties
    }
  ]

Questions to consider:

• the type may be generic, e.g. "DV_INTERVAL<DV_QUANTITY>"?
  • my view: we use proper types names, which may include generics; we understand "@type" to mean type, distinct from "@Class";
• use openEHR canonical type names, i.e. upper snake-case, or use implementation type names, which will be down to whatever generated the JSON. C-based languages, it's pure CamelCase; Ruby and Python use snake_case names and CamelCase class names.
  • my view: for JSON that is used for interop either of data or of design time artefacts, we stick to SCREAMING_SNAKE_CASE for classes and lower_snake_case for methods, etc. Aesthetically, if we had our time again, I think the Ruby/Python CamelCaseClass/method_snake_case convention would be nicer, but... I think changing now is probably not a great idea. Also, if SCREAMING_SNAKE_CASE is detected, you know its a canonical artefact of some kind.

FWIW I don't think FHIR's JSON type convention is a good one, or usable outside FHIR.

from specifications-its-rest.

+1 Agree on SCREAMING_SNAKE_CASE for classes and lower_snake_case for methods/attributes just as in XML-spec and in openEHR specs.

Do we have any class names being reused in different packages? Hopefully not, so that we can use short simple name without package prefixing or other special solutions.

[Edit:
Has anybody looked at what the Archie openEHR implementation does?
Does Marand have a spec of current approach?
]

from specifications-its-rest.

Attached is an example of canonical openEHR XML in a list, and it's saxon transformed JSON format.

One key aspect is how element attributes are handled just like subelements but with the name starting with @, as an example:

"version": {
            "@xsi:type": "ORIGINAL_VERSION",

compo.xml.txt

compo.json.txt

from specifications-its-rest.

Our current JSON format is described here: https://openehr.atlassian.net/wiki/display/spec/AQL+Result+Set+work+area, this page was started to agree on result set, but there is a description of the JSON format there as well.

from specifications-its-rest.

See https://github.com/freshehr/openehr_json for instance examples of a lightly templated
Adverse reaction archetype in canonical XML format and in Marand's 'RAW' JSON format - their equivalent of canonical.

@ppazos - can you and others possibly use that to demo the different formats? I will also upload the .opt to make that easier. Happy to transfer the files to another location but Github does make them easily accessible.

That will let us compare/contrast more directly, I think.

from specifications-its-rest.

@freshehr at a glance the first two differences I see is:

CaboLabs vs. Marand's

1. "@xsi:type": "DV_TEXT" vs. "@Class": "DV_TEXT"
2. "@archetype_node_id": "openEHR-EHR-OBSERVATION.blood_pressure.v1" vs. "archetype_node_id": "openEHR-EHR-EVALUATION.adverse_reaction_risk.v1"

I think in 1. the JSONs are equivalent, and 2. @ should be use as prefix, since archetype_node_id is an attribute on the XML schema and a simple datatype attribute in the IM. If it was a complex object I would agree naming it without the @ prefix.

I don't see big structural differences, just data level (just node naming by now), but I'll need time to analyze it in detail.

from specifications-its-rest.

Have to be careful here on the types. 'xsi:type' is XML specific, and XML-derived JSON has to be distinguished from proper canonical data. That's because XSD types do in general differ from RM types, e.g. generic types are not supported in XSD and have to be faked (DV_INTERVAL_DV_QUANTITY etc) and basic types are not the same either. '@Class' is a different concept again.

So it depends on what data you are putting in the JSON? Converted from XML? True canonical? Something else?

from specifications-its-rest.

BTW, the last comment made me think about the current XSD. Maybe we can modify the XSD in a way that automatic JSON-XML converters can convert the canonical XML into the canonical JSON without customizations. One thing would be to represent current XML attributes, as the mentioned archetype_node_id, as XML elements.

(I know we also need add classes to the XSD, and I remember I made a proposal to allow contribution have many version child elements, right now is the version that has a reference to a parent contribution, I think this in on JIRA). I can open another issue to discuss about XSD improvements.

What do other's think? Should we review the XSD?

from specifications-its-rest.

I think we could close this issue now after the meeting discussion at https://openehr.atlassian.net/wiki/display/spec/2016-06-23+Meeting+notes where we concluded:

JSON format

• use @type = "DV_TEXT"
• for parametrised: @type = DV_INTERVAL<DV_QUANTITY>
• Attribute names are lowercase RM names - snakecase
• Any other meta-data attribute should start with a '@'

Datetime format

Recommendation: Canonical ISO8601 expanded format, i.e. 2016-06-23T13:42:16.117+02:00
But we accept any ISO8601. It should also preserve what was sent (i.e. if composition was saved as narrow format we should also return it that way)

from specifications-its-rest.

@ppazos - with respect to review of the XSD, the primary driver is the RM. We have to issue a 1.0.3 XSD (or maybe a 1.0.4) if we issue that RM release before getting the 1.0.3 mods done to the XSD.

We may invent (I think it is likely) other flavours of XSD for other purposes, e.g. one for more space-efficient XML, one for use with certain Java libs or whatever.

The current PRs for the ITS component (owner of the XSDs) are here; the CRs are here.

Any issue that is not already there can be raised as a new PR.

from specifications-its-rest.

@wolandscat what do you think of updating the XSD to represent current attributes as elements?

That representation is arbitrary and doesn't depend on the IM, since in the IM all the attributes are objects, and in the XML, all objects can be represented as elements without attributes. Is up to us what attributes or elements do we model in the XSD.

One specific attribute that can be an element in the XSD is the archetype_node_id.

I know that the XSD xsi:type should be an attribute in the XMLs. And if we have all our attributes as elements, the representation of "xsi:type" from XML as "type" or "@type" in JSON might be the only custom rule for canonical XML-JSON transformers.

from specifications-its-rest.

The main decider on the XSD is maintaining backward compatibility with existing data. But - there is nothing to stop openEHR publishing more than one XSD, for different purposes.

Having said that, many years ago I also got the idea that XSDs should not use XML attributes, so that the data would be more object-oriented. We tried that for a while, but it became clear that it's the wrong way to think about XSD. XSD isn't a modelling formalism, and is very badly designed, so it can never be (inheritance doesn't work properly, generics don't work at all). In fact, it has only one purpose and meaning: being the specifier for XML text/documents. Nearly everyone wants XML documents to be as small as possible. To achieve that, you want to use XML attributes more, not less.

Now that things have moved on, and JSON is commonly used, there may be an argument for an XSD that mimics the JSON, and this could be as you say - pretty much all elements except items like xsi:type. Note that this will generate huge XML documents. On the other hand, it will make JSON / XML transformation easy.

So if we want to pursue this, I would strongly suggest publishing it as new schema that defines 'JSON-XML' or something, so people get what it is for. We might as want to consider a converter between that and the canonical XML (i.e. the current XSD).

from specifications-its-rest.

@wolandscat I will create an element-only XSD so others can let us know if this causes any data incompatibility. Personally, I don't think this should cause any issues since most implementations use XML for interchange, not for storage, so this change won't affect data stored, just the serializers/unserializers.

I don't think XSD should be used as way to model objects, for me is just a way to model documents, and objects can be serialized to documents. UML is for objects, and for me ADL/AOM is like an UML especialization with a processable format. IMO there are cases where OO attributes can be mapped to XSD attributes, and other cases where using XSD elements is better for OO attributes.

IMO XML and JSON serializations should be independent. I'm not sure about "XSD that mimics JSON". My proposal is to use XSD elements to reduce the custom @... keys in the JSON documents, because that is custom for XML-JSON conversion. It would be cleaner and more independent if the JSON format doesn't include any @... key.

But thinking about JSON, did someone investigated JSON schema?

http://json-schema.org/
https://spacetelescope.github.io/understanding-json-schema/
http://jsonschema.net/#/
http://www.jsonschemavalidator.net/

I would love to have both: non-attribute XSD for XML and a JSON schema (designed in a way that is simpler to map to the XSD, but independent).

from specifications-its-rest.