sandflow / regxmllib Goto Github PK

Array with non-zero type size should be have dedicated type kind of FixedArray since they are encoded without element count and size in MXF. This also maps directly to FixedArray definitions in RegXML.

Include command line option to remove namespaces from Essence Descriptor

regxmllib currently requires Java 8

The generated dictionaries cannot have “rootElement” and “rootObject” attributes.

There are issues with the 4-1 source, and the children are not needed in MXF.

Issues are tracked at [1]

[1] https://github.com/palemieux/regxml/issues

is incorrect. It should be:
https://github.com/sandflow/regxmllib/issues

Note that this has been added to the Types Register but Reg-XML itself has not been ammended (yet).

Add 2 sample XML files + RegXML generated from the two sample files so that output of future revisions can be tested against the previous output.

Create 'test' target.

Properties whose definitions are not found should be included as XML comments

The current XSL transform would be complex to adapt to multiple metadictionaries.

FragmentBuilder (applyRule5_3) currently encodes labels as UL. This could be improved by using the Label symbol if available.

I used smpte_registers-bbc_rd_db_exports-201412092156 and generated www-smpte-ra-org-reg-2003-2012.xml

In this I see:

<TypeDefinitionVariableArray>
<Identification>urn:smpte:ul:060e2b34.01040101.04020700.00000000</Identification>
<Symbol>dupIndexEntryArray1</Symbol>
<Name>IndexEntryArray</Name>
<ElementType>urn:smpte:ul:060e2b34.01040101.04100600.00000000</ElementType>
</TypeDefinitionVariableArray>

Why has the symbol changed from what is in Groups.xml (i.e. "IndexEntryArray")?

Both UTF-16BE and ISO-7 (e.g. RFC5646LanguageCode) are present in MXF file.

Add switch to applyRule5_12() based on elementType of String Definition.

• tns:GroupsRegister/tns:Entries/tns:Entry/tns:Contents/tns:Record/tns:UL should be "required"
• change tns:ElementsRegister/tns:Entries/tns:Entry/tns:TypeUL to tns:ElementsRegister/tns:Entries/tns:Entry/tns:Type

This appears to be an artifact of JAXBContext.generateSchema()

When processing smpte_registers-bbc_rd_db_exports-201412092156 I got quite a few warnings of the form:

Jan 08, 2015 3:26:44 PM com.sandflow.smpte.regxml.dict.importer.XMLRegistryImporter fromRegister
WARNING: Missing Target Set UL at Facet null for Type urn:smpte:ul:060e2b34.01040101.05010c00.00000000

What does this mean? Are these warnings genuine? Is there a problems with the Registers entries?

• change elements name that start with “is” to start with “Is” e.g. “IsDeprecated”
• make “IsConcrete” optional because it only applies to Leaves (as per ST 395)
• make everything in the “Contents” optional except for “UL”

Metadictionary should use

http://www.smpte-ra.org/schemas/2001-1b/2013/metadict

for the Extension element defined in ST 2001-1 instead of

http://sandflow.com/ns/SMPTEST2001-1/baseline

Boolean is defined as an "enumeration" {"1 -> true", "0 -> false"} in the register. ST 377 allows any non-zero value to mean "true".

applyRule5_2 in FragmentBuilder should accept any non-zero value to mean "true" for the special type "boolean".

It would be handy to have some "beginner's instructions" in the README to address the following:

• It seems that you need to have git installed and in your PATH environment variable before opening NetBeans
• I think (perhaps) you have to have JDK8 installed (specifically this version?)
• Recommended basic workflow for building / using the tool:
  1. Use NetBeans to build a jar file (no dependencies at all, except as above) by doing ...
  2. Execute the XMLRegistersToDict tool then the GenerateDictionaryXMLSchema tool from the command line (using the jar). Registers XML files are used as the inputs to XMLRegistersToDict.
  3. Execute the RegXMLDump tool to process MXF files

Change "Array" type kind to "VariableArray"
Remove isSizeImplicit from Set type

RegXMLDump ignores K and L length when computing header bytes.

Generate XML schemas for XML Registers and Metadictionaries using JAXB schemagen at build time. Store the resulting schemas under /schemas.

Also make Record.UL mandatory in Groups

E.g.:

Entry namespace: "http://www.smpte-ra.org/reg/335/2012"
XML schema namespace: "http://www.smpte-ra.org/schemas/335/2012"

@reg:uid should be generated by on whether the class contains an element for which isUniqueIdentifier() is true

Is this a concern?

regxmllib\regxmllib\src\main\java\com\sandflow\smpte\klv\adapters\ULValueAdapter.java:40: warning: [unchecked] fromValue(byte[]) in ULValueAdapter overrides <W>fromValue(byte[]) in TripletValueAdapter
    public static UL fromValue(byte[] value) {
  return type requires unchecked conversion from UL to W
  where W is a type-variable:
    W extends Object declared in method <W>fromValue(byte[])
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
1 warning

NetBeans info:

Product Version: NetBeans IDE 8.0.2 (Build 201411181905)
Updates: Updates available to version NetBeans 8.0.2 Patch 2
Java: 1.8.0_25; Java HotSpot(TM) 64-Bit Server VM 25.25-b02
Runtime: Java(TM) SE Runtime Environment 1.8.0_25-b18
System: Windows 7 version 6.1 running on amd64; Cp1252; en_GB (nb)

WeakReference can be either a UL or a UUID in an MXF file

Specifically, should all AAF-defined class 13 symbols use the AAF namespace

In the Registers, ProductReleaseType is incorrectly modelled as an Enumeration of UInt8 rather than UInt16. ProductReleaseType is used by one of the Record members of ProductVersionType. ProductVersionType is used by Identification.ToolkitVersion and Identification.ApplicationVersion (among other Elements -- although most of those are (currently) erroneous uses of this Type...).

I ran regxmllib dump tool build 34efd99 on an MXF file with the final two bytes of Identification.ToolkitVersion set as 0002h. The result was:

<r2:BuildType>VersionUnknown</r2:BuildType>

If ProductReleaseType was correctly modelled then the answer would be:

<r2:BuildType>VersionDebug</r2:BuildType>

(Note: a further complication is that where the Registers currently say "BuildType" they should actually say "Release"...)

It seems that regxmllib finds 10 bytes for fields such as Identification.ToolkitVersion but the meta-dictionary tells it only 9 are needed and so the final byte is silently discarded. My suggestion would be that an error is raised as this is a serious fault (in this case with the Registers, but it could be a problem with the MXF file). In this case, the result of the data length mis-match is that incorrect data is output (i.e. VersionUnknown instead of VersionDebug).

To help with tracking and debugging, add creation date and toolkit build version to XML Registers and MetaDictionaries as XML comments.

I assume (from looking at various MXF files I've tried) that's the cause of this output:

No Primer Pack found
Exception in thread "main" java.lang.NullPointerException
        at com.sandflow.smpte.klv.LocalSet.fromTriplet(LocalSet.java:113)
        at com.sandflow.smpte.tools.RegXMLDump.main(RegXMLDump.java:176)

This situation is permitted by ST 377-1:2011 Section 9.1. Note that in this MXF file it is a "legacy" KLV Fill key that is used (for KLV Fill byte 8 of the Key needs to be ignored).

I think that KLV Fill bytes not being correctly counted is probably also causing the following warning on a different MXF file:

WARNING: Index Table Segment encountered before Header Byte Count bytes read.

ST 377-1:2011 Section 9.1 notes that the KLV Fill at the end of the Header Metadata is included in the value of HeaderByteCount

SMPTE 30MR has deprecated the spreadsheet versions of the registries.

This would make it more human readable.

The namespace should be something like http://www.smpte-ra.org/schemas/2001-1c/2013 (we may also want a suffix on this (but "metadict" would not be appropriate)). Not sure if it should be 2013 or not...