C++ code generator based on xsdata to generate code base off a schema (YML or XSD).

1. Generate Protobuf converter classes capable of converting a schema defined C++ message class to a protobuf message.
2. Generate C++ Message classes with built in serialization capability based off a schema.
3. Generate .proto files for the Protobuf compiler based off a yaml schema.

Generate Protobuf converters poetry run python -m autogen -t protobuf_converters ./schemas/yaml/sample.yaml

Generate C++ message classes poetry run python -m autogen -t api ./schemas/yaml/sample.yaml

Generate .proto files for the protobuf compiler poetry run python -m autogen -t protobuf ./schemas/yaml/sample.yaml

If you already have an environment set up, or are running inside a built container:

python -m autogen -t [TEMPLATE_TYPE] ./schemas/yaml/sample.yaml

NOTE: if running in a poetry env, prefix commands with poetry run

NOTE: template generation is ADDITIVE meaning existing files in src/ and test/ are not deleted, you should generally delete those directories before generating.

Given an input message/object spec, generate one or more template based sets of code to use or convert data following that spec. Primary support is xsd and yml, see section below about all supported formats.

Templates can live inside the application, or referenced from a location on disk outside the repo. Unlikely you'll have an external set of templates, but you could. See filters.py for the filters or tests that can be used on the variables in template scope (jinja2 speak, docs here).

Each folder under autogen/templates can be considered an output "type" which can be enabled using the -t

The file structure under the first directory or "type" specifies the structure to render the files to. This allows the templates to be stored exactly how they will be rendered (in subdirs). The output directory for non-test classes is src/[expanded namespace]/[template structure]/[filename]

The exception is test, which is rendered based on the structure after test after test/unit

You will notice the filenames are strange, they map to a set of features to define how and when the files are rendered.

These are files that are rendered at most once per package and follow the naming convention :[filter]<[filename_pattern]>.[ext] where the [] are not include, just representing a variable in the string, e.g.: :utils<Clock>.h

For globals, only one filter exists: utils. If set on a file, it will not be rendered in the case were a --utils-ns is specified, meaning this is a required for a utils class, which should be shared for derived api packages.

ignoring the filter, the file <Clock>.h is rendered to Clock.h. the filename_pattern could include python f-string with variables but those variables would have to be passed into tpl.to_path in generator.py to support any variables.

Full template spec, some parts option, see below: _[class_type]:[filter_name]<[filename_pattern]>.[ext]

The _[class_type] prefix is required (again, without "[]", just representing a variable). where class_type is one of: [all, enum, alias, variant, struct]. Except in the case of all, files are specific to the type of class, therefore "struct" class objects only render templates starting with _struct.

If a filter is included (:[filter_name], : not included in the case of no filter), it means that the template has a secondary condition that defines whether it is rendered. filter_name must associate to a method on FilterMethods in overrides/helpers.py, and the response is None if the class should not be rendered, or a dict containing zero or more extra template context params if it should be rendered. e.g. :is_abstract filtered template files are only rendered for abstract base classes and in the context of their template, a new variable derived is available with the list of classes that extend the abstract type.

If a filename_pattern is included (if not, it implies <{type_name}>) then the file generated by the template for a given class is the value within <> supporting f-strings, currently only type_name is available, which is the name of the class.

The ext is required, the shortest filename is _[class_type].[ext].

the api templates have examples of all these features, render the sample Message schema xsd and see the resulting files for an example of it working.

NOTE: This application is intended to be run using poetry, but can be used with a python env or virtual env, use requirements.txt to setup up dependencies, and remove poetry run from the front of all the example commands.

1. Run the installation command curl -sSL https://install.python-poetry.org | python3 -
2. Add Poetry to your PATH export PATH="$HOME/.local/bin:$PATH"
3. Reload your shell configuration
4. Verify the installation poetry --version

Once Poetry is installed, you can install dependencies and create the required virtual environment with a single command:

poetry install

NOTE: If you update pyproject.toml directly and want to update the env, just run poetry update, make sure to keep requirements.txt in sync.

Using poetry add [package] will update pyproject.toml and install the package into your env.

using the poetry env is just like python, except you prefix your commands with poetry run, to run the autogen application: poetry run python -m autogen --help

poetry run python -m autogen -t api -ns api,common::api::cpp ./schemas/yaml/sample.yaml

-t [tpl_dir] is used to specify which specific template directory you would like to use

-ns is a namespace override for autogenerated types. format for usage is [tpl_dir],[namespace]

-nsm allows you to map specific files to specific namespaces and headers with a yaml file. Usage: -nsm [path_to_yaml_file]

-tpl argument for mapping path to template directory. Usage: -tpl [path_to_directory]

The most feature complete input/schema format, fully supported and parsed directly by xsdata before rendering into the output template format.

yaml_mapper.py defines the supported schema transformations that are used for a yaml based format. The trick is to massage a simple yaml format into the structure xsdata uses. Currently the format is loosely defined as follows.

1. The input yaml should be a list of class definitions, each class definition is a yaml dictionary/obj.
2. Each class must have a type, one of: struct, enum, variant, alias, which is stored in the key type.
3. Each class must have a name, assigned to the key name.
4. Each class or field may have a description/documentation embedded in a field, either named help or doc (both are supported)
5. any field with two supported names will clobber each other if you use both, you can see how in yaml_mapper.py

This gets us up to type specific content, ignoring that, we have:

- type: struct
  name: Class1
  doc: A custom class made of fields to use for stuff
  [... struct specific]
- type: enum
  name: Enum1
  [... enum specific]

- type: struct
  name: BaseClass
  fields:
    - type: str
      name: private_key
      repeat: True
- type: struct
  name: SampleStruct
  extends: BaseClass
  fields:
    - type: str
      name: public_key
      required: False
    - type: int
      name: a_number
      optional: True
      doc: A number that might be there

• fields: a list of field defs, defined below
• extends|parent: the name of a class this class extends

• type: the class/type (think class in code speak) of the field
• name: the name of the field, will define the getter/setter/member name
• required(optional): if explicitly False, the field is marked optional, * default=True
• optional(optional): if explicitly True, the field is marked optional, * default=False
• repeat|repeats(optional): if explicitly True, field
• restrictions(optional): dict (key/values) mapping the restrictions fields like min_occurs/max_occurs, see class Restrictions in xsdata.codegen.models for all fields
• help|doc(optional): docs assigned to field

- type: enum
  name: SampleEnum
  base: uint32
  values:
  - SingleStringVal
  - key: AsDict
    value: 10
    doc: Special value when it's a dict?

• base|base_type: type of the enum values, only numeric currently supported
• values: list of Enum values

Either a string name --or--

• key|name: enum name string
• value(optional): currently unused, eventually will support custom values
• help|doc(optional): docs assigned to enum option

- type: alias
  name: SampleAlias
  base: str
  restrictions:
    min_length: 1
    max_length: 10

• base|base_type: type to extend from, usually a native type
• restrictions(optional): dict (key/values) mapping the restrictions fields like min_occurs/max_occurs, see class Restrictions in xsdata.codegen.models for all fields. In the case of Alias, generally used for min/max value constraints or min/max length (strings)

- type: variant
  name: SampleVariant
  types:
  - BaseClass
  - str
  - type: str
    key: SpecialString
    help: This is when the value is a string but it's marked as a different choice with a different name...Quite confusing, but required for OMS :)
  - UUID
  - Datetime

• types: list of variant types

Either a string type name --or--

• type: name of the type associated with the variant choice
• key(optional): a custom key to assign to the type, assigned to the typed accessors, held type checkers, and the Choice enum generated. default = type
• help|doc(optional): variant specific docs

xsdata supports other formats, see their documentation for supported inputs.

• name: [class_name] namespace: [some_namespace] includes:
  • [some_header_file]
  • [some_other_header_file]

1. There's a bit of code that finds your git structure root and would install into the "destination" repo, but didn't get finished, so it's not activated. Requires git on PATH, so doesn't work inside the current container.
2. libclang in python would be an interesting way to auto-detect and generate the conversion code between one of the outputs (api/proto/etc) and an externally defined library. Currently the templates depend on assumptions about naming patterns in external libraries, and are not re-usable for others.