Restructure header of descriptor file about julie HOT 3 OPEN

Hi @christophschubert thanks a lot for your help here. I love to kick an improved design of the descriptor file, however I think this deserve a bit more thinking.

I want to provide a bit more of answers and context here:

1.- we cannot add additional meta-data fields to the 'header' of a topology YAML, at least not between context and projects.

yes, you can. Between the context and the project tag, the list of elements in the deserialiser is open. This is an example: https://github.com/purbon/kafka-topology-builder/blob/master/example/descriptor-with-schemas.yaml

2.- This goes against the YAML spec (https://yaml.org/spec/1.2/spec.html), see quote below. Which in turn means we have to rely on hand-written parsers.

This makes more sense, I was not aware of this design principle in yaml.

I would say, lets give thought to this, with as much people as possible chimming in, so we cover the most out of it.

I think this is an amazing thing to merge in for the upcoming 2.x branch. I would love to release as soon as possible an stable (as much as possible 1.x branch)

from julie.

1.- we cannot add additional meta-data fields to the 'header' of a topology YAML, at least not between context and projects.

yes, you can. Between the context and the project tag, the list of elements in the deserialiser is open. This is an example: https://github.com/purbon/kafka-topology-builder/blob/master/example/descriptor-with-schemas.yaml

But these become part of the topic-names. I was thinking about user-defined meta-data which should not become part of the topic names.

from julie.

Hi @christophschubert, The meta-data part of your enhancement request in this issue has been covered by the solution of #199, leaving mostly open only for this issue the yaml spec compliance

from julie.