question: '-' removed from the attributes of the JSON value about kafka-connect-elasticsearch-source HOT 5 CLOSED

Hey @konstan,
Yes, it is intentional. I'm converting the names in order to be compatible with the Avro serialization (see here: https://avro.apache.org/docs/1.8.1/spec.html#names).

Is this a blocking problem for you?
Actually, I might enable a config to turn on/off this name conversion if you think this might be useful (the conversion logic is isolated in a single Java class ).

from kafka-connect-elasticsearch-source.

Hi @DarioBalinzo ,

Yes, our code generates loads of attributes with dashes. It would be simpler for the processing downstream if the attributes wouldn't change. If it's possible, a configuration parameter would be highly appreciated.
Thank you!

from kafka-connect-elasticsearch-source.

Hi @feed3r, would you like to help me in developing this issue?

Please make AvroName class non static, and create a new interface for its public methods (calling the new interface something like FieldNameConverter).
Then you can create also a NopNameConversion class (that implements FieldNameConverter) that is not changing the original name.

Finally in the ElasticSourceTask class initialize both SchemaConverter and StructConverter, passing in the constructor an instance of AvroName
or NopNameConversion depending on the user configuration.

For testing, please refer to SchemaConverterTest and ElasticSourceTaskTest files.

from kafka-connect-elasticsearch-source.

Hi @konstan,
we just released version 1.2 with the new config.

You can find the description of the new param in the readme (fieldname_converter).

from kafka-connect-elasticsearch-source.

@DarioBalinzo I validated the feature and it works correctly in my case. Thanks a lot!

from kafka-connect-elasticsearch-source.