Found last value Cursor{primaryCursor='null', secondaryCursor='null'} java.lang.NullPointerException about kafka-connect-elasticsearch-source HOT 7 OPEN

I've seen the code and the incrementing.field.name is not compatible yet with nested documents. At this time it works only with the root level fields.

I hope you can find a workaround or a new incrementing field, in the meantime I'm planning to add the compatibility for nested fields also for cursors when finalizing the ssl config branch.

from kafka-connect-elasticsearch-source.

Hi

Any update on this?

On all versions above 1.4, I am facing this issue even when my incrementing.field.name is not a nested field. Following is the connector configuration:

{
             "connector.class":"com.github.dariobalinzo.ElasticSourceConnector",
             "tasks.max": "1",
             "es.scheme": "http",
             "es.host" : "somehost",
             "es.port" : "someport",
             "es.user" : "username",
             "mode":"incrementing",
             "es.password" : "password", 
             "index.prefix" : "index",
             "topic.prefix" : "topic",
             "incrementing.field.name" : "received_on",
             "poll.interval.ms" : 10000

}

from kafka-connect-elasticsearch-source.

Hi @armaankohli91 , sorry for the delay, I will work on it in the next days and I will update here in the issue.
Can you provide a sample document json, especially of the incrementing field that is failing?

Is that an array? array also are not supported

Thanks,
Dario

from kafka-connect-elasticsearch-source.

Hi @armaankohli91 , sorry for the delay, I will work on it in the next days and I will update here in the issue. Can you provide a sample document json, especially of the incrementing field that is failing?

Is that an array? array also are not supported

Thanks, Dario

Hi Dario

Thanks for the response.

No it is not an array. Its a plain number that just increments with each document.

Sorry I misunderstood by what you meant by sample document. Editing the reply to have the same here:

{
                    "da_name": "Rihanna",
                    "da_firstName": "Rihanna",
                    "da_lastName": "Rihanna",
                    "da_address": "TX",
                    "ra_f1": "12",
                    "ra_H1": "11",
                    "created_on": "2022-02-02T16:38:53.0870292",
                    "received_on": 97568911026700
                }

So, received_on is a field that stores current milliseconds while saving the document.

Hope it helps.

-Armaan

from kafka-connect-elasticsearch-source.

hi there ,i am facing same problem .Did anyone find the solution?

from kafka-connect-elasticsearch-source.

Hey, is this issue still open ? I am facing this exact same error when i try to use a timestamp keyword field as the incrementing field in the Kafka connector configuration.

from kafka-connect-elasticsearch-source.

Looks like the nested fields for the incrementing.field.name is still not functional. This is a serious limitation for this great piece of software

from kafka-connect-elasticsearch-source.