I followed the tutorial on https://www.eclipse.org/ditto/2018-10-16-example-mqtt-bidirectional.html

only difference is i am send data via python

i get the error on "Connection Log Details"
Type: mapped
{
"correlationId": "",
"timestamp": "2023-04-13T09:18:14.701242933Z",
"category": "source",
"type": "mapped",
"level": "failure",
"message": "Got exception <json.field.missing> when processing external message with mapper : JSON did not include required field! - Message headers: [mqtt.qos=0, mqtt.retain=false, mqtt.topic=ditto-tutorial/my.test:octopus] - Message payload: {"temp": 25.79, "alt": 329.78, "thingId": "my.test:octopus"}",
"address": "ditto-tutorial/#"
}

type consumed on "Connection Log Details"
{
"correlationId": "",
"timestamp": "2023-04-13T09:18:14.611532107Z",
"category": "source",
"type": "consumed",
"level": "success",
"message": "Received signal. - Message headers: [mqtt.qos=0, mqtt.retain=false, mqtt.topic=ditto-tutorial/my.test:octopus] - Message payload: {"temp": 25.79, "alt": 329.78, "thingId": "my.test:octopus"}",
"address": "ditto-tutorial/#"
}

under Crud Connections template is

{
  "id": "mqtt-example-connection-123",
  "name": null,
  "connectionType": "mqtt",
  "connectionStatus": "open",
  "uri": "tcp://test.mosquitto.org:1883",
  "sources": [
    {
      "addresses": [
        "ditto-tutorial/#"
      ],
      "consumerCount": 1,
      "qos": 0,
      "authorizationContext": [
        "nginx:ditto"
      ],
      "headerMapping": {},
      "replyTarget": {
        "address": "{{header:reply-to}}",
        "headerMapping": {},
        "expectedResponseTypes": [
          "response",
          "error"
        ],
        "enabled": true
      }
    }
  ],
  "targets": [
    {
      "address": "ditto-tutorial/{{ thing:id }}",
      "topics": [
        "_/_/things/twin/events",
        "_/_/things/live/messages"
      ],
      "qos": 0,
      "authorizationContext": [
        "nginx:ditto"
      ],
      "headerMapping": {}
    }
  ],
  "clientCount": 1,
  "failoverEnabled": true,
  "validateCertificates": true,
  "processorPoolSize": 1,
  "mappingDefinitions": {
    "javascript": {
      "mappingEngine": "JavaScript",
      "options": {
        "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {\r\n    const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));\r\n    const jsonData = JSON.parse(jsonString); \r\n    const thingId = jsonData.thingId.split(':'); \r\n    const value = { \r\n        temp_sensor: { \r\n            properties: { \r\n                value: jsonData.temp \r\n            } \r\n        },\r\n        altitude: {            \r\n            properties: {                \r\n                value: jsonData.alt            \r\n            }        \r\n        }    \r\n    };    \r\n    return Ditto.buildDittoProtocolMsg(\r\n        thingId[0], // your namespace \r\n        thingId[1], \r\n        'things', // we deal with a thing\r\n        'twin', // we want to update the twin\r\n        'commands', // create a command to update the twin\r\n        'modify', // modify the twin\r\n        '/features', // modify all features at once\r\n        headers, \r\n        value\r\n    );\r\n}",
        "outgoingScript": ""
      }
    }
  },
  "tags": []
}

i tried with and without the Incoming JavaScript Mapping code

function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {
    const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));
    const jsonData = JSON.parse(jsonString); 
    const thingId = jsonData.thingId.split(':'); 
    const value = { 
        temp_sensor: { 
            properties: { 
                value: jsonData.temp 
            } 
        },
        altitude: {            
            properties: {                
                value: jsonData.alt            
            }        
        }    
    };    
    return Ditto.buildDittoProtocolMsg(
        thingId[0], // your namespace 
        thingId[1], 
        'things', // we deal with a thing
        'twin', // we want to update the twin
        'commands', // create a command to update the twin
        'modify', // modify the twin
        '/features', // modify all features at once
        headers, 
        value
    );
}

my python code that i used

note i tried with
sendData = '{"temp": "+str(round(random.uniform(0.0, 50.0),2))'+', "alt": '+str(round(random.uniform(300.0, 350.0),2))+', "thingId": "my.test:octopus"}'
and got same error above

import paho.mqtt.client as mqtt
import time
import random


connected = 0

# Define callback functions for connection and messages
def on_connect(client, userdata, flags, rc):
    print("Connected with result code ")
    global connected
    connected = 1
    client.subscribe("ditto-tutorial/my.test:octopus")


def on_message(client, userdata, msg):
	print("recieved")
	print(msg.topic+" "+str(msg.payload))



broker = "146.64.8.98"
username = "kapua-sys"
userPassword = "kapua-password"


# Connect to the Mosquitto broker
client = mqtt.Client("rinelpython") 

# client.username_pw_set(username=username,password=userPassword)
client.on_message=on_message
client.on_connect=on_connect

client.on_connect = on_connect

client.on_message = on_message

client.connect("test.mosquitto.org")
client.loop_start()


# Publish a message to the topic
# client.publish("test/topic", "123456789")

# Loop indefinitely to receive messages
# client.foreverloop()

time.sleep(1)

print("before while")
while True:
	time.sleep(3)
	
	if (connected == 1):

		sendData = "{\"temp\": "+str(round(random.uniform(0.0, 50.0),2))+", \"alt\": "+str(round(random.uniform(300.0, 350.0),2))+", \"thingId\": \"my.test:octopus\"}"
		print("messages - ",sendData)
		client.publish("ditto-tutorial/my.test:octopus", sendData)
	else:
		# client.connect("test.mosquitto.org", 1883, 60)
		print("retry")
client.loop_forever()

I am succefully receving data via MQTT (i followed the example https://github.com/eclipse-ditto/ditto-examples/tree/master/mqtt-bidirectional the features are being update on ditto, i have another device that is subscribing to a topic that ditto is supposed to be sending the data to but im not recieving it

how do i configure ditto to send data out?

so i can get the data from ditto and not the actual device, are these the topics?
"//things/twin/events",
"//things/live/messages"

here is the connection config

{
  "id": "ed25e85e-a076-4e9e-9463-12ca5a1ca723",
  "name": null,
  "connectionType": "mqtt",
  "connectionStatus": "open",
  "uri": "tcp://test.mosquitto.org:1883",
  "sources": [
    {
      "addresses": [
        "ditto-tutorial/#"
      ],
      "consumerCount": 1,
      "qos": 0,
      "authorizationContext": [
        "nginx:ditto"
      ],
      "headerMapping": {},
      "payloadMapping": [
        "javascript"
      ],
      "replyTarget": {
        "address": "{{header:reply-to}}",
        "headerMapping": {},
        "expectedResponseTypes": [
          "response",
          "error"
        ],
        "enabled": true
      }
    }
  ],
  "targets": [
    {
      "address": "ditto-tutorial/{{ thing:id }}",
      "topics": [
        "_/_/things/twin/events",
        "_/_/things/live/messages"
      ],
      "qos": 0,
      "authorizationContext": [
        "nginx:ditto"
      ],
      "headerMapping": {}
    }
  ],
  "clientCount": 1,
  "failoverEnabled": true,
  "validateCertificates": true,
  "processorPoolSize": 1,
  "mappingDefinitions": {
    "javascript": {
      "mappingEngine": "JavaScript",
      "options": {
        "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {\r\n    const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));\r\n    const jsonData = JSON.parse(jsonString); \r\n    const thingId = jsonData.thingId.split(':'); \r\n    const value = { \r\n        temp_sensor: { \r\n            properties: { \r\n                value: jsonData.temp \r\n            } \r\n        },\r\n        altitude: {            \r\n            properties: {                \r\n                value: jsonData.alt            \r\n            }        \r\n        }    \r\n    };    \r\n    return Ditto.buildDittoProtocolMsg(\r\n        thingId[0], // your namespace \r\n        thingId[1], \r\n        'things', // we deal with a thing\r\n        'twin', // we want to update the twin\r\n        'commands', // create a command to update the twin\r\n        'modify', // modify the twin\r\n        '/features', // modify all features at once\r\n        headers, \r\n        value\r\n    );\r\n}",
        "outgoingScript": ""
      }
    }
  },
  "tags": []
} 

Hi,
I am trying to implement the custom ditto mapper, while to run the connectivity part added volume with jar using the given yaml I unable to create connection it raise an 408 timeout error.
can you please help me to fix this issue. Thank you

hello, am new to eclipse ditto and Eclipse Mosquitto. I have followed the following process to create a bidirectional MQTT. https://github.com/eclipse/ditto-examples/tree/master/mqtt-bidirectional
Am getting a Payload mapping error .
the code to create MQTT Connection is
{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:createConnection", "connection": { "id": "mqtt-example-connection-1234", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://localhost:1883", "sources": [{ "addresses": ["ditto-tutorial/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "ditto-tutorial/{{ thing:id }}", "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }] } } }
the responce while creating MQTT Connection is

Code for creating payload mapping is as follow
{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:modifyConnection", "connection": { "id": "mqtt-example-connection-1234", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://localhost:1883", "sources": [{ "addresses": ["ditto-tutorial/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "ditto-tutorial/{{ thing:id }}", "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingContext": { "mappingEngine": "JavaScript", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId; const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude: { properties: { value: jsonData.alt } } }; return Ditto.buildDittoProtocolMsg('my.test', thingId, 'things', 'twin', 'commands', 'modify', '/features', headers, value);}" } } } } }
the responce from payload mapping is :

Can you suggest me what to do?

i am new to ditto, i was following the step to create IoT-Device connected directly to Ditto via MQTT
( https://github.com/eclipse/ditto-examples/tree/master/mqtt-bidirectional )
When i tried to create a policy by using Postman i got an error saying 502 Bad Gateway nginx ,
i first installed eclipse ditto, then i run eclipse Mosquitto on docker container on my machine , i am using windows 10.
please suggest me how can i resolve it?

How can I maintain the session of ditto?

I perform the following:
curl -X POST -u devops:foobar -H 'Content-Type: application/json' -d ' { "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:modifyConnection", "connection": { "id": "mqtt-example-connection-123", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://test.mosquitto.org:1883", "sources": [{ "addresses": ["ditto-tutorial/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "ditto-tutorial/{{ thing:id }}", "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingContext": { "mappingEngine": "JavaScript", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId.split(':'); const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude: { properties: { value: jsonData.alt } } }; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], 'things', 'twin', 'commands', 'modify', '/features', headers, value);}" } } } } } ' http://localhost:8080/devops/piggyback/connectivity?timeout=60000ms

and get the following response:
{"?":{"?":{"type":"devops.responses:errorResponse","status":400,"serviceName":null,"instance":null,"payload":{"status":400,"error":"connectivity:connection.configuration.invalid","message":"The message mapper configuration failed due to: syntax error (incomingScript#1) - in line/column #1/242, source:\nfunction mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId.split(:); const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude: { properties: { value: jsonData.alt } } }; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], things, twin, commands, modify, /features, headers, value);}","description":"Check the configuration options of your mapper for errors."}}}}

Can anyone please help?

We have been struggling with this issue as mentioned below. Can you please let us know what's wrong?

The message mapper configuration failed due to: unterminated regular expression literal (incomingScript#1) - in line/column #1/537

We created a connection with message mapping:

curl -X POST 'http://localhost:8080/devops/piggyback/connectivity?timeout=10000' -u 'devops:foobar' -H 'Content-Type: application/json' -d '{

"targetActorSelection": "/system/sharding/connection",
"headers": {
"aggregate": false
},
"piggybackCommand": {
"type": "connectivity.commands:createConnection",
"connection": {
"id": "mqtt-example-connection-123",
"connectionType": "mqtt",
"connectionStatus": "open",
"failoverEnabled": true,
"uri": "tcp://test.mosquitto.org:1883",
"sources": [{
"addresses": ["ditto-tutorial/#"],
"authorizationContext": ["nginx:ditto"],
"qos": 0,
"filters": []
}],
"targets": [{
"address": "ditto-tutorial/{{ thing:id }}",
"topics": [
"//things/twin/events",
"//things/live/messages"
],
"authorizationContext": ["nginx:ditto"],
"qos": 0
}],
"mappingContext": {
"mappingEngine": "JavaScript",
"options": {
"incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId; const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude: { properties: { value: jsonData.alt } } }; return Ditto.buildDittoProtocolMsg('my.test', thingId, 'things', 'twin', 'commands', 'modify', '/features', headers, value);}"
}
}
}
}
}'

Hi,

I get the following warning in the connectivity logs when I try to run a createConnection command with a payload mapping (exactly as in the example).

The log shows:

2019-03-07 16:25:56,390 WARN [] o.e.d.s.c.m.m.MqttClientActor akka://ditto-cluster/system/sharding/connection/1/mqtt-example-connection-123/pa/$a/c1 - failed to start mapping processor due to org.eclipse.ditto.model.connectivity.MessageMapperConfigurationFailedException [message='The message mapper configuration failed due to: unterminated regular expression literal (incomingScript#1) - in line/column #1/539, source:
function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId; const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude: { properties: { value: jsonData.alt } } }; return Ditto.buildDittoProtocolMsg(my.test, thingId, things, twin, commands, modify, /features, headers, value);}', errorCode=connectivity:message.mapper.config.failed, statusCode=BAD_REQUEST, description='Check the configuration options of your mapper for errors.', href=null, dittoHeaders=ImmutableDittoHeaders [{}]]

My Task: monitor any changes to a digital twin and execute some java code.

To achieve the above goal I'm trying to use Ditto Java SDK and came across the class: org.eclipse.ditto.examples.changes.RegisterForChanges. The instructions to use here provided two steps:
Step 1: docker-compose up -d (Already done and managed to create twins and update twin features as needed via web APIs.)
Step 2: mvn compile exec:java -Dexec.mainClass="org.eclipse.ditto.examples.changes.RegisterForChanges" (When I do this I receive build success from Maven).

The question I have is how to utilize the code in the example after successfully building it. I tried the following:

1. Running the resultant JAR by navigating to the target folder and executing the command:
   java -jar ditto-examples-java-client-0-SNAPSHOT.jar
   Error received: no main manifest attribute, in ditto-examples-java-client-0-SNAPSHOT.jar
2. Running the RegisterForChanges class directly by navigating to the target\classes folder and executing the command:
   java org.eclipse.ditto.examples.changes.RegisterForChanges
   Error received:
   Error: Unable to initialize main class org.eclipse.ditto.examples.changes.RegisterForChanges
   Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonProcessingException

Any comment on how to use this code to achieve the above task would be highly appreciated. Thanks.

@thjaeckle Hi, I have deployed Eclipse ditto on my k8s cluster with the helm chart, the ditto-nginx service gateway is up and running at localhost:8080.
I can create policy, the thing but when I try to create connection for MQTT with js mapping in the Ditto explorer ui I got a 401 unauthorized error, I have used devops and foobar credentials.
Also, I have tried to create the connection via curl command on bash by following this tutorial https://github.com/eclipse-ditto/ditto-examples/blob/master/mqtt-bidirectional/README.md, in this case MQTT pub and sub works on the bash but on my web browser I can't see any update of my device.
Any idea?

hi, I am a beginner. May I know how to run the rest to websocket example? Is it possible to directly open index.xml? Do you still need to run the Docker image of Eclipse ditto first?

This is an enhancement suggestion for the mqtt-bidirectional example. I tried this example but I went too quickly through the "Preparing your IDE" part because I already have PlatformIO installed, so I missed the instruction to set the MQTT_MAX_PACKET_SIZE. This caused my device to be unable to receive any messages via Ditto. It took me long to find the cause of the problem, because I first thought it is a problem of Ditto, not of any library.

There is a better way to correct the MQTT_MAX_PACKET_SIZE than editing the files of the library. You can just call client.setBufferSize(2048); before line 130.

Write an example which shows the possibilities connecting a device via Azure IoT Hub to Ditto.

This can be used to evaluate using Ditto with Azure IoT Hub as a broker.

Tried to repeat this example, but can't create a policy with the following commands:

curl -X put 'http://localhost:8080/api/2/policies/my.test:policy' -u 'ditto:ditto' -H 'Content-Type: application/json' -d '{
"entries": {
"owner": {
"subjects": {
"nginx:ditto": {
"type": "nginx basic auth user"
}
},
"resources": {
"thing:/": {
"grant": [
"READ","WRITE"
],
"revoke": []
},
"policy:/": {
"grant": [
"READ","WRITE"
],
"revoke": []
},
"message:/": {
"grant": [
"READ","WRITE"
],
"revoke": []
}
}
}
}
}'

Error message is as below:

400 Bad Request

Whenever I try to run Rest to Web-socket example with my running ditto instance I got this kind of error in my localhost machine

Error images given below :-

I am writting this because I cannot establish a connection through websocket from my raspi 4.
I don't have the GrovePi+, nor the sensors obviously. I have also changed the grovepi library into grovepi_mock, since it's written to do when you want to start the example without GrovePi.
I do have the grovepi modules installed and also the websocket-client module. The raspi is running on the same network as the Ditto instance. The thing is also registered.

Here I am trying to send message to thing "SmartCoffee" but its not responding with any message.

var websocket = new WebSocket('ws://demo1:[email protected]/ws/2');
websocket.onmessage= function(message) {
console.log('received message data: ' + message.data);
};
websocket.onopen = function(ws) {
// register for receiving messages
console.log('Open');
// websocket.send('START-SEND-MESSAGES');
var msg = {
"topic": "org.eclipse.ditto/smartcoffee/things/live/messages/ask",
"headers": {
"content-type": "text/plain",
"correlation-id": "2215ea11-dbe5-413e-a783-24ec9c9f9619"
},
"path": "/inbox/messages/ask",
"value": "Hey, how are you?"
}
var msgtosend = JSON.stringify(msg);
websocket.send(msgtosend);
console.log(msgtosend);
};

Default unit of 'timeout' changed from ms to s. Some examples have invalid timeout values like '10000'.

Hi: all

I have some defferent type of things in ditto, every thing has its own data structure.
So the mapingContext for these things are also different.
My question is can I use just one connection to support all type of things.
Thanks!

Regarding the SmartCoffeeMachine, I am trying to brew a coffee but it doesn't happen because it doesn't pop up any captcha code. Obviously, I have created the twin and I have also connected to it. Heating up water tank and the reverse action work fine.
I can see that the PUT request is sent and also de JSON file is arriving at the machine, but since I cannot solve any captcha, I receive a 408 Request Timeout.

Am using the following step to create an MQTT connection and payload mapping.
[https://github.com/eclipse/ditto-examples/tree/master/mqtt-bidirectional]

when I am Posting the following code in the localhost using postman am getting the error
"The Connection with ID 'mqtt-example-connection-123' failed to connect."
the code to post is as follow:
"POST" URI: http://localhost:8080/devops/piggyback/connectivity?timeout=10000
Body
{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:modifyConnection", "connection": { "id": "mqtt-example-connection-123", "name": "saunak2", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://test.mosquitto.org:1883", "sources": [{ "addresses": ["ditto-tutorial/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "targets": [{ "address": "ditto-tutorial/{{ thing:id }}", "topics": [ "_/_/things/twin/events", "_/_/things/live/messages" ], "authorizationContext": ["nginx:ditto"], "qos": 0 }], "mappingContext": { "mappingEngine": "JavaScript", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) { const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload)); const jsonData = JSON.parse(jsonString); const thingId = 'my.test:octopus' const value = { temp_sensor: { properties: { value: 30.67, } }, altitude: { properties: { value: 360.341, } } }; return Ditto.buildDittoProtocolMsg( 'my.test', // your namespace thingId, 'things', // we deal with a thing 'twin', // we want to update the twin 'commands', // create a command to update the twin 'modify', // modify the twin '/features', // modify all features at once headers, value );}" } } } } }
the output of the code

Suggest me what can be the problem?

Actually, in this example, the topics mentioned in the addresses section of connection in the source.json and target.json file doesn't work.