Giter VIP home page Giter VIP logo

aws-connector's Introduction

Kanto logo

Eclipse Kanto - AWS Connector

Coverage

The AWS Connector is the main coordination center which forwards the local and remote messages. Messages processed by the AWS Connector will typically be related to telemetry data from the devices or command & control from the cloud. Messages sent to the Twin Channel are processed and redirected to the Device Shadow service. Additionally, the connector is responsible for announcing the provisioning thing information to the local MQTT broker subscribers.

For most of the connectivity options (e.g. certificates, alpn, tpm), AWS Connector is using the Suite Connector as a library.

Table of Contents

  1. Transform Ditto message to Shadow message
  2. Exclude message by Ditto topic
  3. Exclude parts of the message payload
  4. Example Ditto message sent to root thing
  5. Example Ditto message sent to child thing

Transform Ditto message to Shadow messages

Messages sent to the Twin Channel will be processed as follows:

  1. Check the message against the topic exclusion filter. See exclude message by Ditto topic.

  2. Check the message payload paths against the payload exclusion filters. See exclude parts of the message payload

  3. Check if the message contains some attributes. Attributes of the root thing will be sent with topic:

    $aws/things/root-thing-name/shadow/update

    While attributes of child things will be sent with topic:

    $aws/things/root-thing-name/shadow/name/child-thing-name/update

  4. Check if the message contains some features. Features of the root thing will be sent with topic:

    $aws/things/root-thing-name/shadow/name/feature-name/update

    While attributes of child things will be sent with topic:

    $aws/things/root-thing-name/shadow/name/child-thing-name:feature-name/update

The transformation may result in multiple smaller messages sent to the Device Shadow service. See the examples Root Thing and Child Thing below for more details.

Exclude message by Ditto topic

Filtering unnecessary messages can save a lot of cloud traffic/cost. AWS Connector can exclude messages by their Ditto Topic. To do so, provide a regex filter via topicFilter command line parameter or its corresponding JSON configuration.

The following example will exclude all of the messages, for which their topic starts with test/device:edge:containers/

-topicFilter "^test/device:edge:containers/.*"

Exclude parts of the message payload

Filtering unnecessary parts of a message can improve the cost savings and reduce the traffic load. The payloadFilters command line parameter or its corresponding JSON configuration can be used to remove the unnecessary parts of it. Multiple payload filters can be provided.

The message payload is in JSON format and each complete JSON path will be tested against all payload filters. If there is a match, the particular part of the message will be removed.

An example Ditto message

{
    ...
    "value":{
        "code":[
            {"value":"test"},
            {"keep":201},
            {"unwanted":500}
        ],
        "status":200,
        "unwanted":1234
    }
}

Applying following payload filters

-payloadFilters "./unwanted$" -payloadFilters "./0/value$"

Will trim the message payload to

{
    ...
    "value":{
        "code":[
            {"keep":201}
        ],
        "status":200
    }
}

Example Ditto message sent to root thing

{
    "topic":"ex/root/things/twin/commands/modify",
    "headers":{"response-required":false},
    "path":"/attributes/test",
    "value":{
        "attributes": {
            "location": {
                "latitude": 44.673856,
                "longitude": 8.261719
            }
        },
        "features": {
            "accelerometer": {
                "properties": {
                    "x": 3.141,
                    "y": 2.718,
                    "z": 1,
                    "unit": "g"
                }
            }
        }
    }
}

The root thing name is ex:root and following messages will be send to Device Shadow service:

  1. Topic $aws/things/ex:root/shadow/update with value
{
    "state": {
        "reported": {
            "location": {
                "latitude": 44.673856,
                "longitude": 8.261719
            }
        }
    }
}
  1. Topic $aws/things/ex:root/shadow/name/accelerometer/update with value
{
    "state": {
        "reported": {
            "x": 3.141,
            "y": 2.718,
            "z": 1,
            "unit": "g"
        }
    }
}

Example Ditto message sent to child thing

{
    "topic":"ex/root:child/things/twin/commands/modify",
    "headers":{"response-required":false},
    "path":"/attributes/test",
    "value":{
        "attributes": {
            "location": {
                "latitude": 44.673856,
                "longitude": 8.261719
            }
        },
        "features": {
            "accelerometer": {
                "properties": {
                    "x": 3.141,
                    "y": 2.718,
                    "z": 1,
                    "unit": "g"
                }
            }
        }
    }
}

The root thing name is ex:root, its child thing name is child and following messages will be send to Device Shadow service:

  1. Topic $aws/things/ex:root/shadow/name/child/update with value
{
    "state": {
        "reported": {
            "location": {
                "latitude": 44.673856,
                "longitude": 8.261719
            }
        }
    }
}
  1. Topic $aws/things/ex:root/shadow/name/child:accelerometer/update with value
{
    "state": {
        "reported": {
            "x": 3.141,
            "y": 2.718,
            "z": 1,
            "unit": "g"
        }
    }
}

Community

aws-connector's People

Contributors

ivanboychevmarinov avatar daniel-milchev avatar dimitar-dimitrow avatar eclipsewebmaster avatar k-gostev avatar yonko-gospodinov avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

bosch-io k-gostev

aws-connector's Issues

Json document too large

Update performs a merge in the shadow document. When a container is deleted, the container management SoftwareUpdatable feature send Ditto installedDependencies property modify with the deleted container missing. As in the AWS shadows a merge is performed the deleted container persist in the installedDependency part of the shadow document. Eventually this leads to over the limit shadow document:

{
   "timestamp": "2023-09-25 13:15:20.495",
   "logLevel": "ERROR",
   "traceId": "8549dcc6-87b6-a5db-75ab-f442872a02fa",
   "accountId": "797412740850",
   "status": "Failure",
   "eventType": "UpdateThingShadow",
   "protocol": "MQTT",
   "deviceShadowName": "edge:SF3-C-0006W",
   "topicName": "$aws/things/edge:SF3-C-0006W/shadow/name/edge:containers:SoftwareUpdatable/update",
   "details": "Json document too large.  Size is 8359, max allowed: 8192"
}

Add coverage badge

Provide a workflow to generate code coverage badge and include it in the readme.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.