Giter VIP home page Giter VIP logo

aws-connected-device-framework's Introduction

AWS Connected Device Framework

Introduction

Managing connected devices involves multiple phases of a device's lifecycle. The phases a typical connected device goes through are : manufacturing, onboarding, operations, support and analytics. In each of these phases, a unique set of capabilities are required. The AWS Connected Device Framework (CDF) encompasses a set of modular micro-services (referred to simply as modules) to cater to connected devices in each of their lifecycle phases.

The framework is particularly well suited for enterprise use cases which require product definition, onboarding and managing a diverse ecosystem of connected devices. The included modules facilitate:

  • Product template definition
  • Provisioning
  • Configuration and software updates
  • Organizing devices into hierarchies
  • Maintaining and updating device configuration
  • Device command and control
  • Device simulation
  • Fleet simulation

FAQ

TLDR: Read the FAQ.

Changelog

Changelog and release artifacts can be found here.

Details on any major changes along with migration instructions can be found in the Migration Guide.

Tags / Branches

If deploying to a production environment, always ensure you are checking out one of the release tags rather than checking out the main branch. This will ensure you do not inadvertently deploy a new version of the framework without testing it first.

Any other branches besides main and the release tags are considered experimental / in-progress and should be used with caution. Once the branch is merged into the main branch, it will be considered stable and will be released as a release.

Challenges

Implementing, deploying and maintaining IoT services can be significantly more complex than traditional software services due to a number of challenges faced:

Significant undifferentiated heavy lifting: It can take months, if not years, to build out an Facade API.

Skills gap: Finding product and IoT specialists is one problem, but then needing to find those same people who possess AWS knowledge is significantly harder.

Bridging historically air-gapped systems: Traditionally different areas within a business, such as manufacturing, operations, and support, have been isolated from one another. Implementing a new IoT service is a once in a generation opportunity to look at the efficiencies of bridging these systems and future proof for growth.

Limitations with turn-key solutions: Off-the-shelf solutions may be opinionated in their implementation such that incompatible limitations are introduced, as well as potential scalability issues.

Legacy devices: There may be an existing population of devices deployed in the wild that need to be transitioned into a new IoT service.

Complex security requirements: Constrained, intermittently connected devices, as well as regional governance, introduce complexity.

Long term maintainability of software: If a software's architecture and implementation does not take into consideration its longevity that comes with unforeseen emerging requirements, its maintainability, scalability and reliability can be significantly impacted as well as a business losing its agility to bring new products and services to market.

The AWS Connected Device Framework (CDF) is a platform comprising of a number of production ready micro-services, all architected and implemented using software and AWS best practices, which builds upon the AWS IoT building blocks to address these challenges.

Device Lifecycle View

The CDF modules span the following life cycle phases:

Life Cycle Phases

Architecture

The CDF modules can be mostly deployed independently. The following shows dependencies between the modules (dotted line is an optional dependency).

Dependencies

The CDF modules form a layer above the AWS building blocks as shown in the following architecture diagram. A typical customer deployment will involve the development of facade layer (the consuming application(s) of CDF modules) that contains the customer's unique business logic and orchestrates the underlying CDF modules.

Dependencies

Development

Deployment

AWS Connected Device Framework Modules

AWS CDF is comprised of the following modules. You only need to deploy and enable the modules that you are interested in using:

Bulk Certificate Creation

With this module a user can request large batches (think 1000's) of device certificates and public/private keys which can later be loaded onto a device. This is useful where customers have a hardware vendor who may not have the ability to create their own device certificates, and the customer does not want to share their CA, so instead can provide access to this module to create the device certificates as required.

See overview.

Provisioning

The provisioning module utilizes AWS IoT Device Provisioning to provide both programmatic and bulk device provisioning capabilities. The provisioning module simplifies the use of AWS IoT Device Provisioning by managing a set of provisioning templates to use with both provisioning approaches.

In addition, it allows for extending the capabilities of the AWS IoT Device Provisioning template functionality. To provide an example, an AWS IoT Device Provisioning template allows for creating certificate resources by providing a certificate signing request (CSR), a certificate ID of an existing device certificate, or a device certificate created with a CA certificate registered with AWS IoT. This module extends these capabilities by also providing the ability to automatically create (and return) new keys and certificates for a device, or to create a device certificate without the CA being registered in the account.

See overview.

Greengrass V2 Provisioning

Takes care of everything cloud side when it come to Greengrass v2. Allows you to define a template of components to deploy as a Greengrass core, as well as the cloud provisioning of Greengrass core and connected devices. In addition allows you to manage and roll out updates at scale..

See overview.

Device Patcher

Allows for the remote installation and configuration of physical devices, such as remotely installing device certificates and the Greengrass SDK on devices intended to be Greengrass core devices.

See overview.

Certificate Renewer

NOTE: Will be released once fully tested.

Identifies soon to expire certificates, and if the device is still active/authorized, will create and register new certificates, then inform the device of the new certificate being available.

Certificate Vendor

Manages the secure delivery of certificates, whether delivered over mqtt or to be downloaded from S3, to a device that can be used for elevating and/or rotating certificates.

See overview.

Certificate Activator

Provides a reference implementation of how to combine JITR (Just In Time Registration) functionality with the rest of CDF: verifies certificates against a whitelist / certificate revocation list, provisions devices, and uses Asset Library profiles to initialize a device’s data.

See overview.

Asset Library

An enhanced device registry that augments (not replaces) the AWS IoT Device Registry, allowing one to manage their fleet of devices placed within multiple hierarchical groups. Each group within a hierarchy can represent something meaningful to the business such as location, manufacturer, device types, firmware versions, etc.

With the Asset Library one can define complex models, such as modeling the components of a vehicle.

See overview.

Notifications

Allows one to configure types of events (such as a low battery alert) from multiple different event sources (AWS IoT Core, DynamoDB Stream, Kinesis Data Stream, API Gateway), which interested parties (user, service) can subscribe to receive alerts on events via SNS, MQTT republish, mobile push, or to store in a DynamoDB table.

See overview.

Device Monitoring

Detects the connected status of a device (replaced by Fleet Indexing capabilities, but still useful if a customer is not using Fleet Indexing).

See overview.

Commands

Note: this module is deprecated and has been replaced with the Command & Control module.

Utilizes AWS IoT Jobs to issue commands to a device or set of devices, and optionally inspect their execution status. It augments AWS IoT jobs by providing the ability to create Job templates (job document, parameters, and files), and enforcing that each requested command adheres to a template before executing.

Also allows for sending jobs to thousands of devices, by automatically managing temporary groups to overcome any limitations with the no. of allowed targets. Can optionally use Asset Library devices, groups, and search queries as Job targets.

See overview.

Command & Control

This module provides a simple zero-code approach to implement command and control functionality using AWS IoT Shadows, AWS IoT Jobs, and/or MQTT topics as desired. In addition, it supports sending to a variety of different targets regardless of the delivery method configured: a thing or list of things, a thing group or list of thing groups, a dynamic group or list of dynamic groups, an Asset Library device or list of devices, an Asset Library group or list of groups, an Asset Library search query, or any combination of.

Finally, any messages sent back from the device related to the command message are correlated to the original message where the entire conversation can be easily retrieved.

See overview.

Asset Library History

Tracks and stores all changes made to the Asset Library (devices, groups, policies and/or templates) for auditing purposes.

See overview.

Reference implementations

NOTE: Not released yet.

A simple demo implementation, as well as a full featured implementation (Connected Mobility Solution) are available for reference.

CI/CD

Fully automated AWS CodePipeline based continuous delivery pipeline, managing the building, testing, and deployment of modules.

Logging

CloudWatch based logging. X-Ray support.

Authn / Authz

Supports multiple authentication options: Lambda (request and token) authorizers, IAM, Cognito, API keys, and private API Gateway.

The Asset Library supports a fine-grained access control mode, suitable for multi-tenancy scenarios.

Device simulator

NOTE: Reference implementation on how to use is not released yet.

A framework that can be followed to implement a device simulator. Includes a reference implementation of a smart Kettle device, as well as a more advanced reference implementation of a vehicle simulator.

Fleet simulator

See overview.

Scales out device simulators for load testing your platform, as well as executing test plans to test other areas of your platform.

Copyright

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at

http://www.apache.org/licenses/

or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.

aws-connected-device-framework's People

Contributors

aaronatbissell avatar anish-kunduru avatar bahavar avatar benjithec avatar boardthatpowder avatar canavandl avatar commanderroot avatar dependabot[bot] avatar hassankhokhar avatar jaychung-aws avatar jonasneu-aws avatar jonemo avatar joysl avatar kaerukoara avatar komushi avatar mbreissi-aws avatar pcozzi avatar pittenga avatar rrangnekar-aws avatar tonysherman avatar ts-amz avatar tsugunao avatar tusharpatel44 avatar williamsia avatar yuma124 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-connected-device-framework's Issues

AssetLibrary AUTHORIZATION_ENABLED=true configuration returning 403 Forbidden error

Aws Connected Device Framework Affected Module(s):

  • AssetLibrary

I'm submitting a ...

  • bug report
  • feature request

Description:
The default "AUTHORIZATION_ENABLED=true" setting causes AssetLibrary requests to not work.

Current behavior:

curl -v -k -X 'POST' \
-H 'accept: */*' \
-H 'Content-Type: application/vnd.aws-cdf-v2.0+json' \
-H 'x-api-key: <api-key>' \
-d '{"properties": {"code": {"type": "string"}}}' \
'https://<stage>.execute-api.us-west-2.amazonaws.com/Prod/templates/device/robotWithSensor'

returns 403 Forbidden error

Expected behavior:
The request should create the resource and return a 200 status code

Steps to reproduce:

  1. Deploy main HEAD (f3fb09d) using the ApiKey ApiGW configuration.
  2. Run the above curl request
  3. Observe the 403 error

Additional Information:

APIGW Cannot Be Private and Have an Authorization Mechanism

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

Currently, it is not possible to deploy CDF in a manner where the APIGW is restricted from public access while also checking through another authorization mechanism.

Current behavior:

Deployed APIGWs can either be private or auth protected.

Expected behavior:

APIGW can be deployed with multiple properties. For example, private & IAM auth or private & Cognito.

Steps to reproduce:

Additional Information:

rush bundle failing

Aws Connected Device Framework Affected Module(s):
All.

I'm submitting a ...

  • bug report
  • feature request

Description:
rush bundle is failing with error:

aws-connected-device-framework/source/packages/services/simulation-manager:
 ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL  @cdf/[email protected] bundle: `rush deploy --overwrite --project @cdf/simulation-manager --target-folder ../../../deploy/simulation-manager`
Exit status 1

This is happening as the prebundle npm package script is not being run as part of the rush bundle global command. rush bundle executes source/infrastructure/bundle-core.bash which as one of its steps runs npx pnpm recursive run bundle.

That bundle command is no longer executing the prebundle and postbundle npm scripts. Pnpm did break this behavior at one point (see pnpm/pnpm#2891) where they introduced a workaround of adding enable-pre-post-scripts=true to source/common/config/rush/.npmrc, but has now stopped working.

Until we provide a new solution, a temporary workaround is to replace the following in source/infrastructure/bundle-core.bash :

npx pnpm recursive run bundle

with:

npx pnpm recursive run prebundle
npx pnpm recursive run bundle
npx pnpm recursive run postbundle

Current behavior:

Expected behavior:

Steps to reproduce:

Additional Information:

APIGW Snippets Should Enable APIGW to CW Service Role.

Aws Connected Device Framework Affected Module(s):

snippets/cfn-apiGateway*

I'm submitting a ...

  • bug report
  • feature request

Description:

CDF deploy fails with the error: "CloudWatch Logs role ARN must be set in account settings to enable logging."

Current behavior:

Any account where CDF is configured with EnableApiGatewayAccessLogs = true, and the service-linked role hasn't been defined.

Expected behavior:

The pipeline should deploy the SLR.

Steps to reproduce:

Deploy CDF with an APIGW configured for access logs in a fresh AWS account.

Additional Information:

Notfication enable/disable function feature request

Aws Connected Device Framework Affected Module(s):
Notifications

I'm submitting a ...

  • bug report
  • feature request

Description:

Currently, disabling notification temporarily is not supported. This is a feature request to disable/enable notifications.

Current behavior:
Currently, disabling notification temporarily is not supported. In the EventResoruce definition, there is an attribute of "enabled", but is is just a placeholder and not being validated currently.

Expected behavior:
This is a request to implement a feature to enable/disable notifications by using the attribute of enabled.

Steps to reproduce:
N/A

Additional Information:

Neptune Graviton Instances

Aws Connected Device Framework Affected Module(s):

asset-library

I'm submitting a ...

  • bug report
  • feature request

Description:

It would be nice if CDF supported the latest Neptune instance types.

Current behavior:

Currently, only r5 instances are supported.

Expected behavior:

Steps to reproduce:

Additional Information:

provisioning: cannot delete thing if still attached to a principal

Aws Connected Device Framework Affected Module(s):

provisioning

I'm submitting a ...

  • bug report
  • feature request

Description:

There are times (not very reproducible) where we call deleteThing from the provisioning service, which detaches that thing from it's principals (certs, policies, etc), and then deletes the thing from IOT core, and the service returns a 500 error and reports:

InvalidRequestException: Cannot delete. Thing <thingName> is still attached to one or more principals

Current behavior:

500 Error is thrown

Expected behavior:

Provisioning service catches this error and attempts to re-detach principals if the first attempt didn't work for some reason.

Steps to reproduce:

Use the provisioning service to fully provision a device, then delete it using the provisioning service. Do this over-and-over and you should eventually see some errors.

Additional Information:
I think this is a problem with the AWS javascript SDK where it returns from the detachThingPrincipal function prior to it fully detatching the thing principal.

Screen Shot 2022-06-16 at 11 40 17 AM

CDF Asset Library Group Association Dupes

Aws Connected Device Framework Affected Module(s):

asset-library

I'm submitting a ...

  • bug report
  • feature request

Description:

There is a bug in associating a group with another group. Example given the following API call:
PUT /groups/{sourceGroupPath}/{relationship}/groups/{targetGroupPath}

If the API is called multiple times, it does not check whether the relationship exists and instead creates multiple relationships. This creates problem when the user tries to list, as it shows same group multiple times.

Current behavior:

Multiple relationships are created without checking the current status.

Expected behavior:

Code checks to see whether a relationship exists before creating another one.

Steps to reproduce:

Additional Information:

authorization error on iot:RegisterCertificateWithoutCA of cdf-provisioning

Aws Connected Device Framework Affected Module(s):

cdf-provisioning

I'm submitting a ...

  • [x ] bug report
  • feature request

Description:

An iot:RegisterCertificateWithoutCA on resource error is thrown due to restrictive policy in cfn-provisioning.yml when triggering bulkprovisioning through s3 events that execute a lambda (in the facade).

Current behavior:
Error: "error\":\"User: arn:aws:sts::ACCOUNT_ID:assumed-role/cdf-provisioning-LambdaExecutionRole/cdf-provisioning-rest-development is not authorized to perform: iot:RegisterCertificateWithoutCA on resource: * because no identity-based policy allows the iot:RegisterCertificateWithoutCA action

Expected behavior:
AWS IoT things created through bulk-provisioning.

Steps to reproduce:
-setup s3 prefix with event that triggers a call to a lambda.
-lambda to read the file and lambda invoke cdf-provision:
response = lambda_invoke(
function=PROVISIONLAMBDA,
path='/things',
method='POST',
headers=HEADERS,
query_string_params=None,
stage_variables=None,
request_context=None,
body=build_payload(dev_id, certificate)
)
Additional Information:
a very permissive policy in cfn-provisioning.yml allows this to work:

  BulkProvisioningPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      Description: "cdf-provisioning policy for enabling bulk provisoning"
      Path: "/cdf/provisioning/"
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Action:
              - 'iam:PassRole'
            Effect: Allow
            Resource: !Sub "arn:aws:iam::${AWS::AccountId}:role/cdf-provisioning-*"
          - Action:
              - 'iot:RegisterCertificateWithoutCA'
            Effect: Allow
            Resource: "*"

I tried, something more restrictive which doesn't:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::ACCOUNT_ID:role/cdf-provisioning-*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "iot:RegisterCertificateWithoutCA"
            ],
            "Resource": [
                "arn:aws:iot:us-west-2:ACCOUNT_ID:*",
                "arn:aws:lambda:us-west-2:ACCOUNT_ID:*",
                "arn:aws:s3:::cdfdevreg-us-west-2-ACCOUNT_ID/shortfiles/*"
            ],
            "Effect": "Allow"
        }
    ]
}
2021-11-24T17:33:59.919Z error: User: arn:aws:sts::ACCOUNT_ID:assumed-role/cdf-provisioning-LambdaExecutionRole/cdf-provisioning-rest-development is not authorized to perform: iot:RegisterCertificateWithoutCA on resource: * because no identity-based policy allows the iot:RegisterCertificateWithoutCA action
--
2021-11-24T17:33:59.919Z error: handleError: AccessDeniedException: User: arn:aws:sts::ACCOUNT_ID:assumed-role/cdf-provisioning-development-LambdaExecutionRole/cdf-provisioning-rest-development is not authorized to perform: ot:RegisterCertificateWithoutCA on resource: * because no identity-based policy allows the iot:RegisterCertificateWithoutCA action
2021-11-24T17:33:59.922Z error: handleError: res.status: function status(code) {
this.statusCode = code;
return this;
}

Device Monitoring: errors calling asset-library don't show as lambda errors

Aws Connected Device Framework Affected Module(s):

device-monitoring

I'm submitting a ...

  • bug report
  • feature request

Description:

When there is an error calling into asset library (specifically an Internal Server Error, but i'm sure other errors), that does not translate into a lambda error.

Current behavior:

When an error happens, the lambda still returns a successful return code

Expected behavior:

When an error happens, the lambda should return an unsuccessful return code

Steps to reproduce:

Additional Information:

Screen Shot 2023-02-21 at 11 00 58 AM
Screen Shot 2023-02-21 at 11 00 01 AM

CloudWatch Logs Should Have a Retention Policy

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

A log retention policy should be applied to prevent unbounded logs.

Current behavior:

No retention policy is set.

Expected behavior:

The user can supply a log retention policy via the config.

Steps to reproduce:

Additional Information:

Neptune DB version and instance type error

While deploying the CDF stack its observed that AssetLibrary stack fails to create due to Neptune DB error where the Neptune DB version is not supported with the r4.xlarge instance type (Default instance type).

Documentation is missing mandatory argument

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

The documentation does not mention the mandatory -c argument. The deployment script mentions it.

Current behavior:

Following the documentation the user is not able to execute the deployment.

Expected behavior:

Document the mandatory -c flag and show an example of how to use it.

Steps to reproduce:

Navigate to the documentation and follow all steps for deploying in multiple stack mode.

Additional Information:

openssl-layer: stack names are not backwards compatible

The openssl-layer stack used to be named cdf-openssl-${environment}, with using the new installer package, it's called cdf-openssl-layer-${environment}. This breaks backwards compatibility.

Current stack name:

Previous stack name:

Documentation Issue

Aws Connected Device Framework Affected Module(s):
Command And Control

I'm submitting a ...

  • bug report
  • feature request

Description:
The examples and the swagger definition for POST /commands state that expectReply is optional and will default to false, however if we do not include it in the request it will error at commands.assembler toItem
Current behavior:
When not including expectReply in the body of the request the code will fail to create a new command.
Expected behavior:
When not including expectReply in the body of the request the code will successfully create a new command.
Steps to reproduce:

  1. Deploy Command And Control
  2. Make the request for the POST /commands in the first example of the readme

Additional Information:

timestamp,message
1675118741733,"INIT_START Runtime Version: nodejs:16.v11	Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:ee071345fbee0f4751fcbcd47a5784052d8f7d4256ca7f31e6056b294cee515c
"
1675118741900,"2023-01-30T22:45:41.900Z	undefined	INFO	Module config-inject loaded config:
"
1675118741902,"2023-01-30T22:45:41.902Z	undefined	INFO	{
  AWS_LAMBDA_FUNCTION_VERSION: '1',
  AWS_ACCOUNTID: '029488977242',
  AWS_S3_BUCKET: 'cdf-learning-bucket',
  AWS_S3_ROLE_ARN: 'arn:aws:iam::029488977242:role/cdf-commandandcontrol-test-LambdaExecutionRole-4QY59K2ZTZ96',
  AWS_SESSION_TOKEN: 'IQoJb3JpZ2luX2VjEB8aCXVzLWVhc3QtMSJGMEQCICMXHepz8SmH5WqPR7jjY/V+LjOKrGzU58jatyJXkIXAAiArjWTvYAaXgswa20gsApJMei07MKRK8x+vwrRzhQMOIiqcAwiY//////////8BEAAaDDAyOTQ4ODk3NzI0MiIMJ5YTOOeMFMtt8qpEKvACMol1dZzOsK+WecBgACivz7z8MuFvaqmeW4YkCLbjmFb4NyC676ZrC437It0e55BEWGtCmkmUWu2qchviRMzEB9u2HtZ2xTY+jzr5TLeD1acQiWS6TJuYqmXPjHUgTTc6Tc2CpAllsAAX9zUqCBcHn7gIzl4T8FTXbOAYcmT6DZWkrWNWIduL9tCgZ6HRkC2H5ohXUSpP4hKPxsKOUCKlZXaCnHeWi0tr0rKwrS0ZbL29Z8e9f04Z1lrnTdCW3HiiJtacD0gsHVkD1cAPVXLCXE4YgtimJPZM7J3DimLYb9YB8TUfAzg+9bkaPxoIUaN2z0X0wByl/iJgodCOclJlYG9ErIyoOsecqgRAXGrkaYGpx1BNxt/CU3HA5CN3dUQqQkE7EdPwgLF1YGPFFdatVAs8FdfbD5EabBVo+FJY8RJM0aXq07NgmQdpCIyC3f1V734/pI9Wh9rfXncb8Ycq06qMluMhz6ejWIOfhbbvE5gwlZHhngY6ngHYmC7Yan+TsFUCnP8glsnsNIPek0M0AEeKpOar7JCVCdsNiDJl3tD5KOl9kCoNNsg/NlYDoZhhDFwS7w43IBfABIaS5B+vTJF832E9JSSonce0EmH2VyRyF447faz8yiL4uyCyTp8C3FlXOai1DsP62LJVpv1VsorMXgBtePg1w281LI68tkTCNZFAT+hcGltSS4O2W5kaDhWL0J0ojQ==',
  APP_CONFIG: '',
  LD_LIBRARY_PATH: '/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib',
  LAMBDA_TASK_ROOT: '/var/task',
  AWS_LAMBDA_LOG_GROUP_NAME: '/aws/lambda/cdf-commandandcontrol-rest-test',
  AWS_LAMBDA_LOG_STREAM_NAME: '2023/01/30/[1]d828a8de92584728a1cf745e80cfbaaa',
  AWS_LAMBDA_RUNTIME_API: '127.0.0.1:9001',
  AWS_EXECUTION_ENV: 'AWS_Lambda_nodejs16.x',
  APP_CONFIG_DIR: 'packages/services/command-and-control/dist/config',
  AWS_IOT_ENDPOINT: 'a1c3ag58fpsoi8-ats.iot.us-east-1.amazonaws.com',
  AWS_LAMBDA_FUNCTION_NAME: 'cdf-commandandcontrol-rest-test',
  AWS_XRAY_DAEMON_ADDRESS: '169.254.79.129:2000',
  PATH: '/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin',
  AWS_DEFAULT_REGION: 'us-east-1',
  PWD: '/var/task',
  AWS_SECRET_ACCESS_KEY: '*',
  LAMBDA_RUNTIME_DIR: '/var/runtime',
  LANG: 'en_US.UTF-8',
  AWS_LAMBDA_INITIALIZATION_TYPE: 'on-demand',
  NODE_PATH: '/opt/nodejs/node16/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task',
  AWS_REGION: 'us-east-1',
  AWS_SQS_QUEUES_MESSAGES_QUEUEURL: 'https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-MessagesQueue-LAG9hVTysU8T',
  TZ: ':UTC',
  AWS_SQS_QUEUES_COMMANDS_QUEUEURL: 'https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-CommandsQueue-fkoQXR6ZrV19',
  AWS_ACCESS_KEY_ID: '*****',
  SHLVL: '0',
  _AWS_XRAY_DAEMON_ADDRESS: '169.254.79.129',
  _AWS_XRAY_DAEMON_PORT: '2000',
  AWS_XRAY_CONTEXT_MISSING: 'LOG_ERROR',
  _HANDLER: 'packages/services/command-and-control/dist/lambda_rest_handler.handler',
  AWS_DYNAMODB_TABLE: 'cdf-commandandcontrol-test',
  AWS_LAMBDA_FUNCTION_MEMORY_SIZE: '512',
  NODE_EXTRA_CA_CERTS: '/etc/pki/tls/certs/ca-bundle.crt',
  AWS_IOT_SHADOW_NAME: 'cac',
  AWS_JOBS_MAXTARGETS: '100',
  AWS_SQS_QUEUES_MESSAGES_TOPIC_BATCHSIZE: '100',
  AWS_SQS_QUEUES_MESSAGES_SHADOW_BATCHSIZE: '100',
  AWS_SQS_QUEUES_MESSAGES_JOBS_BATCHSIZE: '5000',
  AWS_SQS_QUEUES_COMMANDS_BATCHSIZE: '100',
  PROMISES_CONCURRENCY: '5',
  DELIVERYMETHOD_TOPIC_MQTTTTOPIC: 'cmd/cdf/cac/${thingName}',
  PROVISIONING_TEMPLATES_ADDTHINGTOTHINGGROUP: 'add_thing_to_group',
  CORS_ORIGIN: '*',
  CORS_EXPOSED_HEADERS: 'content-type,location',
  CUSTOMDOMAIN_BASEPATH: '',
  SUPPORTED_API_VERSIONS: 'application/vnd.aws-cdf-v1.0+json',
  LOGGING_LEVEL: 'info',
  ASSETLIBRARY_MODE: 'lambda',
  PORT: '3022'
}
"
1675118742299,"2023-01-30T22:45:42.299Z	undefined	INFO	Module assetlibrary-client loaded config: {
  ""AWS_LAMBDA_FUNCTION_VERSION"": ""1"",
  ""AWS_ACCOUNTID"": ""029488977242"",
  ""AWS_S3_BUCKET"": ""cdf-learning-bucket"",
  ""AWS_S3_ROLE_ARN"": ""arn:aws:iam::029488977242:role/cdf-commandandcontrol-test-LambdaExecutionRole-4QY59K2ZTZ96"",
  ""AWS_SESSION_TOKEN"": ""IQoJb3JpZ2luX2VjEB8aCXVzLWVhc3QtMSJGMEQCICMXHepz8SmH5WqPR7jjY/V+LjOKrGzU58jatyJXkIXAAiArjWTvYAaXgswa20gsApJMei07MKRK8x+vwrRzhQMOIiqcAwiY//////////8BEAAaDDAyOTQ4ODk3NzI0MiIMJ5YTOOeMFMtt8qpEKvACMol1dZzOsK+WecBgACivz7z8MuFvaqmeW4YkCLbjmFb4NyC676ZrC437It0e55BEWGtCmkmUWu2qchviRMzEB9u2HtZ2xTY+jzr5TLeD1acQiWS6TJuYqmXPjHUgTTc6Tc2CpAllsAAX9zUqCBcHn7gIzl4T8FTXbOAYcmT6DZWkrWNWIduL9tCgZ6HRkC2H5ohXUSpP4hKPxsKOUCKlZXaCnHeWi0tr0rKwrS0ZbL29Z8e9f04Z1lrnTdCW3HiiJtacD0gsHVkD1cAPVXLCXE4YgtimJPZM7J3DimLYb9YB8TUfAzg+9bkaPxoIUaN2z0X0wByl/iJgodCOclJlYG9ErIyoOsecqgRAXGrkaYGpx1BNxt/CU3HA5CN3dUQqQkE7EdPwgLF1YGPFFdatVAs8FdfbD5EabBVo+FJY8RJM0aXq07NgmQdpCIyC3f1V734/pI9Wh9rfXncb8Ycq06qMluMhz6ejWIOfhbbvE5gwlZHhngY6ngHYmC7Yan+TsFUCnP8glsnsNIPek0M0AEeKpOar7JCVCdsNiDJl3tD5KOl9kCoNNsg/NlYDoZhhDFwS7w43IBfABIaS5B+vTJF832E9JSSonce0EmH2VyRyF447faz8yiL4uyCyTp8C3FlXOai1DsP62LJVpv1VsorMXgBtePg1w281LI68tkTCNZFAT+hcGltSS4O2W5kaDhWL0J0ojQ=="",
  ""APP_CONFIG"": """",
  ""LD_LIBRARY_PATH"": ""/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"",
  ""LAMBDA_TASK_ROOT"": ""/var/task"",
  ""AWS_LAMBDA_LOG_GROUP_NAME"": ""/aws/lambda/cdf-commandandcontrol-rest-test"",
  ""AWS_LAMBDA_LOG_STREAM_NAME"": ""2023/01/30/[1]d828a8de92584728a1cf745e80cfbaaa"",
  ""AWS_LAMBDA_RUNTIME_API"": ""127.0.0.1:9001"",
  ""AWS_EXECUTION_ENV"": ""AWS_Lambda_nodejs16.x"",
  ""APP_CONFIG_DIR"": ""packages/services/command-and-control/dist/config"",
  ""AWS_IOT_ENDPOINT"": ""a1c3ag58fpsoi8-ats.iot.us-east-1.amazonaws.com"",
  ""AWS_LAMBDA_FUNCTION_NAME"": ""cdf-commandandcontrol-rest-test"",
  ""AWS_XRAY_DAEMON_ADDRESS"": ""169.254.79.129:2000"",
  ""PATH"": ""/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin"",
  ""AWS_DEFAULT_REGION"": ""us-east-1"",
  ""PWD"": ""/var/task"",
  ""AWS_SECRET_ACCESS_KEY"": ""*"",
  ""LAMBDA_RUNTIME_DIR"": ""/var/runtime"",
  ""LANG"": ""en_US.UTF-8"",
  ""AWS_LAMBDA_INITIALIZATION_TYPE"": ""on-demand"",
  ""NODE_PATH"": ""/opt/nodejs/node16/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task"",
  ""AWS_REGION"": ""us-east-1"",
  ""AWS_SQS_QUEUES_MESSAGES_QUEUEURL"": ""https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-MessagesQueue-LAG9hVTysU8T"",
  ""TZ"": "":UTC"",
  ""AWS_SQS_QUEUES_COMMANDS_QUEUEURL"": ""https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-CommandsQueue-fkoQXR6ZrV19"",
  ""AWS_ACCESS_KEY_ID"": ""*"",
  ""SHLVL"": ""0"",
  ""_AWS_XRAY_DAEMON_ADDRESS"": ""169.254.79.129"",
  ""_AWS_XRAY_DAEMON_PORT"": ""2000"",
  ""AWS_XRAY_CONTEXT_MISSING"": ""LOG_ERROR"",
  ""_HANDLER"": ""packages/services/command-and-control/dist/lambda_rest_handler.handler"",
  ""AWS_DYNAMODB_TABLE"": ""cdf-commandandcontrol-test"",
  ""AWS_LAMBDA_FUNCTION_MEMORY_SIZE"": ""512"",
  ""NODE_EXTRA_CA_CERTS"": ""/etc/pki/tls/certs/ca-bundle.crt"",
  ""AWS_IOT_SHADOW_NAME"": ""cac"",
  ""AWS_JOBS_MAXTARGETS"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_TOPIC_BATCHSIZE"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_SHADOW_BATCHSIZE"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_JOBS_BATCHSIZE"": ""5000"",
  ""AWS_SQS_QUEUES_COMMANDS_BATCHSIZE"": ""100"",
  ""PROMISES_CONCURRENCY"": ""5"",
  ""DELIVERYMETHOD_TOPIC_MQTTTTOPIC"": ""cmd/cdf/cac/${thingName}"",
  ""PROVISIONING_TEMPLATES_ADDTHINGTOTHINGGROUP"": ""add_thing_to_group"",
  ""CORS_ORIGIN"": ""*"",
  ""CORS_EXPOSED_HEADERS"": ""content-type,location"",
  ""CUSTOMDOMAIN_BASEPATH"": """",
  ""SUPPORTED_API_VERSIONS"": ""application/vnd.aws-cdf-v1.0+json"",
  ""LOGGING_LEVEL"": ""info"",
  ""ASSETLIBRARY_MODE"": ""lambda"",
  ""PORT"": ""3022""
}
"
1675118742835,"2023-01-30T22:45:42.835Z	undefined	INFO	Module provisioning-client loaded config: {
  ""AWS_LAMBDA_FUNCTION_VERSION"": ""1"",
  ""AWS_ACCOUNTID"": ""029488977242"",
  ""AWS_S3_BUCKET"": ""cdf-learning-bucket"",
  ""AWS_S3_ROLE_ARN"": ""arn:aws:iam::029488977242:role/cdf-commandandcontrol-test-LambdaExecutionRole-4QY59K2ZTZ96"",
  ""AWS_SESSION_TOKEN"": ""IQoJb3JpZ2luX2VjEB8aCXVzLWVhc3QtMSJGMEQCICMXHepz8SmH5WqPR7jjY/V+LjOKrGzU58jatyJXkIXAAiArjWTvYAaXgswa20gsApJMei07MKRK8x+vwrRzhQMOIiqcAwiY//////////8BEAAaDDAyOTQ4ODk3NzI0MiIMJ5YTOOeMFMtt8qpEKvACMol1dZzOsK+WecBgACivz7z8MuFvaqmeW4YkCLbjmFb4NyC676ZrC437It0e55BEWGtCmkmUWu2qchviRMzEB9u2HtZ2xTY+jzr5TLeD1acQiWS6TJuYqmXPjHUgTTc6Tc2CpAllsAAX9zUqCBcHn7gIzl4T8FTXbOAYcmT6DZWkrWNWIduL9tCgZ6HRkC2H5ohXUSpP4hKPxsKOUCKlZXaCnHeWi0tr0rKwrS0ZbL29Z8e9f04Z1lrnTdCW3HiiJtacD0gsHVkD1cAPVXLCXE4YgtimJPZM7J3DimLYb9YB8TUfAzg+9bkaPxoIUaN2z0X0wByl/iJgodCOclJlYG9ErIyoOsecqgRAXGrkaYGpx1BNxt/CU3HA5CN3dUQqQkE7EdPwgLF1YGPFFdatVAs8FdfbD5EabBVo+FJY8RJM0aXq07NgmQdpCIyC3f1V734/pI9Wh9rfXncb8Ycq06qMluMhz6ejWIOfhbbvE5gwlZHhngY6ngHYmC7Yan+TsFUCnP8glsnsNIPek0M0AEeKpOar7JCVCdsNiDJl3tD5KOl9kCoNNsg/NlYDoZhhDFwS7w43IBfABIaS5B+vTJF832E9JSSonce0EmH2VyRyF447faz8yiL4uyCyTp8C3FlXOai1DsP62LJVpv1VsorMXgBtePg1w281LI68tkTCNZFAT+hcGltSS4O2W5kaDhWL0J0ojQ=="",
  ""APP_CONFIG"": """",
  ""LD_LIBRARY_PATH"": ""/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"",
  ""LAMBDA_TASK_ROOT"": ""/var/task"",
  ""AWS_LAMBDA_LOG_GROUP_NAME"": ""/aws/lambda/cdf-commandandcontrol-rest-test"",
  ""AWS_LAMBDA_LOG_STREAM_NAME"": ""2023/01/30/[1]d828a8de92584728a1cf745e80cfbaaa"",
  ""AWS_LAMBDA_RUNTIME_API"": ""127.0.0.1:9001"",
  ""AWS_EXECUTION_ENV"": ""AWS_Lambda_nodejs16.x"",
  ""APP_CONFIG_DIR"": ""packages/services/command-and-control/dist/config"",
  ""AWS_IOT_ENDPOINT"": ""a1c3ag58fpsoi8-ats.iot.us-east-1.amazonaws.com"",
  ""AWS_LAMBDA_FUNCTION_NAME"": ""cdf-commandandcontrol-rest-test"",
  ""AWS_XRAY_DAEMON_ADDRESS"": ""169.254.79.129:2000"",
  ""PATH"": ""/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin"",
  ""AWS_DEFAULT_REGION"": ""us-east-1"",
  ""PWD"": ""/var/task"",
  ""AWS_SECRET_ACCESS_KEY"": ""*"",
  ""LAMBDA_RUNTIME_DIR"": ""/var/runtime"",
  ""LANG"": ""en_US.UTF-8"",
  ""AWS_LAMBDA_INITIALIZATION_TYPE"": ""on-demand"",
  ""NODE_PATH"": ""/opt/nodejs/node16/node_modules:/opt/nodejs/node_modules:/var/runtime/node_modules:/var/runtime:/var/task"",
  ""AWS_REGION"": ""us-east-1"",
  ""AWS_SQS_QUEUES_MESSAGES_QUEUEURL"": ""https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-MessagesQueue-LAG9hVTysU8T"",
  ""TZ"": "":UTC"",
  ""AWS_SQS_QUEUES_COMMANDS_QUEUEURL"": ""https://sqs.us-east-1.amazonaws.com/029488977242/cdf-commandandcontrol-test-CommandsQueue-fkoQXR6ZrV19"",
  ""AWS_ACCESS_KEY_ID"": ""*"",
  ""SHLVL"": ""0"",
  ""_AWS_XRAY_DAEMON_ADDRESS"": ""169.254.79.129"",
  ""_AWS_XRAY_DAEMON_PORT"": ""2000"",
  ""AWS_XRAY_CONTEXT_MISSING"": ""LOG_ERROR"",
  ""_HANDLER"": ""packages/services/command-and-control/dist/lambda_rest_handler.handler"",
  ""AWS_DYNAMODB_TABLE"": ""cdf-commandandcontrol-test"",
  ""AWS_LAMBDA_FUNCTION_MEMORY_SIZE"": ""512"",
  ""NODE_EXTRA_CA_CERTS"": ""/etc/pki/tls/certs/ca-bundle.crt"",
  ""AWS_IOT_SHADOW_NAME"": ""cac"",
  ""AWS_JOBS_MAXTARGETS"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_TOPIC_BATCHSIZE"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_SHADOW_BATCHSIZE"": ""100"",
  ""AWS_SQS_QUEUES_MESSAGES_JOBS_BATCHSIZE"": ""5000"",
  ""AWS_SQS_QUEUES_COMMANDS_BATCHSIZE"": ""100"",
  ""PROMISES_CONCURRENCY"": ""5"",
  ""DELIVERYMETHOD_TOPIC_MQTTTTOPIC"": ""cmd/cdf/cac/${thingName}"",
  ""PROVISIONING_TEMPLATES_ADDTHINGTOTHINGGROUP"": ""add_thing_to_group"",
  ""CORS_ORIGIN"": ""*"",
  ""CORS_EXPOSED_HEADERS"": ""content-type,location"",
  ""CUSTOMDOMAIN_BASEPATH"": """",
  ""SUPPORTED_API_VERSIONS"": ""application/vnd.aws-cdf-v1.0+json"",
  ""LOGGING_LEVEL"": ""info"",
  ""ASSETLIBRARY_MODE"": ""lambda"",
  ""PORT"": ""3022""
}
"
1675118743466,"2023-01-30T22:45:43.465Z info: Server started on port 3022 :)
"
1675118743476,"START RequestId: f43b2baf-6e32-4116-98bd-c237cd04f982 Version: 1
"
1675118743535,"2023-01-30T22:45:43.534Z info: commands.controller createCommand: in: resource: {""operation"":""reboot"",""deliveryMethod"":{""type"":""TOPIC"",""onlineOnly"":true}}
"
1675118743538,"2023-01-30T22:45:43.537Z error: handleError: ArgumentError: Expected `(_b = c.deliveryMethod` to be of type `boolean` but received type `undefined`
"
1675118743556,"2023-01-30T22:45:43.556Z error: handleError: res.status: function status(code) {
"
1675118743556,"this.statusCode = code;
"
1675118743556,"return this;
"
1675118743556,"}
"
1675118743572,"END RequestId: f43b2baf-6e32-4116-98bd-c237cd04f982
"
1675118743572,"REPORT RequestId: f43b2baf-6e32-4116-98bd-c237cd04f982	Duration: 96.46 ms	Billed Duration: 97 ms	Memory Size: 512 MB	Max Memory Used: 149 MB	Init Duration: 1742.06 ms	
XRAY TraceId: 1-63d84895-21fd7066062d9dde03fb9dc4	SegmentId: 6de09de76ed6f29e	Sampled: true	
"

Upgrade to AWS SDK JavaScript v3

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

CDF uses AWS SDK for JavaScript v2, which will be put into maintenance mode (Phase 2) in 2023. Per the SDK maintenance policy, this typically means 18 months before EoL.

Current behavior:

Expected behavior:

CDF is updated to use the v3 SDK for JS before end-of-support.

Steps to reproduce:

Additional Information:

assetlibrary: search with colon character using HTTP integration fails

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

I think API gateway is decoding the query string parameters before they make their way into the assetlibrary Lambda handler function. Then the lambda handler is expecting that the query string parameters are URL encoded, but they aren't

Current behavior:

When trying to search for groups/devices in assetlibrary using the contains, endsWith or startsWith filters (although it may not be only limited to those), if the thing I am searching for has a colon : character, the search fails.

Expected behavior:

Search should treat the : character as a part of the search term and should return results.

Steps to reproduce:

Create a group or device that has an attribute with a : character in it. Then try to search for that attribute using postman or invoke using HTTP client.

Additional Information:

Search in POSTMAN using:
{{assetlibrary_base_url}}//search?type=device&contains=macAddress:F1%3AFC

Yields this in the assetlibrary lambda logs (note, URL-decoded params)

search.controller search: in: types:device, ancestorPath:undefined, eqs:undefined, neqs:undefined, lts:undefined, ltes:undefined, gts:undefined, gtes:undefined, startsWiths:undefined, endsWiths:undefined, containses:macAddress:F1:FC, exists:undefined, nexists:undefined, facetField:undefined, summarize:undefined, offset:undefined, count:undefined, sort:undefined

Searching using the lambda looks good

const req = new SearchRequestModel();
req.types = ["device"];
req.contains = [
{
  field: "macAddress",
  value: "F1:FC",
},
];

Yields this in assetlibrary lambda logs (note URL-encoded params)

search.controller search: in: types:device, ancestorPath:undefined, eqs:undefined, neqs:undefined, lts:undefined, ltes:undefined, gts:undefined, gtes:undefined, startsWiths:undefined, endsWiths:undefined, containses:macAddress:F1%3AFC, exists:undefined, nexists:undefined, facetField:undefined, summarize:undefined, offset:undefined, count:undefined, sort:undefined

Search traverser not constructed properly

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

A change in version 1.0.1 caused searches having multiple clauses, including a traversal to sometimes not find results, depending if the traversal comes first or second.

Current behavior:

Search for a group with the following conditions (note traversal first):

"type": ["firmware"]
"eq":[
  {
  "traversals":
    [
    {"relation":"installable_on","direction":"out"}
    ],
  "field":"name","value":"my_sku_name"
  },
  {"field":"name","value":"my_firmware_name"}
]

This will yield 0 results even if that firmware exists. But if you swap the order in the eq array, the search will find a result:

"type": ["firmware"]
"eq":[
  {"field":"name","value":"my_firmware_name"},
  {
  "traversals":
    [
    {"relation":"installable_on","direction":"out"}
    ],
  "field":"name","value":"my_sku_name"
  }
]

Expected behavior:

No matter the order in the eq array - the same search result should show.

Steps to reproduce:

See above

Additional Information:

I believe this is because of how the gremlin queries are generated. Looking at the assetlibrary logs, having the traversal first (generating no search results) generates a Search Traverser that looks like this:

search.full.dao search: traverser:
"[["V"],["as","a"],
["select","a"],
["hasLabel","firmware"],
["select","a"],
["out","installable_on"],["has","name","my_sku_name"],
["has","name","my_firmware_name"],
["select","a"],["dedup"],["fold"],["unfold"],["as","matched"],
["select","matched"],["dedup"],["range",0,200],["valueMap"],["with","~tinkerpop.valueMap.tokens"]]"

Having the traversal last yields a Search Traverser that looks like this:

search.full.dao search: traverser:
"[["V"],["as","a"],
["select","a"],["hasLabel","firmware"],["has","name","my_firmware_name"],
["select","a"],["out","installable_on"],["has","name","my_sku_name"],
["select","a"],["dedup"],["fold"],["unfold"],["as","matched"],["select","matched"],["dedup"],["range",0,200],["valueMap"],["with","~tinkerpop.valueMap.tokens"]]"

I think this is due to the addition of the "select" when the a traversal is present.

If you let me know the general idea you'd like to see for a fix, I'm happy to implement it.

Assetlibrary Client has a typo

Aws Connected Device Framework Affected Module(s):

Assetlibrary-client and any services that use assetlibrary-client
I'm submitting a ...

  • bug report
  • feature request

Description:

Just a typo. It looks like it's being called with the misspelled ASSTLIBRARY_CLIENT_TYPES everywhere but it provides a little better developer experience to have it spelled correctly.

Current behavior:

No current impact to users.

Expected behavior:

Steps to reproduce:

Additional Information:
Submitting a PR with correction.

CloudWatch Logs Should use KMS

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • [] feature request

Description:

Per AWS best practices, CloudWatch logs should be encrypted.

Current behavior:

The logs are not encrypted with the customer-provided key when one is provided.

Expected behavior:

Logs are encrypted with the defined key.

Steps to reproduce:

Additional Information:

Organization Manager Lambda Should Export Role Arn

Aws Connected Device Framework Affected Module(s):

organization-manager

I'm submitting a ...

  • bug report
  • feature request

Description:

The OM module exports the role arn.

Current behavior:

Currently, the organization manager module doesn't export the lambda role arn, just the lambda arn.

Expected behavior:

Role arn is exported so that the facade can grant KMS access when a CMK is used.

Steps to reproduce:

Additional Information:

Bulkcerts installer does not allow for configuring supplier ca aliases

Aws Connected Device Framework Affected Module(s):

  • bulkcerts
  • installer

I'm submitting a ...

  • bug report
  • feature request

Description:

The bulkcerts POST /suppliers/<alias>/certificates requires an alias configuring that points to either a registered custom CA certificateId or AwsIotDefault.

Current behavior:

The installer module is not asking for this information. As a workaround, the SUPPLIER_CA_<ALIAS> environment variables can be configured manually against the lambda function but these will be wiped by the installer if a cdf upgrade is run.

Expected behavior:

  • If the installer detects the presence of the cdf-bulkCerts-sns-<env> lambda function, its current environment configuration should be queried and current SUPPLIER_CA_<ALIAS> configuration added to the installer config
  • The installer module should then prompt the user if updates to the supplier ca aliases is required

Steps to reproduce:

Additional Information:

assetlibrary-export: doesn't handle large volumes of data

Aws Connected Device Framework Affected Module(s):

assetlibrary-export

I'm submitting a ...

  • bug report
  • feature request

Description:

The asset library export tool runs into a timeout issue when trying to list all the devices in our system. We have 1M+ devices and that may be causing issues.

Current behavior:

Load 1M+ devices in the system. Try to run the asset library export tool. See a timeout error.

Screen Shot 2021-11-08 at 7 59 18 AM

Expected behavior:

No timeouts - it should export the data

Steps to reproduce:

Load 1M+ devices in the system. Try to run the asset library export tool. See a timeout error.

Additional Information:
I think this is already a know issue, I just wanted something in place to track it...

"ResourceNotFoundException" with 1.0.15

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • [o] bug report
  • feature request

Description:

Try to make CDF stack for CMS.
but I got a fail.

How to :
https://github.com/aws-solutions/aws-connected-mobility-solution/tree/develop

  • Install AWS Connected Device Framework
  • CDF version : 1.0.15 (Latest)

----------------- change string for build (path change or delete value (maybe unused))

  • source/infrastructure/cfn-cdf-core-services-A.yaml
  • source/infrastructure/cfn-cdf-core-shared.yaml

Current behavior:

---------------------------------------- cloud watch log message --------------------------------------------------
2022-09-01T07:44:26.117Z 38aa31d0-d9f6-4de7-9641-15d84fff6597 INFO Response body:
{
"Status": "FAILED",
"Reason": "See the details in CloudWatch Log Stream: 2022/09/01/[$LATEST]a4542dddd4624447bc60d92d3b2de1e7",
"PhysicalResourceId": "2022/09/01/[$LATEST]a4542dddd4624447bc60d92d3b2de1e7",
"StackId": "arn:aws:cloudformation:ap-northeast-2:921766097917:stack/cdf-core-development-ServicesA-VT0D1WKAX1J6-CertificateVendor-5WEQIJOSGMK5/de4b8550-29c9-11ed-9007-062d16d63498",
"RequestId": "448eb660-a690-4d4c-b3cd-577fdecb7fde",
"LogicalResourceId": "RotateCertificatesJob",
"Data": {
"message": "Function not found: arn:aws:lambda:ap-northeast-2:921766097917:function:cdf-commandandcontrol-rest-development",
"code": "ResourceNotFoundException",
"time": "2022-09-01T07:44:26.099Z",
"requestId": "72cdc592-5e78-434c-8c92-0fe44977922d",
"statusCode": 404,
"retryable": false,
"retryDelay": 4.934783612978189,
"level": "error",
"timestamp": "2022-09-01T07:44:26.100Z"
}
}

Expected behavior:

stack created without fail.

Steps to reproduce:

just follow below url.
https://github.com/aws-solutions/aws-connected-mobility-solution/tree/develop#install-aws-connected-device-framework

Additional Information:

Asset Library Remove Device to Device Association

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

Current behavior:

API for deleting association from Device to Device is not working:
DELETE /devices/{deviceId}/{relationship}/devices/{otherDeviceId}
Expected behavior:

This API should delete existing association between two devices
Steps to reproduce:

Create relationship: PUT /devices/{deviceId}/{relationship}/devices/{otherDeviceId} then delete using: DELETE /devices/{deviceId}/{relationship}/devices/{otherDeviceId}Additional Information:

Asset Library Supports Regex Search

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

With the introduction of TinkerPop 3.6.x, it is now possible to search via regex, which should make it much easier to find matches. The request is to support this functionality by adding updating the minimum cluster version to 1.2.1.0.R2, which includes TinkerPop 3.6 and updating the relevant CDF code to take regex input as an argument.

Current behavior:

CDF supports the ability to search, but doesn't support regex capability.

Expected behavior:

Steps to reproduce:

Additional Information:
Currently WIP on a branch in my fork.

Neptune Doesn't Use The Specified KMS Key

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

Neptune is encrypted with the default RDS key instead of using the customer provided key or the one that is generated by CDF.

Current behavior:

Expected behavior:

The same encryption key is used to encrypt Neptune as the key used for other services.

Steps to reproduce:

Additional Information:

Asset library client -> templates service -> updateTemplate method throws error

Aws Connected Device Framework Affected Module(s):

Asset library client, templates service, updateTemplate method

I'm submitting a ...

  • bug report
  • feature request

Description:

When trying to update a template a 404 error is thrown

Current behavior:

404 Not Found error when updating a template,
{
"errors": {
" [object Object]": "should NOT have additional properties"
},
"isValid": false
}

Expected behavior:

204 response expected

Steps to reproduce:

await this.assetLibraryDeviceTypeClient.updateTemplate({
                category: CategoryEnum.device,
                templateId: "template1",
                properties: {},
                required: [],
                components: [],
                relations: {}
            } as TypeResource);

Additional Information:
The request body should only contain properties, required, components and relations objects, otherwise it fails, please change the request body data type from TypeResource to a new UpdateTypeResource.

bulkcerts's list generator creates duplicated CommonNames without regard to multiple specified comman names.

Aws Connected Device Framework Affected Module(s):

bulkcerts

I'm submitting a ...

  • bug report
  • feature request

Description:

bulkcerts's list generator creates duplicated CommonNames without regard to multiple elements specified in the commanNameList parameter.

Current behavior:

All the generated certificates share the same CommonName based on the first element in the commanNameList parameter.

Expected behavior:

Each of the generated certificate has its own CommonName based on each element in the commanNameList parameter.

Steps to reproduce:

POST {{baseUrl}}/supplier/:supplierId/certificates

:supplierId is registered as 'AwsIotDefault'

{
"quantity": 3,
"certInfo": {
"commonName": "templateFoo::${list}" ,
"commonNameList":["AB1CD79EF1","AB1CD79EF2","AB1CD79EF3"],
}
}

All the CN values are same - dGVtcGxhdGVGb286OkFCMUNENzlFRjE which is the encoded value of 'templateFoo::AB1CD79EF1'.

Additional Information:

config-inject documentation is out-of-date

Aws Connected Device Framework Affected Module(s):

config-inject
I'm submitting a ...

  • bug report
  • feature request
  • documentation

Description:

The config-inject documentation is out-of-date. Still references node-config and we have switched to dotenv-flow

Current behavior:

Out-of-date documentation

Expected behavior:

Documentation is up to date

Steps to reproduce:

Additional Information:

Provisioning module: No matching bindings found for serviceIdentifier: features.delete.certificates

Aws Connected Device Framework Affected Module(s):

  • provisioning

I'm submitting a ...

  • bug report

Description:
Provisioning module fails to following documented instructions on how to configure with the error message No matching bindings found for serviceIdentifier: features.delete.certificates.

Current behavior:
Fails to start.

Expected behavior:
Provisioning module starts without errors.

Steps to reproduce:

Additional Information:
The issue is that the documentation, and default app config, contains the properties features.delete.deleteCertificates and features.delete.deletePolicies, whereas the module itself uses features.delete.certificates and features.delete.policies. As a workaround until this is fixed, set features.delete.certificates and features.delete.policies instead.

device-monitoring: lambda concurrent execution limits

Aws Connected Device Framework Affected Module(s):

device-monitoring

I'm submitting a ...

  • bug report
  • feature request

Description:

The device monitoring service keeps track of a device's online/offline status based off the $aws/events/presence MQTT topics. This is a straight IOT Core Rule -> Lambda. This is great during "normal operation" and for low-volume systems, but higher-volume systems (millions of devices) or when AWS does maintenance on IOT Core and initiates a SERVER_INITIATED_DISCONNECT, you may have 10's of thousands of devices all disconnecting and reconnecting at the same time. This will overwhelm Lambda fairly quickly and you will hit your max lambda concurrency quota very quickly (especially because each device monitoring lambda invokes an assetlibrary lambda).

Current behavior:

Lambda hits the concurrency limit and other operations going on in the system are throttled at that time

Expected behavior:

These system events are handled in a batched, methodical process

Steps to reproduce:

Additional Information:
Possibly could fix this by using SQS to queue and batch the records. This would also help with execution time, as this wouldn't require as many lambda cold starts

Screen Shot 2022-10-20 at 1 37 21 PM

assetlibrary: updating a device or group doesn't always apply all attributes

Aws Connected Device Framework Affected Module(s):

assetlibrary (full)
I'm submitting a ...

  • bug report
  • feature request

Description:

When doing a PATCH on a device or a group, if one of the elements you are trying to update is set to null (so it gets removed), then all the other elements in the request that come AFTER the null element will not be applied to the update.

Current behavior:

Only the elements prior to (and including) the null element get updated

Expected behavior:

All attributes in the request get updated

Steps to reproduce:

  1. Create a device or group with a template that includes a null-able attribute.
  2. Send a PATCH to update that object with a few attributes in the order of
  • non-null attribute
  • null attribute
  • non-null attribute
  1. See that the first and second attribute get updated but the third does not

Additional Information:

CDF Using an Older Version of Node

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

Currently, CDF uses node version 16, which will be EoL 2023-09-11. The request for this issue would be upgrade to a more recent LTS build.

Current behavior:

Expected behavior:

Steps to reproduce:

*Additional Information:
Node Releases

CDF No Longer Needs NAT or IGW

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

When CDF was first created, the IoT Core endpoints didn't support PrivateLink, so it wasn't possible to deploy CDF lambdas into a private subnet without IGW & NAT. Since all the IoT Core endpoints we currently use now support VPC-E, we should be able to remove these dependencies entirely.

Current behavior:

Expected behavior:

Steps to reproduce:

Additional Information:
IGW & NAT will be required if a customer is using SPARQL, but this not something currently supported by CDF.

Using asset-library tagged aliases in other cdf services

Aws Connected Device Framework Affected Module(s):

device-monitoring
possibly others

I'm submitting a ...

  • bug report
  • feature request

Description:

Asset library (full) uses a live tag to specify the version that should be used by all calling functions. There is currently no way for other CDF services to call into a tagged version of the the asset-library lambda.

Current behavior:

Other cdf services (like device-monitoring) will always call into $LATEST instead of live

Expected behavior:

We should be able to configure those services to use a tagged version of the asset library lambda

Steps to reproduce:

Additional Information:

CommandAndControl failing integration tests

Aws Connected Device Framework Affected Module(s):
CommandAndControl

I'm submitting a ...

  • bug report
  • feature request

Description:

Deploying CommandAndControl module and running integration tests results in errors

Current behavior:

Deploying CommandAndControl, then running integration tests results in errors

Expected behavior:

Integration tests should pass

Steps to reproduce:

Deploy with default settings, IAM auth, and assetLibrary in full mode
"assetLibrary",
"assetLibraryHistory",
"assetLibraryExport",
"certificateActivator",
"certificateVendor",
"commandAndControl",
"deviceMonitoring",
"devicePatcher",
"fleetSimulator",
"greengrass2InstallerConfigGenerators",
"greengrass2Provisioning",
"notifications",
"provisioning"

Run integration tests with

ENVIRONMENT=dev DEPLOY_ARTIFACTS_STORE_BUCKET=your-bucket-name ASSETLIBRARY_MODE='full' ./source/cicd/integrationtestsproject_build.bash

Additional Information:

Test failures are somewhat inconsistent. They are as follows

  1. Scenario: Send a message to a thing group # features/commandandcontrol/jobs.feature:76
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✔ Given command-and-control command with operation "cdf-integration-test-ota" exists # features/dist/step_definitions/commandandcontrol/commands.steps.js:16
    ✔ When I send command-and-control message to last command with attributes: # features/dist/step_definitions/commandandcontrol/messages.steps.js:15
    | targets | {"awsIoT": {"thingGroups": [{"name":"cdf-integration-test-cac-jobs-group1", "expand":true}]}} |
    | payloadParamValues | {"s3Url":"%property:GREENGRASS_TEMPLATE_S3_LOCATION%"} |
    ✔ And I wait until last command-and-control message has "awaiting_replies" status # features/dist/step_definitions/commandandcontrol/messages.steps.js:20
    ✔ Then last command-and-control message exists with attributes: # features/dist/step_definitions/commandandcontrol/messages.steps.js:38
    | $.id | world:lastMessageId |
    | $.commandId | world:lastCommand.id |
    | $.targets.awsIoT.thingGroups.length | 1 |
    | $.targets.awsIoT.thingGroups.[0].name | cdf-integration-test-cac-jobs-group1 |
    | $.status | awaiting_replies |
    | $.createdAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    | $.updatedAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    ✔ When I pause for 3000ms # features/dist/step_definitions/common/common.steps.js:73
    ✖ Then last command-and-control message has recipients: # features/dist/step_definitions/commandandcontrol/messages.steps.js:42
    | $.recipients.length | 2 |
    | $.recipients.[0].id | cdf-integration-test-cac-jobs-device1 |
    | $.recipients.[0].status | success |
    | $.recipients.[0].correlationId | regex:^[a-z0-9]{9}$ |
    | $.recipients.[1].id | cdf-integration-test-cac-jobs-device2 |
    | $.recipients.[1].status | success |
    | $.recipients.[1].correlationId | regex:^[a-z0-9]{9}$ |
    AssertionError: $.recipients.[0].status: expected 'pending' to equal 'success'
    + expected - actual

        -pending
        +success
    
        at /workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/common/common.steps.js:124:110
        at Array.forEach (<anonymous>)
        at Object.validateExpectedAttributes (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/common/common.steps.js:83:20)
        at World.<anonymous> (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/messages.steps.js:44:20)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    
  2. Scenario: A recipient replies to a message # features/commandandcontrol/jobs.feature:100
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✖ When thing "cdf-integration-test-cac-jobs-device1" replies to last command-and-control message as "IN_PROGRESS" # features/dist/step_definitions/commandandcontrol/replies.steps.js:48
    TypeError: Cannot destructure property 'requestId' of 'e.$metadata' as it is undefined.
    at reply (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/replies.steps.js:89:25)
    at async World. (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/replies.steps.js:49:5)

  3. Scenario: A recipient replies to a message # features/commandandcontrol/shadows.feature:52
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✔ When thing "cdf-integration-test-cac-shadows-device1" replies to last command-and-control message as "accepted" # features/dist/step_definitions/commandandcontrol/replies.steps.js:48
    ✔ And thing "cdf-integration-test-cac-shadows-device1" replies to last command-and-control message with payload: # features/dist/step_definitions/commandandcontrol/replies.steps.js:51
    | response | first |
    ✔ And thing "cdf-integration-test-cac-shadows-device1" replies to last command-and-control message with payload: # features/dist/step_definitions/commandandcontrol/replies.steps.js:51
    | response | second |
    ✔ When I pause for 3000ms # features/dist/step_definitions/common/common.steps.js:73
    ✖ Then last command-and-control message has replies from "cdf-integration-test-cac-shadows-device1": # features/dist/step_definitions/commandandcontrol/replies.steps.js:98
    | $.replies.length | 3 |
    | $.replies.[0].receivedAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    | $.replies.[0].action | accepted |
    | $.replies.[0].payload | undefined |
    | $.replies.[1].receivedAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    | $.replies.[1].action | reply |
    | $.replies.[1].payload | {"response":"first"} |
    | $.replies.[2].receivedAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    | $.replies.[2].action | reply |
    | $.replies.[2].payload | {"response":"second"} |
    AssertionError: $.replies.length: expected '2' to equal '3'
    + expected - actual

        -2
        +3
    
        at /workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/common/common.steps.js:124:110
        at Array.forEach (<anonymous>)
        at Object.validateExpectedAttributes (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/common/common.steps.js:83:20)
        at World.<anonymous> (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/replies.steps.js:100:20)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    
  4. Scenario: Create a command # features/commandandcontrol/topics.feature:8
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✔ Given command-and-control command with operation "cdf-integration-test-reboot" does not exist # features/dist/step_definitions/commandandcontrol/commands.steps.js:24
    ✔ When I create command-and-control command with attributes: # features/dist/step_definitions/commandandcontrol/commands.steps.js:31
    | operation | cdf-integration-test-reboot |
    | deliveryMethod | { "type": "TOPIC", "onlineOnly": true, "expectReply": true } |
    | payloadTemplate | "{\"category\": ${cat} }" |
    | payloadParams | ["cat"] |
    | tags | {"cdf-integration-test": true} |
    ✖ Then last command-and-control command exists with attributes: # features/dist/step_definitions/commandandcontrol/commands.steps.js:43
    | $.id | regex:^[a-z0-9]{9}$ |
    | $.operation | cdf-integration-test-reboot |
    | $.deliveryMethod.type | TOPIC |
    | $.deliveryMethod.expectReply | true |
    | $.deliveryMethod.onlineOnly | true |
    | $.payloadTemplate | "{\"category\": ${cat} }" |
    | $.payloadParams.length | 1 |
    | $.payloadParams.[0] | cat |
    | $.tags.cdf-integration-test | true |
    | $.createdAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    | $.updatedAt | regex:^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}).(\d{3})Z$ |
    InternalServerError: {"message": "Internal server error"}
    at /workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/commands.apigw.service.js:112:40
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CommandsApigwService.getCommand (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/commands.apigw.service.js:104:16)
    at async World. (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/commands.steps.js:44:21)

  5. Scenario: Send a message to multiple things # features/commandandcontrol/topics.feature:29
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✖ Given command-and-control command with operation "cdf-integration-test-reboot" exists # features/dist/step_definitions/commandandcontrol/commands.steps.js:16
    InternalServerError: {"message": "Internal server error"}
    at /workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/commands.apigw.service.js:99:40
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CommandsApigwService.listCommands (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/commands.apigw.service.js:91:16)
    at async listCommands (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/commands.steps.js:52:13)
    at async World. (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/commands.steps.js:19:22)

  6. Scenario: A recipient replies to a message # features/commandandcontrol/topics.feature:54
    ✔ Before # features/dist/step_definitions/provisioning/things.steps.js:26
    ✖ When thing "cdf-integration-test-cac-topics-device1" replies to last command-and-control message as "accepted" # features/dist/step_definitions/commandandcontrol/replies.steps.js:48
    InternalServerError: {"message": "Internal server error"}
    at /workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/messages.apigw.service.js:82:40
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MessagesApigwService.getRecipient (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/libraries/clients/commandandcontrol-client/dist/client/messages.apigw.service.js:74:16)
    at async reply (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/replies.steps.js:71:23)
    at async World. (/workplace/pcozzi/os-cdf/aws-connected-device-framework/source/packages/integration-tests/features/dist/step_definitions/commandandcontrol/replies.steps.js:49:5)

Better error handling for network timeouts on asset library called through lambda-invoke

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

When asset library is invoked using lambda-invoke, if there is a network connection issue or other problem, lambda-invoke will wait for the default amount of time before retrying (120 seconds, according to this). This will show misleading errors in the calling lambda, as default API Gateway/Lambda timeouts are 30 seconds.

Current behavior:

Facade function using lambda-invoke will respond with a 504 Gateway Timeout

Expected behavior:

Lambda-invoke retries before default lambda timeout expires. Either lambda-invoke fails after retry and proper error message is returned to calling function, or lambda-invoke succeeds and calling lambda is only slowed down slightly.

Steps to reproduce:

  1. Use Lambda-invoke to perform an asset library call (like a search or get group)
  2. Network (or other) issue causes request to timeout and not make it to asset library
  3. Calling lambda times out before lambda-invoke has opportunity to retry

Additional Information:

Here's a timeout that occurred. This is a log message of a facade lambda invoking asset library (using lambda). It shows it waited for about 30 seconds until the lambda hit it's timeout.
facade-log-invoke-assetlibrary
The corresponding asset library logs don't even show any invocation at that time.

Neptune: Ability to create a replica instance with deploy command

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

cfn-neptune.yaml appears to have the ability to specify a replica instance, but there's no way, from the deployment script, to specify to turn that functionality on.

Current behavior:

No way, from the deployment script, to specify a Neptune replica instance

Expected behavior:

A flag in the deployment script that sets the replica instance flag to true

Steps to reproduce:

Additional Information:
I'm not sure if this is intended behavior. I'm happy to write the fix if this is something the team wants to have. Just wanted to start the conversation.

Requests to AWS Neptune Should use IAM DB Auth

Aws Connected Device Framework Affected Module(s):

assetlibrary

I'm submitting a ...

  • bug report
  • feature request

Description:

The asset library module should be updated to deploy Neptune with IAM DB auth enabled.

Current behavior:

Currently, there is no auth mechanism for Neptune.

Expected behavior:

IAM DB auth is required, and the associated lambdas are updated to send sigv4 signed requests.

Steps to reproduce:

Additional Information:

Add bulkGetGroups to asset library client

Aws Connected Device Framework Affected Module(s):
libraries/clients/assetlibrary-client

I'm submitting a ...

  • bug report
  • feature request

Description:
Expose "bulkGetGroups" method via asset library client. This method is available in swagger definition, but not exposed via client library

Current behavior:
"bulkGetGroups" not exposed via asset library client

Expected behavior:
"bulkGetGroups" should be exposed via asset library client

Steps to reproduce:

  1. Navigate to libraries/clients/assetlibrary-client/groups.apigw.service.ts or libraries/clients/assetlibrary-client/groups.lambda.service.ts
  2. "bulkGetGroups" method definition is not present.

Additional Information:

cdf installer returns exit code 0 on failures

Aws Connected Device Framework Affected Module(s):

installer

I'm submitting a ...

  • bug report
  • feature request

Description:

When the installer fails, an exit code of 0 is still returned. This is problematic for CICD since pipelines appear to pass when they don't.

Current behavior:

A failed install returns an exit code of 0

Expected behavior:

A failed install returns a non zero exit code

Steps to reproduce:

Run the cdf-cli deploy command with incorrect arguments. Check exit code.

Additional Information:

AWS_PROFILE=<profile> cdf-cli deploy <env> <region> -c <bad-config-path>

(node:54210) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open '/config.json'
(Use `node --trace-warnings ...` to show where the warning was created)
(node:54210) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 12)
(node:54210) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

echo $? shows 0

CDF Should Create S3 Buckets

Aws Connected Device Framework Affected Module(s):

I'm submitting a ...

  • bug report
  • feature request

Description:

CDF uses one or more S3 buckets to deploy assets into. Functionality should be provided to create those buckets for the user.

Current behavior:

Currently, the user needs to create the first bucket themselves.

Expected behavior:

Buckets are created for the user based on configs, just like the Organizational Manager module.

Steps to reproduce:

Additional Information:
The code should be written in such a way that it uses a unique identifier for the bucket name. One potential way to handle this would be to hash the account ID against a user-provided salt. Another option would be to check if the bucket name has already been taken to ensure the deployment doesn't fail with a cryptic error.

Doc problem

Aws Connected Device Framework Affected Module(s):
Provisioning

I'm submitting a ...

  • [X ] bug report
  • feature request

Description:
The link to the High Level Architecture on the Readme.md page for the Provisioning Service leads a page containing the Notification service diagram.

Current behavior:
See description.

Expected behavior:
It would take me to a diagram of the architecture for the Provisioning service.

Steps to reproduce:
Go to the Provisioning service Readme.md page. Click on the link for the High Level Architecture at the bottom of the page.

Additional Information:
It appears that the actual png in the docs/images folder contains the incorrect content.

Support for Neptune Serverless

Aws Connected Device Framework Affected Module(s):

asset-library

I'm submitting a ...

  • bug report
  • feature request

Description:

It would be nice if CDF had support for Neptune Serverless.

Current behavior:

Expected behavior:

Steps to reproduce:

Additional Information:

Lambdas Are Not Deployed in VPCs

Aws Connected Device Framework Affected Module(s):

greengrass2-provisioning
installer
commands
device-patcher

I'm submitting a ...

  • bug report
  • feature request

Description:

CDF provides the option to configure that lambdas are deployed into a VPC, but this option doesn't work reliably.

Current behavior:

Several lambdas are not correctly deployed into a VPC, even when a VPC is specified via the config.

Expected behavior:

All lambdas are deployed into a VPC, when that option is selected.

Steps to reproduce:

Additional Information:
This is likely not an exhaustive list of the impacted modules, but finding all the modules would involve searching the IaC for all lambdas that are defined without an optional VPC config.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.