Giter VIP home page Giter VIP logo

aws-iot-device-sdk-java-v2's Introduction

AWS IoT Device SDK for Java v2

This document provides information about the AWS IoT device SDK for Java V2. This SDK is built on the AWS Common Runtime

Jump To:

Installation

Minimum Requirements

Step-by-step instructions

Requirements to build the AWS CRT locally

  • C++ 11 or higher
    • Clang 3.9+ or GCC 4.4+ or MSVC 2015+
  • CMake 3.1+

Step-by-step instructions

Consuming IoT Device SDK from Maven in your application

Consuming this SDK via Maven is the preferred method of consuming it and using it within your application. To consume the Java V2 SDK in your application, add the following to your pom.xml dependencies:

<dependency>
  <groupId>software.amazon.awssdk.iotdevicesdk</groupId>
  <artifactId>aws-iot-device-sdk</artifactId>
  <version>1.21.0</version>
</dependency>

Replace 1.21.0 in <version>1.21.0</version> with the latest release version for the SDK. Look up the latest SDK version here: https://github.com/aws/aws-iot-device-sdk-java-v2/releases

Build IoT Device SDK from source

Install Maven and Set PATH

# Create a workspace directory to hold all the SDK files
mkdir sdk-workspace
cd sdk-workspace
# Clone the repository
git clone https://github.com/awslabs/aws-iot-device-sdk-java-v2.git
cd aws-iot-device-sdk-java-v2
# Compile and install
mvn clean install

If you wish to use the latest CRT release, rather than the latest tested with the IoT SDK, you can run the following before running mvn clean install:

# Update the version of the CRT being used
mvn versions:use-latest-versions -Dincludes="software.amazon.awssdk.crt*"

Samples

Samples README

Getting Help

The best way to interact with our team is through GitHub. You can open a discussion for guidance questions or an issue for bug reports, or feature requests. You may also find help on community resources such as StackOverFlow with the tag #aws-iot or if you have a support plan with AWS Support, you can also create a new support case.

Please make sure to check out our resources too before opening an issue:

License

This library is licensed under the Apache 2.0 License.

Latest released version: v1.21.0

aws-iot-device-sdk-java-v2's People

Contributors

alfred2g avatar bgklika avatar bobwatso-aws avatar bretambrose avatar davidogunsaws avatar dependabot[bot] avatar fthompsonaws avatar graebm avatar jmklix avatar jonathanhenson avatar jpeddicord avatar justinboswell avatar kaibalopez avatar luyangli avatar mikedombo avatar prubcheuski avatar rccarper avatar sbstevek avatar sfod avatar sfodagain avatar singku-china avatar somayab avatar timmattison avatar tingdaok avatar twistedtwigleg avatar waahm7 avatar webermaster avatar xiazhvera avatar ytwig avatar zayeshaataws avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-iot-device-sdk-java-v2's Issues

Exception encountered: software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context)

Just wanted to let you know that the sample program in this example [Dec 08, 2020], as is not working, producing below error. Is this library good for production use?

Exception encountered: software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context)

I see that the exception coming from a JNI call.

Mint Linux V. 20, 64 bit

Found that the certificate downloaded had "undefined" in download file.

Deleted and regenerated the certificate and device , now it is working .closing this issue.

Problem not finding AWS CRT lib in Windows development environment.

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
※ My Development environment

  • OS : Windows 10
  • JAVA : Amazon Corretto 11
  • Spring Boot v2 with gradle
  • aws-iot-device-sdk v1.0.5

Describe the question

There was no problem until version 1.0.1, but the following error occurs in version 1.0.5.

Unable to unpack AWS CRT lib: java.io.IOException: Unable to open library in jar for AWS CRT: /windows/x86_64/aws-crt-jni.dll java.io.IOException: Unable to open library in jar for AWS CRT: /windows/x86_64/aws-crt-jni.dll at software.amazon.awssdk.crt.CRT.loadLibraryFromJar(CRT.java:132) at software.amazon.awssdk.crt.CRT.<clinit>(CRT.java:38) at software.amazon.awssdk.crt.CrtResource.<clinit>(CrtResource.java:99)

Please let me know if I missed anything.

Thank you in advance.

Is it possible to know reason when connection to server is failed?

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on?
Android

Describe the question
I am using java-v2 SDK for Android.
To establish an MQTT connection I am using the connect API of MqttClientConnection.
When the connection fails, it provides ExecutionException but is there any way by which we can know the error number or some other detail by which we can know the cause of connection error like not able to connect to the server, certificate error or any other error. So, it can be used to take appropriate action like try to recover error if possible or show user appropriate error on UI.

I have opened below issue but was not able to provide the requested detail in time and it is closed so I am creating new one.
#99

Below are some logs for different conditions.

  1. When Internet is not available
    W/System.err: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: System call failure
    W/System.err: at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:359)
    W/System.err: at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1921)
    ....
    W/System.err: Caused by: software.amazon.awssdk.crt.mqtt.MqttException: System call failureW/System.err: at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:142)

  1. When endpoint URL is incorrect
    W/System.err: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
    W/System.err: at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:359)
    W/System.err: at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1921)
    ....
    W/System.err: Caused by: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
    W/System.err: at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:142)

  1. With invalid certificate (certificate is not valid)
    W/System.err: software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context)
    W/System.err: at software.amazon.awssdk.crt.io.TlsContext.tlsContextNew(Native Method)
    W/System.err: at software.amazon.awssdk.crt.io.TlsContext.(TlsContext.java:34)
    W/System.err: at software.amazon.awssdk.crt.io.ClientTlsContext.(ClientTlsContext.java:36)
    W/System.err: at software.amazon.awssdk.iot.AwsIotMqttConnectionBuilder.build(AwsIotMqttConnectionBuilder.java:401)
    ....

  1. With incorrect certificate (certificate is not allowed for connection)
    W/System.err: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: Mqtt operation interrupted by connection shutdown
    W/System.err: at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:359)
    W/System.err: at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1921)
    ....
    W/System.err: Caused by: software.amazon.awssdk.crt.mqtt.MqttException: Mqtt operation interrupted by connection shutdown

  1. With incorrect Root CA
    W/System.err: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
    W/System.err: at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:359)
    W/System.err: at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1921)
    ....
    W/System.err: Caused by: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
    W/System.err: at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:142)

Please let me know in case more information required.

X509CredentialsProviderBuilder Does not use HttpProxyOptions when requesting credentials from the endpoint

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
A customer is trying to connect to websockets over proxy, but all traffic not through the proxy is blocked by a firewall. The MQTT requests after being connected go through the proxy, but not the credentials requests before hand, so the device can never be connected because the credentials provider is connecting directly.

SDK version number
SDK number: 0.5.6

Platform/OS/Hardware/Device
What are you running the sdk on?
Samsung galaxy S9 - Oreo

To Reproduce (observed behavior)
Run the credentials provider example and note that the request to get the credentials does not go through the proxy.

Expected behavior
The request to get the credentials goes through the proxy.

Logs/output
If applicable, add logs or error output.
No logs available. Monitor traffic on the proxy during test.

To enable logging, set the following system properties:

REMEMBER TO SANITIZE YOUR PERSONAL INFO

-Daws.crt.debugnative=true
-Daws.crt.log.destination=File
-Daws.crt.log.level=Trace
-Daws.crt.log.filename=<path and filename>

Additional context
Add any other context about the problem here.

Cannot connect to MQTT and publish message

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Describe the bug
I cannot connect to the following AWS IoT endpoint:
<prefix>-ats.iot.eu-west-2.amazonaws.com
using device certificate (ECC, P-256). I use mutual authentication (mTLS).
This certificate has been issued by a CA that is under my control. Both the CA and this certificate were registered to the AWS IoT Core before running any tests.
An attempt to publish an MQTT message fails with the following exception:

Exception in thread "main" java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: Old requests from the previous session are cancelled, and offline request will not be accept.
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
	at MqttCheck.main(MqttCheck.java:42)
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: Old requests from the previous session are cancelled, and offline request will not be accept.

Potentially, similar issue:
aws/aws-iot-device-sdk-python#157

SDK version number
1.2.14 (latest released)

Platform/OS/Hardware/Device
What are you running the sdk on?

  • MS Windows 10 x64
  • Windows Subsystem for Linux 1 (Debian Buster x64)

To Reproduce (observed behavior)
Steps to reproduce the behavior (please share code)

Expected behavior

AwsIotMqttConnectionBuilder builder = AwsIotMqttConnectionBuilder
                .newMtlsBuilderFromPath("src/main/resources/device.crt",
                        "src/main/resources/device_priv.pem");

        ClientBootstrap clientBootstrap = new ClientBootstrap(null, null);

        MqttClientConnection clientConnection = builder
                .withBootstrap(clientBootstrap)
                .withClientId(clientId)
                .withCleanSession(true)
                .withPassword(keyPassword)
                .withCertificateAuthorityFromPath(null, "src/main/resources/AmazonRootCA1.pem")
                .withCertificateAuthorityFromPath(null, "src/main/resources/factory.crt")
                .withEndpoint(endpoint)
                .build();

        clientConnection.connect();
        System.out.println("Connected");

        CompletableFuture<Integer> future = clientConnection.publish(new MqttMessage("myTopic", "Hello World! from v2".getBytes()),
                QualityOfService.AT_LEAST_ONCE,
                false);

        System.out.println("Status of future=" + future.get());  // **exception thrown here**
        System.out.println("Message published");

        clientConnection.disconnect();
        System.out.println("Disconnected");

Logs/output

Exception in thread "main" java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: Old requests from the previous session are cancelled, and offline request will not be accept.
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
	at MqttCheck.main(MqttCheck.java:42)
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: Old requests from the previous session are cancelled, and offline request will not be accept.

Code comments

Is your feature request related to a problem? Please describe.
Some definitions are very unclear, for example what is a ClientsBootstrap or an EventLoopGroup (meanwhile i figured that out). Above that what are minimum requirements to create a MQTT-Connection. There is just no information about what are these methods doing. I think this is a very important information at all.

Describe the solution you'd like
There should be a short description for every method to insure a good quality and a good understanding for users of this project.

Describe alternatives you've considered
An alternative was the old java-client, but it get no more updates. I can't argument this in my current Project, so i have to stick to the newer one. But the older was is more easily to understand and it's full of comments explaining everything.

Can we use java-v2 maven release for Android?

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on?
Android

Describe the question
I am using java-v2 SDK for Android. I am aware that it is possible to build SDK for Android but it is much easier to use maven repository build especially when need to update to the latest version for required fixes. I have already tried to use package 'software.amazon.awssdk.iotdevicesdk:aws-iot-device-sdk:1.2.6' but it is not working for Android due to the below error.
Unable to open library in jar for AWS CRT: /android/arm64-v8a/libaws-crt-jni.so

I have read below issues but it is not clear if it is possible and if it is then which aws-crt package is required.
#69
#87

Trying to use Greengrass sample

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
IOS: Debian 9
Java: Openjdk version "1.8.0_242"
Maven: Apache Maven 3.3.9

Describe the question
Hi guys,

I'm trying to use Greengrass sample following this guide (the arguments change from the python version to the java version).

When I run the following command:
mvn exec:java -pl samples/Greengrass -Dexec.mainClass=greengrass.BasicDiscovery -Dexec.args='--region us-east-1 --rootca /path/root-ca-cert.pem --cert /path/hash-certificate.pem.crt --key /path/hash-private.pem.key --thingName test-device --topic "hello/world/pubsub" --mode publish'

This error appears:

Exception thrown: software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context
)
software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context)
at software.amazon.awssdk.crt.io.TlsContext.tlsContextNew(Native Method)
at software.amazon.awssdk.crt.io.TlsContext.(TlsContext.java:34)
at software.amazon.awssdk.iot.discovery.DiscoveryClientConfig.(DiscoveryClientConfig.java:25)
at greengrass.BasicDiscovery.main(BasicDiscovery.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)

Any idea why this might be happening?

Thank you all.

"software.amazon.awssdk.crt.mqtt.MqttException" when connecting to MQTT over port 443 on Windows

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
Hello, could you please help with the following error? Thank you.
On windows 10 when I connect to MQTT over port 443, I got this exception:
"software.amazon.awssdk.crt.mqtt.MqttException"

SDK version number
<aws-iot-sdk-v2.version>1.1.0</aws-iot-sdk-v2.version>

Platform/OS/Hardware/Device
Windows 10 Enterprise x64
Dell Latitude 5400
Visual Studio Community 2019 C++
JDK 11.0.4

To Reproduce (observed behavior)
Here is our code

      MqttClientConnection mqttClientConnection = getConnection(deviceId);
      CompletableFuture<Boolean> connected = mqttClientConnection.connect();
      log.info("Connected to IoT. New session: {}.", connected.get());

Expected behavior
On Linux/Mac environment it works. But on Windows it doesn't.
I expect it to connect successfully without exception.

Logs/output

Expected no exception to be thrown, but got 'com.rsa.nws.control.DeviceException'

	at spock.lang.Specification.noExceptionThrown(Specification.java:118)
	at com.rsa.nws.control.DeviceClientMqttIntegrationTest.Device connects to IoT and register to listen cloud messages(DeviceClientMqttIntegrationTest.groovy:135)
Caused by: com.rsa.nws.control.DeviceException: Unable to connect to IoT
	at com.rsa.nws.control.DefaultDeviceClient.connect(DefaultDeviceClient.java:261)
	at com.rsa.nws.control.DeviceClientMqttIntegrationTest.Device connects to IoT and register to listen cloud messages(DeviceClientMqttIntegrationTest.groovy:132)
Caused by: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: System call failure
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
	at com.rsa.nws.control.DefaultDeviceClient.connect(DefaultDeviceClient.java:252)
	... 1 more
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: System call failure
	at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:142)

Additional context
Add any other context about the problem here.

Can we run Java-v2 SDK for Android App?

Hello,
I am an embedded developer. I have designed an IoT system with few micro-controllers (as IoT Device) and Raspberry Pi (as GreenGrassCore). I can successfully communicate between devices (via core) through MQTT pub-sub (Note - Since all these devices are connected on the same network the communication works irrespective of the availability of internet). Now, to move a step ahead I am learning Android and communicate messages from/to Android App to IoT Devices (via GreenGrassCore). To test the feasibility I used the Python SDK for GreenGrass and ran the sample code on Laptop and it executed successfully as expected. The communication worked with or without the availability of the Internet. Now I want the same thing to be coded for Android. Is the Java-V2 SDK the correct thing to start from? If yes, can you specify some examples and developer guide for GreenGrassCommunication? If no, can you please help with an alternate way around to make it work?

Thanking you in advance.

Unable to resolve Dependency

I have imported android project into my android studio synced it is giving following issue what should i do

Failed to resolve: software.amazon.awssdk.crt:android:0.6.2
Show in Project Structure dialog
Affected Modules: app

i tried every possible thing it didn't work

[URGENT] Unable to connect to ATS endpoint

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on?
Samsung Galaxy S8
Android 8

Describe the question

I am currently using this SDK to connect through a proxy and it worked on 2 out of 3 of our AWS accounts. The two accounts it worked on are more lenient with permissions than the third that is not currently working. During connection it is successfully able to go through the proxy, hit the credentials provider and get a response back. It does NOT hit the proxy when attempting to connect to the ATS endpoint and eventually our app says "Application is not responding".
These are the logs I see:

V/AuthCredentialsProvider: [00000074ee35b4f0]  - (id=0x7502a20080) X509 credentials provider received 27 response bytes
D/http-stream: [00000074ee35b4f0]  - id=0x74ed351600: Client request complete, response status: 403 (Forbidden).
V/http-stream: [00000074ee35b4f0]  - id=0x74ed351600: Stream refcount released, 1 remaining.
E/AuthCredentialsProvider: [00000074ee35b4f0]  - Failed to parse IoT Core response as Json document.
W/AuthCredentialsProvider: [00000074ee35b4f0]  - (id=0x7502a20080) X509 credentials provider failed to query credentials with error 6156(Unsuccessful status code returned from credentials-fetching http request)
E/AuthSigning: [00000074ee35b4f0]  - (id=0x7502a17580) Credentials Provider failed to source credentials with error 6156(aws-c-auth: AWS_AUTH_CREDENTIALS_PROVIDER_HTTP_STATUS_FAILURE, Unsuccessful status code returned from credentials-fetching http request)

Between the working aws account and the not working account, I have verified our existing policy has "iot:AssumeRoleWithCertificate" added to it, that the IAM policy role alias is as expected, and verified the IAM policies match.

Is there a permission that could be missing in order to use this SDK to connect to aws/proxy? Is there anywhere you can tell us to check from one account to another and make sure they match?

TLS negotiation fails

Known Issue

  • [x ] I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
I'm trying to connect to the IoT data endpoint using Websockets and a proxy and failing with java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed. I enabled the Apache HTTP wire logs as well as the netty logs and also captured the traffic using tcpdump. I can see the SDK trying to retrieve temporary credentials from the IoT credential endpoint, and apparently succeeding. Subsequently the SDK tries to connect to the data endpoint, succeeds and then gets disconnected. The above exception is thrown.

SDK version number
1.2.5

Platform/OS/Hardware/Device
Amazon Linux, java SE 1.8.0_92

To Reproduce (observed behavior)

MqttClientConnectionEvents callbacks = new MqttClientConnectionEvents() {
	@Override
	public void onConnectionInterrupted(int errorCode) {
		if (errorCode != 0) {
			logger.warn("Connection interrupted: " + errorCode + ": " + CRT.awsErrorString(errorCode) + ", messagingClientId = " + messagingClientId);
		}
	}

	@Override
	public void onConnectionResumed(boolean sessionPresent) {
		logger.warn("Connection resumed: " + (sessionPresent ? "existing session" : "clean session") + ", messagingClientId = " + messagingClientId);
	}
};
logger.debug("Initiating connection to AWS IoT. Messaging client ID: {}, endpoint: {}, certificate length: {}, key length: {}, CA length: {}", messagingClientId, iotEndpoint, certificate.length(), privateKey.length(), certificateAuthority.length());
eventLoopGroup = new EventLoopGroup(1);
resolver = new HostResolver(eventLoopGroup);
clientBootstrap = new ClientBootstrap(eventLoopGroup, resolver);
builder = AwsIotMqttConnectionBuilder.newMtlsBuilder(certificate, privateKey);

builder.withCertificateAuthority(certificateAuthority)
		.withBootstrap(clientBootstrap)
		.withCleanSession(false)
		.withEndpoint(iotEndpoint)
		.withConnectionEventCallbacks(callbacks)
		.withClientId(messagingClientId);

if (proxyHost != null) {
	logger.info("Using proxy {}:{}, IoT credential endpoint: {}, role alias {}", proxyHost, proxyPort, iotCredentialEndpoint, roleAlias);
	HttpProxyOptions proxyOptions = new HttpProxyOptions();
	proxyOptions.setHost(proxyHost);
	proxyOptions.setPort(proxyPort);
	builder.withWebsockets(true).withWebsocketSigningRegion(iotCredentialRegion.id()).withWebsocketProxyOptions(proxyOptions);
	try (TlsContextOptions x509TlsOptions = TlsContextOptions.createWithMtls(certificate, privateKey)) {
		x509TlsOptions.withCertificateAuthority(certificateAuthority);

		try (ClientTlsContext x509TlsContext = new ClientTlsContext(x509TlsOptions)) {
			X509CredentialsProvider.X509CredentialsProviderBuilder x509builder = new X509CredentialsProvider.X509CredentialsProviderBuilder()
					.withClientBootstrap(clientBootstrap)
					.withTlsContext(x509TlsContext)
					.withEndpoint(iotCredentialEndpoint)
					.withRoleAlias(roleAlias)
					.withThingName(messagingClientId)
					.withProxyOptions(proxyOptions);
			try (X509CredentialsProvider provider = x509builder.build()) {
				builder.withWebsocketCredentialsProvider(provider);
				logger.debug("All proxy options set");
			}
		}
	}
}

connection = builder.build();
try {
	boolean isResumed = connection.connect().get(30, TimeUnit.SECONDS);
	this.logger.info("Connected to AWS IoT. {}", isResumed ? "Resuming session" : "New session");
} catch (InterruptedException | ExecutionException | TimeoutException e) {
	throw new IllegalStateException("Unable to connect to AWS IoT", e);
} 

Expected behavior
Connection should be established with no error.

Logs/output

[INFO ] [2020-12-02T13:59:07Z] [00007fcc30abc700] [http-connection] - (0x7fcc2802b5c0) Proxy connection made successful CONNECT request to "xxxxxxx-ats.iot.us-west-2.amazonaws.com" via proxy
[INFO ] [2020-12-02T13:59:07Z] [00007fcc30abc700] [http-connection] - (0x7fcc2802b5c0) Beginning TLS negotiation
[WARN ] [2020-12-02T13:59:07Z] [00007fcc30abc700] [tls-handler] - id=0x7fcc28021600: negotiation failed with error Bad message encountered (Error encountered in /codebuild/output/src990005344/src/aws-crt-java/aws-common-runtime/s2n/tls/s2n_handshake_io.c line 1001)
[ERROR] [2020-12-02T13:59:07Z] [00007fcc30abc700] [http-connection] - (0x7fcc2802b5c0) Proxy connection failed origin server TLS negotiation with error 1029(TLS (SSL) negotiation failed)
[INFO ] [2020-12-02T13:59:07Z] [00007fcc30abc700] [http-connection] - 0x7fcc2802b5c0: Client shutdown completed with error 0 (AWS_ERROR_SUCCESS).
[ERROR] [2020-12-02T13:59:07Z] [00007fcc30abc700] [http-connection] - (0x7fcc2802b5c0) Error 1029 while connecting to "xxxxxx-ats.iot.us-west-2.amazonaws.com" via proxy.
[ERROR] [2020-12-02T13:59:07Z] [00007fcc30abc700] [websocket-setup] - id=0x7fcc28033880: Websocket setup failed to establish HTTP connection, error 1029 (AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE).

Additional context
I know the server I'm running this on together with the role alias/policy/certificate/proxy combination is ok since running theBasicPubSub sample on this server with the same role alias/policy/certificate/proxy combination works fine. I also know the code itself is ok since it works on a different server.

build android SDK failed in win10 x64 environment

Describe the bug
I am trying to build the SDK for android, but get failed.

Platform/OS/Hardware/Device
Build the SDK with Windows PowerShell in WIn10 x64 2004

Expected behavior
Implement the SDK in AndroidStudio

Logs/output

PS D:\SyncMainDoc\AndroidStudioDoc\awssdk> cd aws-crt-java/android
PS D:\SyncMainDoc\AndroidStudioDoc\awssdk\aws-crt-java\android> ./gradlew publishToMavenLocal

> Configure project :crt
Installing libcrypto prebuilt binaries

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file 'D:\SyncMainDoc\AndroidStudioDoc\awssdk\aws-crt-java\android\crt\build.gradle' line: 10

* What went wrong:
A problem occurred evaluating project ':crt'.
> Cannot run program "D:\SyncMainDoc\AndroidStudioDoc\awssdk\aws-crt-java\android\crt\fetch_libcrypto.sh": CreateProcess error=193, %1 is not a valid Win32 application

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
A problem occurred configuring project ':crt'.
> compileSdkVersion is not specified. Please add it to build.gradle

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

Add documentation for Windows

Is your feature request related to a problem? Please describe.
Provide clearer instructions on how to build the dependencies (like the CRT) on Wondows

Describe the solution you'd like
Updated documentation

JVM hangs at the end with 100% cpu

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
Since 1.2.14 the JVM hangs at the end of the process with one core with 100% cpu. Happens in Eclipse and Maven-CLI. No Java thread exists anymore (in the IDE) only the java-process itself hangs. I have to stop it in the task manager manually.
The problem is the aws-crt, because 1.2.15 works again if I use the aws-crt from 1.2.13.

SDK version number
1.2.14/15

Platform/OS/Hardware/Device
What are you running the sdk on?
OSX 11.2.2, Java8 and 11.

To Reproduce (observed behavior)

  • build a mqtt client connection
  • optional: connect or publish
  • close
  • finish JVM
    -> JVM hangs with 100%

Expected behavior
JVM doesn't hang.

Logs/output
If applicable, add logs or error output.

To enable logging, set the following system properties:

REMEMBER TO SANITIZE YOUR PERSONAL INFO

-Daws.crt.debugnative=true
-Daws.crt.log.destination=File
-Daws.crt.log.level=Trace
-Daws.crt.log.filename=<path and filename>

Additional context
Add any other context about the problem here.
aws.log

Interfaces to make it possible to use different MQTT clients

The current concrete implementation of MqttClientConnection makes it so that there isn't a way to use a different MQTT client. A customer requested an example Greengrass Lambda function that uses this SDK to handle jobs on Greengrass - aws-samples/aws-greengrass-lambda-functions#792 - and if there was an interface I could use to replace the implementation with a Greengrass compatible one I could create the example.

I'll work on a PR for this.

Proxy or other firewall bypassing solution

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on? Windows.

Describe the question
If I have my application run behind a client's firewall it won't be able to make a regular connection to AWS IoT because AWS' IPs will be blocked. Is there a way to have all communication be routed through an IP I control, and which the client has opened access to? I know I can do the authentication via proxy but is there a way to then route all further traffic through the proxy as well?

I ran across the docs for secure tunneling but I'm not sure it's relevant for my use case as it seems to require a direct connection to AWS.

Connecting to IoT fails with TLS negotiation timeout

Is your feature request related to a problem? Please describe.
I'm trying to connect to IoT via a proxy to a broker in a region that's far from me. This fails with:

[INFO ] [2020-12-09T11:10:44Z] [00000eb4] [http-connection] - 00000000009082E0: Client shutdown completed with error 1067 (AWS_IO_TLS_NEGOTIATION_TIMEOUT).
[ERROR] [2020-12-09T11:10:44Z] [00000eb4] [http-connection] - (00000000009082E0) Error 1067 while connecting to "xxxxxxx.credentials.iot.xxxxxx.amazonaws.com" via proxy.
[WARN ] [2020-12-09T11:10:44Z] [00000eb4] [connection-manager] - id=000000000088DB10: Failed to obtain new connection from http layer, error 1067(Channel shutdown due to tls negotiation timeout)

So basically if the network is slow or the latency high I won't be able to connect.

Describe the solution you'd like
TLS negotiation timeout is currently set to 4 seconds in the native layer. This is non-configurable. I'd like to be able to set it.

Describe alternatives you've considered
I can have a retry mechanism, but that won't work in a high latency setup.

Additional context
This feature request already exists in the python crt repo.

Setting SNI for Enhanced Custom Authentication

Hi,

I am trying to use the AWS IoT Java SDK v2 to build a device and connect to AWS IoT. We use enhanced custom authentication to authenticate the devices. I don't see an option to send the SNI TLS extension through this SDK. How do I do it ?

Devices that use enhanced custom authentication must send the Server Name Indication (SNI) TLS extension with a value that matches the domain of the appropriate domain configuration.

source: https://docs.amazonaws.cn/en_us/iot/latest/developerguide/enhanced-custom-auth-using.html

Crash in AWS IoT Device SDK

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Describe the bug
I am using aws-iot-device-sdk to communicate with AWS IoT Core. I met some crash problem while sending the MQTT message to AWS IoT Core quickly/repeatably. Please refer to the steps below:
[STEP1] make MQTT message in Java client program.
[STEP2] send MQTT message repeatably using aws-iot-device-sdk Java version.
Cycle: 100 MQTT messages per second.
Sample:
[STEP3] The crash appears in my client program. (The crash timing is random but I could reproduce this issue easily)

SDK version number

Platform/OS/Hardware/Device

  • Hardware: Desktop
  • OS: Windows 10
  • Device: Java Sample Application in aws-iot-device-sdk

To Reproduce (observed behavior)
Steps to reproduce the behavior (please share code)

  1. Just run the sample application of aws-iot-device-sdk
  2. Send MQTT message quickly/repeatably. (100 messages per second)
    We've used the sample below:
    https://github.com/aws/aws-iot-device-sdk-java-v2/blob/master/samples/BasicPubSub/src/main/java/pubsub/PubSub.java

int count = 0;
while (count++ < messagesToPublish)
{
CompletableFuture published = connection.publish(new MqttMessage(topic, message.getBytes()), QualityOfService.AT_LEAST_ONCE, false);
published.get();
Thread.sleep(1000);
}

Expected behavior

  • It should be run without any problem.

Logs/output

Fatal error condition occurred in C:\Program Files (x86)\Jenkins\workspace\aws-crt-java-build-dll-win64\aws-crt-java\aws-common-runtime\aws-c-common\include\aws/common/linked_list.inl:271: aws_linked_list_node_prev_is_valid(node)
Exiting Application
at 0x00007FFDFA19F827: Failed to lookup symbol: error 126
at 0x00007FFDFA19FD59: Failed to lookup symbol: error 126
at 0x00007FFDFA18CC70: Failed to lookup symbol: error 126
at 0x00007FFDFA164838: Failed to lookup symbol: error 126
at 0x00007FFDFA167F80: Failed to lookup symbol: error 126
at 0x00007FFDFA1AA5BC: Failed to lookup symbol: error 126
at 0x00007FFDFA1AB126: Failed to lookup symbol: error 126
at 0x00007FFDFA1AADF6: Failed to lookup symbol: error 126
at 0x00007FFDFA183960: Failed to lookup symbol: error 126
at 0x00007FFDFA19F412: Failed to lookup symbol: error 126
at 0x7FFE82347930: BaseThreadInitThunk
at 0x7FFE8298CE50: RtlUserThreadStart

crash point - 271 line

265
266/**
267 * Removes the specified node from the list (prev/next point to each other) and
268 * returns the next node in the list.
269 */
270AWS_STATIC_IMPL void aws_linked_list_remove(struct aws_linked_list_node *node) {
271 AWS_PRECONDITION(aws_linked_list_node_prev_is_valid(node));
272 AWS_PRECONDITION(aws_linked_list_node_next_is_valid(node));
273 node->prev->next = node->next;
274 node->next->prev = node->prev;
275 aws_linked_list_node_reset(node);
276 AWS_POSTCONDITION(node->next == NULL && node->prev == NULL);
277}
278

SDK doesn't compile with aws-crt 0.5.7

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
This is the stack trace from running the application:

    io.reactivex.exceptions.UndeliverableException: java.lang.UnsatisfiedLinkError: No implementation found for void software.amazon.awssdk.crt.CRT.awsCrtInit(int, boolean) (tried Java_software_amazon_awssdk_crt_CRT_awsCrtInit and Java_software_amazon_awssdk_crt_CRT_awsCrtInit__IZ)
        at io.reactivex.plugins.RxJavaPlugins.onError(RxJavaPlugins.java:367)
        at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:69)
        at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)

This is the stack trace from trying to build the SDK with the new aws-crt version:

> Task :iotdevicesdk:compileReleaseJavaWithJavac FAILED
/Users/alyman/aws-iot-device-sdk-java-v2/sdk/src/main/java/software/amazon/awssdk/iot/AwsIotMqttConnectionBuilder.java:449: error: cannot find symbol
                    signingConfig.setAlgorithm(AwsSigningConfig.AwsSigningAlgorithm.SIGV4);
                                                                                   ^
  symbol:   variable SIGV4
  location: class AwsSigningAlgorithm
/Users/alyman/aws-iot-device-sdk-java-v2/sdk/src/main/java/software/amazon/awssdk/iot/AwsIotMqttConnectionBuilder.java:450: error: cannot find symbol
                    signingConfig.setSignatureType(AwsSigningConfig.AwsSignatureType.HTTP_REQUEST_VIA_QUERY_PARAMS);
                                                                   ^
  symbol:   variable AwsSignatureType
  location: class AwsSigningConfig
/Users/alyman/aws-iot-device-sdk-java-v2/sdk/src/main/java/software/amazon/awssdk/iot/AwsIotMqttConnectionBuilder.java:454: error: cannot find symbol
                    signingConfig.setOmitSessionToken(true);
                                 ^
  symbol:   method setOmitSessionToken(boolean)
  location: variable signingConfig of type AwsSigningConfig
3 errors

FAILURE: Build failed with an exception.

SDK version number
SDK number: 0.5.7

Platform/OS/Hardware/Device
Samsung galaxy S9 - Oreo

To Reproduce (observed behavior)
To reproduce the first stack trace: Update the aws-crt version to 0.5.7 and try to run a sample application.
To reproduce the second stack trace: Update the build.gradle aws-crt version to 0.5.7 for iotdevicesdk and build the project.

Expected behavior
No error when running an application with the 0.5.7 version.

Logs/output
See above.

To enable logging, set the following system properties:

REMEMBER TO SANITIZE YOUR PERSONAL INFO

-Daws.crt.debugnative=true
-Daws.crt.log.destination=File
-Daws.crt.log.level=Trace
-Daws.crt.log.filename=<path and filename>

Additional context
Still trying to get proxy to work with the X509CredentialsProvider.

Unable to run BasicPubSub from the sample

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on?
Microsoft windows 10 Pro
x64 Based PC
RAM : 16GB
Processor: Intel(R) Core(TM) i5-7440HQ [email protected]

Describe the question
Not able to run sample BasicPubSub, getting following error during connect:

Version used
SDK v2 -> 1.2.10
Fatal error condition occurred in C:\Program Files (x86)\Jenkins\workspace\aws-crt-java-build-dll-win64\aws-crt-java\crt\aws-c-mqtt\source\client.c:54: aws_mutex_try_lock(&(connection)->synced_data.lock) == (-1)
Exiting Application
at 0x00007FFF8F350462: Failed to lookup symbol: error 126
at 0x00007FFF8F35068F: Failed to lookup symbol: error 126
at 0x00007FFF8F344C07: Failed to lookup symbol: error 126
at 0x00007FFF8F2E5345: Failed to lookup symbol: error 126
at 0x00007FFF8F2E56AE: Failed to lookup symbol: error 126

Can you please help, what exactly, I am missing

No implementation found for void software.amazon.awssdk.crt.CRT.awsCrtInit

I'm trying to build the mobile app to using aws-crt-java from software.amazon.awssdk.crt:aws-crt:0.11.5
but the system throws the message "No implementation found for void software.amazon.awssdk.crt.CRT.awsCrtInit"

Describe the bug
This is the stack trace from running the application:
W/System.err: java.io.IOException: Unable to open library in jar for AWS CRT: /android/x86/libaws-crt-jni.so W/System.err: at software.amazon.awssdk.crt.CRT.loadLibraryFromJar(CRT.java:155) W/System.err: at software.amazon.awssdk.crt.CRT.<clinit>(CRT.java:32) W/System.err: at software.amazon.awssdk.crt.CrtResource.<clinit>(CrtResource.java:95) W/System.err: at pubsub.PubSub.main(PubSub.java:237) W/System.err: at java.lang.reflect.Method.invoke(Native Method) W/System.err: at software.amazon.awssdk.iotsamples.MainActivity$runSample$1.invoke(MainActivity.kt:151) W/System.err: at software.amazon.awssdk.iotsamples.MainActivity$runSample$1.invoke(MainActivity.kt:24) W/System.err: at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30) E/ssdk.iotsample: No implementation found for void software.amazon.awssdk.crt.CRT.awsCrtInit(int, boolean) (tried

I found the same issue about CRT.awsCrtInit the and SDK have some conflict about version
so in lasted aws-iot-device-sdk-java-v2 and software.amazon.awssdk.crt:aws-crt:0.11.5 have same root cause at here?
I still hope to use X509 to connect my android app to Iotcore.

Platform/OS/Hardware/Device
I'm using Pixel 4 XL

PublishToTopicRequest jsonMessage does not fully support JSON

In trying to connect to Greengrass v2 with an unsupported language I ran across an issue with IPC where the jsonMessage parameter does not support JSON as one would expect. I have detailed the issue in Github issue:

aws-greengrass/aws-greengrass-nucleus#863

If a component sends a message of the form:

{ "topic": "test/topic", "publishMessage" : { "jsonMessage": { "message" : {"foo": "bar" }}}}

It will publish to the topic because the object { "foo":"bar"} map successfully to the Map<String,Object> on line 30 of the model.

But if you send a perfectly reasonable message like:

{"topic": "test/topic", "publishMessage": { "jsonMessage": { "message" : [ 1, 2, 3 ] }}}

The server will return the string "InternalServerError" as the serviceModel.fromJSON() deserialization call throws an exception. on line 269 of OperationContinuationHandler.java.

https://github.com/aws-greengrass/aws-greengrass-nucleus/blob/d65d4338fb208094c0e7edd63643d3144f126c28/src/main/java/software/amazon/awssdk/eventstreamrpc/OperationContinuationHandler.java#L269

The resulting error message doesn't conform to the typical JSON responses returned by Greengrass. Additionally, passing other valid json messages will also result in this behavior. This particularly seems like a design flaw especially for an operation which is intended to publish messages to other components. Additionally, for large messages, parsing the full json just to route to the appropriate component (which must then re-encode the value) seems incredibly wasteful in terms of resources.

Following the principle of least surprise, as a component developer, I would expect jsonMessage to pass JSON to the consumers of the topic without mutating or decoding the entire payload.

Not able to connect to AWS-IoTusing self signed certificate which is registerd in AWS IoT core

Confirm by changing [ ] to [x] below:

  • I've searched for previous similar issues and found few solutions which said that we need to pass on right root CA

What are you running the sdk on? Windows 10 machine I am trying to connect to IoT where I am using the RawPubSub.java class and I am passing args are below.

clientId = "PCMA4GMULTIACCT"
rootCaPath = "C:\Users\pu00165\Documents\GitHub_New\aws-iot-device-sdk-java-v2\samples\RawPubSub\SFSRootCAG2.pem"
certPath = "C:\Users\pu00165\Documents\GitHub_New\aws-iot-device-sdk-java-v2\samples\RawPubSub\iotmultiacct.crt"
keyPath= "C:\Users\pu00165\Documents\GitHub_New\aws-iot-device-sdk-java-v2\samples\RawPubSub\iotmultiacct.key"
endpoint = "a2eg3przjv23yh-ats.iot.us-east-1.amazonaws.com"
topic = "yukon/v1/t2h/21/392/PCMA4GMULTIACCT"

Describe the question

I am trying to use AWS iot sdk 2 (https://github.com/aws/aws-iot-device-sdk-java-v2 ) for java to implement an IoT client.

I am referring to https://github.com/aws/aws-iot-device-sdk-java-v2/blob/main/samples/RawPubSub/src/main/java/rawpubsub/RawPubSub.java class to implement the client.

To get the rootCA details I have referred to https://docs.aws.amazon.com/iot/latest/developerguide/server-authentication.html#server-authentication-certs .

However, as I am trying to connect I am getting below exception.

Here the certificate which I am using to connect with the thing (arn:aws:iot:us-east-1:637306973112:thing/PCMA4GMULTIACCT) is registered in AWS.

Caused by: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at rawpubsub.RawPubSub.main(RawPubSub.java:95)
... 5 more
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:133)

I have reached out to AWS Support (Premium) and they to tried connecting to IoT using RawPubSub and are also facing the same issue. Can you please help here?

Connection to MQTT over Websockets with HTTP Proxy hangs indefinitely

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
OS: Mac High Sierra
Device: Android 8.0
4in WVGA Nexus S Emulator

Describe the question
I'm trying to connect to MQTT over Websockets with a proxy through my machine as a test for our android app, but once I call "connection.connect()" it just seems to not do anything. When I run the PubSub.java sample and configure with websockets, it executes correctly and I see the activity in my proxy, so I'm thinking there's something I'm missing here or a weird limitation in Android I haven't discovered.

If I remove the websockets configuration, it connects fine and I'm able to publish to a topic without any issues. The device itself has its wifi settings configured for the proxy as well, and I see traffic for google play just fine.

Edit: I tried removing the proxy and just using websockets, and I get the same behavior - it hangs indefinitely.

When I execute this code:

     private void connectWithNewStuff() throws Exception {
        LOG.debug("/*-+-*/ New connection start!");
        MqttMessage lwt = new MqttMessage(getString(R.string.messaging_service_lwt_topic),
                getString(R.string.messaging_service_aws_lwt).getBytes());

        try (EventLoopGroup eventLoopGroup = new EventLoopGroup(1);
             HostResolver resolver = new HostResolver(eventLoopGroup);
             ClientBootstrap bootstrap = new ClientBootstrap(eventLoopGroup, resolver);
             AwsIotMqttConnectionBuilder builder = AwsIotMqttConnectionBuilder
                     .newMtlsBuilder(getCertificate(), getPrivateKey())
                     .withBootstrap(bootstrap)
                     .withWill(lwt, QualityOfService.AT_MOST_ONCE, false)
                     .withClientId("02:00:00:44:55:46")
                     .withCertificateAuthority(getRootCa())
                     .withCleanSession(true)
                     .withConnectionEventCallbacks(callbacks)
                     .withEndpoint("[ats endpoint].iot.us-east-1.amazonaws.com")) {
            LOG.debug("/*-+-*/ Builder Established");
            HttpProxyOptions proxy = new HttpProxyOptions();
            proxy.setHost("192.168.1.217");
            proxy.setPort(8888);
            builder.withWebsockets(true)
                    .withWebsocketSigningRegion("us-east-1")
                    .withWebsocketProxyOptions(proxy);
            try (MqttClientConnection connection = builder.build()) {
                LOG.debug("/*-+-*/ Connection built~");
                CompletableFuture<Boolean> connected = connection.connect();
                LOG.debug("/*-+-*/ Connection set to completable future.");
                try {
                    boolean sessionPresent = connected.get();
                    LOG.debug("/*-+-*/ Connected to " + (!sessionPresent ? "new" : "existing")
                            + " session!");
                } catch (Exception ex) {
                    throw new RuntimeException(
                            "/*-+-*/ Exception occurred during new connect stuff", ex);
                }
                LOG.debug("/*-+-*/ Connection established!");
            }
        }
    }

I only see the following logs before it stops entirely. I've let it sit there for a few minutes, but nothing happens.

D/com.tracelink.core.service.MessagingService: clientId = 02:00:00:44:55:46
D/FA: Connected to remote service
D/com.amazonaws.request: Initializing the regions with default regions
V/FA: Processing queued up service tasks: 6
D/com.tracelink.core.service.MessagingService: /*-+-*/ New connection start!
W/View: dispatchProvideAutofillStructure(): not laid out, ignoring
I/AssistStructure: Flattened final assist data: 4560 bytes, containing 1 windows, 17 views
D/com.tracelink.core.service.MessagingService: /*-+-*/ Builder Established
D/com.tracelink.core.service.MessagingService: /*-+-*/ Connection built~
V/FA: Inactivity, disconnecting from the service
I/zygote: Do full code cache collection, code=116KB, data=99KB
I/zygote: After code cache collection, code=102KB, data=58KB
I/zygote: Do partial code cache collection, code=122KB, data=79KB
I/zygote: After code cache collection, code=122KB, data=79KB
    Increasing code cache capacity to 512KB
D/com.tracelink.core.service.MessagingService: /*-+-*/ Connection set to completable future.

Connection issue with aws-iot-device-sdk-java-v2 (1.2.15) in Win 7

Hi team,

We are facing below issue while connecting to iot broker (MqttClientConnection) using java SDK v2 with JDK 11 (open jdk and correto both) and protocol as "x-amzn-mqtt-ca" in window 7 both 32 and 64 bit.

java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: socket is closed

Link to referred code to connect to client and publish message: https://github.com/aws/aws-iot-device-sdk-java-v2/blob/main/samples/RawPubSub/src/main/java/rawpubsub/RawPubSub.java

  1. But, The application is working fine in windows 10 with SDK v2 and JDK 11 (open jdk and correto both).

  2. In win 7 with JDK 8 and SDK v1 also the application is working fine .

  3. We think there is no issue with certification as certificates as the following cmd works fine

mosquitto_pub --cafile root.cert --cert deviceCertAndCACert.crt --key deviceCert.key -h %host% -p 443 -q 1 -t platform -I uniqueClientID --tls-version tlsv1.2 --tls-alpn x-amzn-mqtt-ca -m "sample Request" -d

  1. Also, Port 443 is open in the system.
  2. For certification we are following JITP as per this link[https://aws.amazon.com/blogs/iot/setting-up-just-in-time-provisioning-with-aws-iot-core/]
    does aws-iot-device-sdk-java-v2 (1.2.15) supports Windows 7?
    SDK Name and Version: aws-iot-device-sdk-java-v2
    Hardware / Kit Used: Windows-7

Documentation for the Java SDK

Is your feature request related to a problem? Please describe.
The python and javascript SDKs have an accompanying documentation which is missing from the Java SDK

Describe the solution you'd like
Documentation for the SDK similar to the one available for the other languages and the CRT https://awslabs.github.io/aws-crt-java/

Describe alternatives you've considered

Additional context
Ensure consistency across SDKs

Problem UnsatisfiedLinkError AWS CRT lib in Windows development environment.

I've searched for previous similar issues and didn't find any solution

Platform/OS/Hardware/Device

OS : Windows 10
JAVA : jdk1.8.0_121
aws-iot-device-sdk v1.0.6

Describe the question

I get following Error when IoT class is trying to initiate CRT:

Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\Users\lukas\AppData\Local\Temp\AWSCRT_15918817021667267130506483563287aws-crt-jni.dll: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at software.amazon.awssdk.crt.CRT.loadLibraryFromJar(CRT.java:154)
at software.amazon.awssdk.crt.CRT.(CRT.java:38)
at software.amazon.awssdk.crt.CrtResource.(CrtResource.java:99)
at IOTDevice.main(IOTDevice.java:50)

The file (AWSCRT_15918817021667267130506483563287aws-crt-jni.dll) is available under indicated location, I'm not sure what dependent libraries might be missing.

Thank you in advance,
Lukasz

Is it possible to know reason for connection error?

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
What are you running the sdk on?
Android

Describe the question
I am using java-v2 SDK for Android. To establish an MQTT connection I am using the connect API of MqttClientConnection. When the connection fails, it provides ExecutionException but is there any way by which we can know the error number or some other detail by which we can know the cause of connection error like not able to connect to the server, certificate error or any other error. So, it can be used to take appropriate action like try to recover error if possible or show user appropriate error on UI.

SubscribeToTopic: blobs are mutated and not RFC4648 compliant

In trying to connect to Greengrass v2 via an unsupported language, I encountered a surprising encoding issue blobs. For reference the Github issue for greengrass v2 is here:

aws-greengrass/aws-greengrass-nucleus#864

The core issue is when you publish a simple message like "hello world" to a local topic using a message like:

{ "topic": "test/topic", "publishMessage": {"binaryMessage" : { "message" : "aGVsbG8gd29ybGQ=" }}}

The subscriber receives the message

{ "binaryMessage": {"message":"aGVsbG8gd29ybGQ\u003d"}}

While this is valid JSON, the message payload is not a valid base64 encoded string according to RFC4648. Instead, one must double decode the string value, first by decoding the unnecessarily encoded UTF8 code points, (there may be multiple if more characters of padding are required).

This clearly also violates the principle of least surprise as the output value is mutated from the value sent by the publisher. When I publish a message via an IPC / RPC mechanism I expect to get the same value out, not some arbitrarily transformed equivalent.

It should be noted that the deserialization of the serivceModel cause the entire blob to both be decoded and re-encoded when passing between components. This produces large unnecessary memory and processing overheads for certain workloads. As the base64 encoding already incurs a 33% size overhead, this additional encoding/decoding step also incurs additional overhead on the consumer who must now once again buffer and decode the needlessly encoded code point.

TLS negotiation timeout

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
I'm getting TLS negotiation timeouts on a specific network, on other networks the code works fine. This is a continuation of this issue which was closed a month ago. I upgraded the SDK version as recommended but still having the same problem. The error from the CRT log is:

[INFO] [2021-02-15T15:06:38Z] [000019e0] [http-connection] - (000000000155B3A0) Beginning TLS negotiation
[INFO] [2021-02-15T15:06:48Z] [000019e0] [http-connection] - 000000000155B3A0: Client shutdown completed with error 1067 (AWS_IO_TLS_NEGOTIATION_TIMEOUT).
[ERROR] [2021-02-15T15:06:48Z] [000019e0] [http-connection] - (000000000155B3A0) Error 1067 while connecting to "xxxxxxxxx.credentials.iot.eu-central-1.amazonaws.com" via proxy.

SDK version number
1.2.12. (crt 0.10.3)

Platform/OS/Hardware/Device
Windows Server 2019 Standard.

To Reproduce (observed behavior)
See below for code.

Expected behavior
I expected the SDK to connect to the IoT broker.

Additional context
The network has 90Mbps download and 40Mpbs upload. This code works fine on other networks, but the speed I measured on the problematic network seems fine.

MqttClientConnectionEvents callbacks = new MqttClientConnectionEvents() {
            @Override
            public void onConnectionInterrupted(int errorCode) {
                if (errorCode != 0) {
                    logger.warn("Connection interrupted: " + errorCode + ": " + CRT.awsErrorString(errorCode) + ", messagingClientId = " + messagingClientId);
                }
            }

            @Override
            public void onConnectionResumed(boolean sessionPresent) {
                logger.warn("Connection resumed: " + (sessionPresent ? "existing session" : "clean session") + ", messagingClientId = " + messagingClientId);
            }
        };
        
        eventLoopGroup = new EventLoopGroup(1);

        resolver = new HostResolver(eventLoopGroup);
        clientBootstrap = new ClientBootstrap(eventLoopGroup, resolver);


        try {
            builder = AwsIotMqttConnectionBuilder.newDefaultBuilder(); 
        } catch (UnsupportedEncodingException e) {
            throw new IllegalStateException(e);
        }

        builder.withCertificateAuthority(certificateAuthority)
                .withBootstrap(clientBootstrap)
                .withCleanSession(false)
                .withEndpoint(iotEndpoint)
                .withConnectionEventCallbacks(callbacks)
                .withClientId(messagingClientId);

        if (proxyHost != null) {
            logger.info("Using proxy {}:{}, IoT credential endpoint: {}, role alias {}", proxyHost, proxyPort, iotCredentialEndpoint, roleAlias);
            HttpProxyOptions proxyOptions = new HttpProxyOptions();
            proxyOptions.setHost(proxyHost);
            proxyOptions.setPort(proxyPort);
            builder.withWebsockets(true).withWebsocketSigningRegion(iotCredentialRegion.id()).withWebsocketProxyOptions(proxyOptions);
            try (TlsContextOptions x509TlsOptions = TlsContextOptions.createWithMtls(certificate, privateKey)) {
                x509TlsOptions.withCertificateAuthority(certificateAuthority);

                try (ClientTlsContext x509TlsContext = new ClientTlsContext(x509TlsOptions)) {
                    X509CredentialsProvider.X509CredentialsProviderBuilder x509builder = new X509CredentialsProvider.X509CredentialsProviderBuilder()
                            .withClientBootstrap(clientBootstrap)
                            .withTlsContext(x509TlsContext)
                            .withEndpoint(iotCredentialEndpoint)
                            .withRoleAlias(roleAlias)
                            .withThingName(messagingClientId)
                            .withProxyOptions(proxyOptions);
                    try (X509CredentialsProvider provider = x509builder.build()) {
                        builder.withWebsocketCredentialsProvider(provider);
                    }
                }
            }
        }

        connection = builder.build();
        try {
            boolean isResumed = connection.connect().get(30, TimeUnit.SECONDS);
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
            throw new IllegalStateException("Unable to connect to AWS IoT", e);
        }

Client using custom authorizer unable to receive messages, but cert client can

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
We are attempting to use RawPubSub with a custom authorizer to publish and receive.

When they use a certificate and the BasicPubSub, connect/publish/subscribe/receive works as expected.

When they use a custom authorizer that returns the same policy, connect/publish/subscribe works as expected but no messages are received. Even a policy allowing all AWSIOT operations on all resources (action: iot:, resource:) resulted in no messages being received.

We are returning the same policy text in the custom authorizer as they had in the certificate but there are no messages received or errors thrown, however a subscription is stil establish per $aws/event/subscribe/#

SDK version number
built from tag v1.2.8

Platform/OS/Hardware/Device
What are you running the sdk on? Mac OSX 10.15.7

To Reproduce (observed behavior)

  • checkout git tag v1.2.8
  • make RawPubSub.java and PubSub.java classes public
  • remove publish block from RawPubSub.java
  • remove subscribe block from PubSub.java
  • build sdk and both samples via mvn clean install
  • create default custom authorizer that returns following policy:
{
	'Version': '2012-10-17',
	'Statement': [{
		'Effect': 'Allow',
		'Action': ['iot:Connect'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:client/CLIENTID']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Subscribe'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topicfilter/iot-2/type/*/id/CLIENTID/cmd/*']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Receive'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topic/iot-2/type/*/id/CLIENTID/cmd/*']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Publish'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topic/iot-2/type/*/id/CLIENTID/evt/*']
	}]
}

create a "Thing" with the following policy:

{
	'Version': '2012-10-17',
	'Statement': [{
		'Effect': 'Allow',
		'Action': ['iot:Connect'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:client/CLIENTID']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Subscribe'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topicfilter/iot-2/type/*/id/CLIENTID/evt/*']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Receive'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topic/iot-2/type/*/id/CLIENTID/evt/*']
	}, {
		'Effect': 'Allow',
		'Action': ['iot:Publish'],
		'Resource': ['arn:aws:iot:us-east-2:ACCOUNT:topic/iot-2/type/*/id/CLIENTID/cmd/*']
	}]
}

execute the following commands in separate terminals:

mvn exec:java -pl samples/RawPubSub -Dexec.mainClass=rawpubsub.RawPubSub -Dexec.args='--endpoint <blah>-ats.iot.us-east-2.amazonaws.com --rootca AmazonRootCA1.pem --username CLIENTID --password PASSWORD --clientId CLIENTID --topic iot-2/type/WP/id/CLIENTID/cmd/foobar'

mvn exec:java -pl samples/PubSubStress -Dexec.mainClass=pubsubstress.PubSubStress -Dexec.args='--endpoint <blah>-ats.iot.us-east-2.amazonaws.com --rootca ca.pem --cert cert.pem --key key.pem --clientId CLIENTID --topic iot-2/type/WP/id/CLIENTID/cmd/foobar'

Observe in the test console that the RawPubSub will connect and subscribe but it does not print any messages to the terminal and completes without error

Expected behavior
RawPubSub should print messages to the terminal

Release Date

Hello,

I am currently running into a roadblock because I want to use either the Java or the Nodejs SDK for a large IoT project. The released Java SDK doesn't have Jobs support. The official NodeJS SDK doesn't seem to be actively maintained.

What is the V2 Java SDK release ETA?

Thanks much

"TLS (SSL) negotiation failed" When trying to connect MqttClientConnection

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
Mac High Sierra

Describe the question
I'm trying to follow the basic pub-sub example to try and get a handle for this SDK, but I'm getting a connection issue. I've done some searching in various areas including previous issues of this github page but nothing has worked for me so far. I'll provide the specifics below.

I'm getting the follow exception:

Caused by: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:359)
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1921)
        at com.tracelink.core.service.MessagingService.connectWithNewStuff(MessagingService.java:512)
        at com.tracelink.core.service.MessagingService.connectToMqtt(MessagingService.java:483) 
        at com.tracelink.core.service.MessagingService.establishConnection(MessagingService.java:458) 
        at com.tracelink.core.service.MessagingService.connect(MessagingService.java:371) 
        at com.tracelink.core.service.-$$Lambda$QGvInWxgQaux-XIMYYPhle8KTFw.run(Unknown Source:2) 
        at com.tracelink.core.service.MessagingService.lambda$wrap$0(MessagingService.java:391) 
        at com.tracelink.core.service.-$$Lambda$MessagingService$knrcCIKmsULdFEyFPCA7CjogF58.call(Unknown Source:2) 
        at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44) 
        at io.reactivex.Single.subscribe(Single.java:3096) 
        at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89) 
        at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:571) 
        at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66) 
        at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) 
        at java.lang.Thread.run(Thread.java:764) 
     Caused by: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed
        at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:145)

When I try to run "boolean sessionPresent = connected.get();" in this snippit:

private void connectWithNewStuff() throws Exception {
        LOG.debug("/*-+-*/ New connection start!");
        MqttMessage lwt = new MqttMessage(getString(R.string.messaging_service_lwt_topic),
                getString(R.string.messaging_service_aws_lwt).getBytes());
        String endpoint = SharedPreferenceHelper.getAwsConnectionURL(this);
        LOG.debug("/*-+-*/ Endpoint url: " + endpoint);

        try (EventLoopGroup eventLoopGroup = new EventLoopGroup(1);
             HostResolver resolver = new HostResolver(eventLoopGroup);
             ClientBootstrap bootstrap = new ClientBootstrap(eventLoopGroup, resolver);
             AwsIotMqttConnectionBuilder builder = AwsIotMqttConnectionBuilder
                     .newMtlsBuilder(getCertificate(), getPrivateKey())
                     .withPort((short) 8883)
                     .withBootstrap(bootstrap)
                     .withWill(lwt, QualityOfService.AT_MOST_ONCE, false)
                     .withClientId("02:00:00:44:55:46")
                     .withCertificateAuthority(getRootCa())
                     .withEndpoint(endpoint)) {
            LOG.debug("/*-+-*/ Builder Established");
            try (MqttClientConnection connection = builder.build()) {
                LOG.debug("/*-+-*/ Connection built~");
                CompletableFuture<Boolean> connected = connection.connect();
                try {
                    boolean sessionPresent = connected.get();
                    LOG.debug("/*-+-*/ Connected to " + (!sessionPresent ? "new" : "existing")
                            + " session!");
                } catch (Exception ex) {
                    throw new RuntimeException(
                            "/*-+-*/ Exception occurred during new connect stuff", ex);
                }
        }
}

Not able to establish MQTT connection

Platform/OS/Hardware/Device
Windows 10

Describe the question
I am trying to use JITP feature to onboard my client certificate.
Its working with aws iot device sdk for java old version(https://github.com/aws/aws-iot-device-sdk-java) using PublishSubscribeSample.java but with this new version the same I try to do using PubSub.java sample, its not working.

Issue 1 is that as mentioned for JITP (https://aws.amazon.com/blogs/iot/setting-up-just-in-time-provisioning-with-aws-iot-core), for first connection attempt, a concatenated(device + ca) certificate should be used. If I try to do that I get the error: TLS (SSL) negotiation failed
Issue 2 is that If I do the client JITP using old sdk for java and use this new sdk sample code to just connect with client certificate, I still get the error: TLS (SSL) negotiation failed.

If I use RSA certificates, issue one still appears but issue 2 does not and with rsa certificates connection is established.
If I use ECC certificates, I get both issue 1 and issue 2.

However, I get no issue if I do same steps with old sdk for java. With old sdk, it work fine for both rsa and ecc certificates.

What additional need to be done for ECC certificates to make the connection work and why concatenated certificate not work as explained here: https://aws.amazon.com/blogs/iot/setting-up-just-in-time-provisioning-with-aws-iot-core/.

I am using ats endpoint.

Please help.

IoT doesn't like newline character in device cert

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Describe the bug
To Reproduce (observed behavior)
step 1:
openssl genrsa -out ./private.key 2048
openssl req -new -key private.key -out ./sample.csr
aws iot create-certificate-from-csr --certificate-signing-request=file://sample.csr, in the response I will receive the certificatePem and I saved it to a file "cert.pem":
"certificatePem": "-----BEGIN CERTIFICATE-----\nblablablablablabla=\n-----END CERTIFICATE-----\n"
step 2: activate my device cert and attach IoT policy so I can test
step 3: mvn exec:java -pl samples/BasicPubSub -Dexec.mainClass=pubsub.PubSub -Dexec.args="--endpoint xxxxx-ats.iot.us-east-1.amazonaws.com --clientId clientID --cert cert.pem --key private.key"

Here is the error I got:
Exception encountered: software.amazon.awssdk.crt.CrtRuntimeException: TlsContext.tls_ctx_new: Failed to create new aws_tls_ctx (aws_last_error: AWS_IO_TLS_CTX_ERROR(1033), Failed to create tls context)

Workaround
If I log into IoT console and download the cert.pem there, it works fine. I was able to connect/pub/sub. The cert.pem which has issues contains literal newline character \n, the one downloaded have actually multiple lines. I am not sure if this is indeed an issue? Does the device certificate pem file has to follow:
-----BEGIN CERTIFICATE-----
blablablablablabla
-----END CERTIFICATE-----
NOT:
-----BEGIN CERTIFICATE-----\nblablablablablabla=\n-----END CERTIFICATE-----\n
?

SDK version number
1.2.0

Platform/OS/Hardware/Device
Linux+Windows

Does auto reconnect works same as before?

Platform/OS/Hardware/Device
What are you running the sdk on?
Android, with OS version 11 on Pixel 3a

Describe the question
We are migrating from {com.amazonaws:aws-android-sdk-iot:2.16.3 ,com.amazonaws:aws-android-sdk-core:2.16.3} to software.amazon.awssdk.iotdevicesdk:android:1.2.9

We were using autoReconnect = true (https://github.com/aws-amplify/aws-sdk-android/blob/main/aws-android-sdk-iot/src/main/java/com/amazonaws/mobileconnectors/iot/AWSIotMqttManager.java#L134), I couldn't find something similar in the new api.

Impact:
In case when the app starts and there is no internet connectivity, the connection doesn't change to successful automatically.

Can someone please point me to configuration for the v2 sdk to have the same auto-reconnect functionality?
If that’s not possible in the v2 sdk at the moment, will it be added in future versions?

Cannot connect to SOCKS proxy

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
A clear and concise description of what the bug is.
While using this new SDK I am able to connect to an HTTP proxy but when my proxy is SOCKS, it does not connect. I don't see a stack trace but it doesn't not connect and I don't see any communication through my proxy.
Is it expected that this SDK will support SOCKS proxy? Is there anything specific that has to be configured for SOCKS vs HTTP?

SDK version number
SDK: 1.2.4
CRT: 0.6.2

Platform/OS/Hardware/Device
What are you running the sdk on?
Samsung Galaxy S9
Android 8.0.0
Charles Proxy (free)

To Reproduce (observed behavior)
Steps to reproduce the behavior (please share code)
Attempt to run a sample application and connect to a SOCKS proxy.

Expected behavior
A clear and concise description of what you expected to happen.
Connect to the SOCKS proxy.

Logs/output
If applicable, add logs or error output.
No logs available.

Custom headers support for AWS iot custom authentication

Confirm by changing [ ] to [x] below:

Platform/OS/Hardware/Device
MacOS Catalina

Describe the question
I am trying to use the custom authentication (https://docs.aws.amazon.com/iot/latest/developerguide/custom-auth.html) to authenticate my devices.
As per the instructions, we need to set few headers such as

  • x-amz-customauthorizer-name: <authorizer-name>
  • x-amz-customauthorizer-signature: <token-signature>
  • <token-key-name>: <some-token>

I am trying to figure out how to set these headers in the SDK.
As per RawPubSub sample, setting --auth_params should set as headers but when I looked at the sample code, it does not use this argument when there is no username set.

Any guidance or sample would be of much help.

Not able to connect to AWS-IoT with custom CA and just-in-time-registration

Is your feature request related to a problem? Please describe.
I created a custom CA at AWS IoT, created a key value pair and activated just-in-time-registration for this CA. But I found no working solution for aws-iot to connect to AWS and activate the registration. The custom CA must be a part of the "keystore" but for me it seems like the existing CA-options only apply for the "truststore".
I could activate the registration with paho, hivemq und mosquitto cli - but not with aws-iot.

Describe the solution you'd like
A config option to activate the CA for the keystore.

Greengrass discovery sample should iteratively try connectivity entries

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
After performing discovery, the greengrass discovery sample picks a single connectivity entry to try. This is incorrect; it should iterate all of them until either a success or all fail. See all of the other greengrass discovery samples. Trying to connect based on an endpoint property sort is not going to succeed. For example, it fails in my setup where there's a raspberry pi running the greengrass core behind the same router as the desktop that I'm running the sample on.

SDK version number
1.2.8

Platform/OS/Hardware/Device
N/A

To Reproduce (observed behavior)
Attempt to use the discovery client when the gg core is not on the same machine as you, but is in the same subnet.

Expected behavior
Successful sample run.

Logs/output
Exception thrown: java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: socket connection refused.
java.util.concurrent.ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: socket connection refused.
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at greengrass.BasicDiscovery.main(BasicDiscovery.java:170)
Caused by: software.amazon.awssdk.crt.mqtt.MqttException: socket connection refused.
at software.amazon.awssdk.crt.mqtt.MqttClientConnection.onConnectionComplete(MqttClientConnection.java:132)
Complete!

AndroidKeyStore support

Hi,

I would like to use this sdk for an Android-Device.
Currently I am working on the fleet provisioning. For creating/storing the device' private key, I would like to use Android Keystore or KeyChain. (https://developer.android.com/training/articles/keystore)
The SDK Api seems to expect direct access to the private key, either as bytearray or file. This is something Android's Keystore does not support.

So my question is, can I use the Android KeyStore with this sdk, and if not, what is the recommended way to store the private key in a secure manner on an Android Device?

A technical detail for which I would greatly appreciate, but don't really expect, support:
Our current software is actually written in Python. We ported it to Android using python-for-android. (Including aws-iot-device-sdk-python-v2). Due to schedule constraints we would like to avoid re-writing everything in Java/Kotlin.
Can you think of way to do the above in python? :)

Thanks in advance and best regards,

Niklas

AWSIotMqttClient support for CredentialsProvider

In order to create instance of AWSIotMqttClient the one have to explicitly provide certificate
and private key (if certificate authentication is used for a device). The problem is that Mqtt cliet has
state (subscriptions), which is supposed to be kept between connections (client code does not care about
connection issues). But credentials may be subject to change, consider the following scenario:

Device uses two types of certificates: bootstrap and main. When connecting for the first time there is only
bootstrap certificate. After bootstrap procedure is done and device has got its main certificate, the bootstrap
certificate can be removed.

It would be nice to have AWSIotMqttClient instance created at device startup and injected it wherever needed.
The client code basicaly does not care about credentials type (bootstrap or main), it just need a client.
I could implement custom CredentialsProvider, which

  • searches first for main certificate in dedicated location
  • if failed, searches for bootstrap certificate in dedicated location
  • throws an error

and this is transparent for the rest of the client.

Original issue for V1: aws/aws-iot-device-sdk-java#143

Doesn't work with AWS credentials providers (e.g Cognito)

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug

Unable to use Cognito as a credentials provider for this library (at least straightforwardly) on Android

SDK version number

    implementation 'software.amazon.awssdk.iotdevicesdk:aws-iot-device-sdk:1.2.11'
    implementation 'com.amazonaws:aws-android-sdk-mobile-client:2.22.0'

Platform/OS/Hardware/Device

Android Studio 4
Mac OSX
Android 10

To Reproduce (observed behavior)

Initialise the cognito credentials provider. Plug that into the IoT MQTT Builder (compile error, AWSCredentialsProvider != CredentialsProvider

        val awsCredentialsProvider : AWSCredentialsProvider = CognitoCachingCredentialsProvider(
            context,
            "eu-west-1:xxxx-xxxxx-xxxxx-xxxx",  
            Regions.EU_WEST_1
        )
        

        val builder = AwsIotMqttConnectionBuilder.newDefaultBuilder().
            withWebsocketCredentialsProvider(awsCredentialsProvider) // This takes CredentialsProvider, not AWSCredentialsProvider
            .build()

At this juncture, I'm like ... oookkkkaaaaay ... I'll just extend CredentialsProvider with a custom class and provide some bridging code so I can join these two disparate AWS objects together since they are effectively the same thing ... i.e something like:

class CustomCognitoCredentialsProvider(val cognitoCachingCredentialsProvider: CognitoCachingCredentialsProvider) : CredentialsProvider() {

    override fun getCredentials(): CompletableFuture<Credentials> {

        val completableFuture = CompletableFuture<Credentials>()

        val awsSessionCredentials =  cognitoCachingCredentialsProvider.credentials

        completableFuture.complete(Credentials(awsSessionCredentials.awsAccessKeyId.toByteArray(),
            awsSessionCredentials.awsSecretKey.toByteArray(),
            awsSessionCredentials.sessionToken.toByteArray()))

        return completableFuture
    }

}

Result: Futures are only supported in Android >= 7.0, so I can't do that either.

Expected behavior

Should be able to plumb in an AWSCredentialsProvider, like most other AWS libraries.

Logs/output

n/a

Additional context

The only solution for us was to move back to v1 of the device SDK (which we do use elsewhere)

Support for older ARM processors.

Is it possible to add support for older ARM processors to the SDK and CRT? Such as Arm926EJ-S which is based on ARM v5 architecture?

We are currently using the V1 Java SDK and would like to upgrade to V2, however the SDK and CRT do not support our processor. This issue is related to AWS support ticket 7817113101.

Proposed changes to the API

I've been working with the SDK for a few months now. Here are a couple of problems with the API I think should be fixed:

  1. It's possible to pass in a proxy to AwsIotMqttConnectionBuilder and then not pass it to X509CredentialsProvider.X509CredentialsProviderBuilder. I can't think of a use case where I'd want the actual connection to go via proxy but not the authorization but even if there is it's probably the less common use case. It would be better to have a default where if a proxy is used it's used for everything. Otherwise you're providing the user with a footgun (which I used of course).
  2. It's possible to pass in a certificate to the AwsIotMqttConnectionBuilder as well as X509CredentialsProvider.X509CredentialsProviderBuilder in which case the connection will sometimes fail on TLS negotiation. The sensible default would've been allowing the user to pass the certificate to one builder, but not both.

How to Create an AWS IOT custom job

Hi there!
I'm trying to create an AWS IOT custom job in spring -boot framework by referring sample code.

here

In the sample code, I was looking for some keywords like create a job or create custom-job.
docs

unfortunately, I did not find any class/method/interface, which can help me to implement
my use case.

Can someone guide or point me in the right direction?.

MQTT Publish message with QoS0 always return success even if there is some error at network level

Confirm by changing [ ] to [x] below to ensure that it's a bug:

Known Issue

  • I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Describe the bug
While publishing any message asynchronously to MQTT with QoS0, it always return SUCCESS as a status .

SDK version number
2.14.11

Platform/OS/Hardware/Device
What are you running the sdk on?

Microsoft windows 10 Pro
x64 Based PC
RAM : 16GB
Processor: Intel(R) Core(TM) i5-7440HQ [email protected]

To Reproduce (observed behavior)
Can be reproduced with following code

@SneakyThrows
public static void main(String[] args) {
    publish("/test/testDevice", "HelloWorld");
}

private static void publish(String topic, String payload) {
    AWSIotMqttClient awsIotMqttClient = AWSIotMqttClientHelper.getInstance();
    /**
     * public class NonBlockingPublishListener extends AWSIotMessage {
     *    public NonBlockingPublishListener(String topic, AWSIotQos qos, String payload) {
     *         super(topic, qos, payload);
     *     }
     *
     *     @Override
     *     public void onSuccess() {
     *         System.out.println("Received success");

     *     }
     * }
     */
    NonBlockingPublishListener responseMessage = new NonBlockingPublishListener(topic, AWSIotQos.QOS0, payload);
    try {
        awsIotMqttClient.publish(responseMessage,12000);
    } catch (AWSIotException e) {
        e.printStackTrace();
    }
}

Expected behavior
In case of network disconnect, should receive a failure, instead of success

Additional context

Found similar issue reported for C# here:
aws/amazon-freertos#2155

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.