Giter VIP home page Giter VIP logo

aws-crt-cpp's Introduction

AWS Crt Cpp

C++ wrapper around the aws-c-* libraries. Provides Cross-Platform Transport Protocols and SSL/TLS implementations for C++.

Documentation

https://awslabs.github.io/aws-crt-cpp/

Currently Included:

  • aws-c-common: Cross-platform primitives and data structures.
  • aws-c-io: Cross-platform event-loops, non-blocking I/O, and TLS implementations.
  • aws-c-mqtt: MQTT client.
  • aws-c-auth: Auth signers such as Aws-auth sigv4
  • aws-c-http: HTTP 1.1 client, and websockets (H2 coming soon)
  • aws-checksums: Cross-Platform HW accelerated CRC32c and CRC32 with fallback to efficient SW implementations.
  • aws-c-event-stream: C99 implementation of the vnd.amazon.event-stream content-type.

More protocols and utilities are coming soon, so stay tuned.

Building

The C99 libraries are already included for your convenience as submodules. You should perform a recursive clone git clone --recursive or initialize the submodules via git submodule update --init. These dependencies are compiled by CMake as part of the build process.

If you want to manage these dependencies manually (e.g. you're using them in other projects), configure CMake with -DBUILD_DEPS=OFF and -DCMAKE_PREFIX_PATH=<install> pointing to the absolute path where you have them installed.

MSVC

If you want to use a statically linked MSVCRT (/MT, /MTd), you can add -DAWS_STATIC_MSVC_RUNTIME_LIBRARY=ON to your cmake configuration.

Apple Silicon (aka M1) and Universal Binaries

aws-crt-cpp supports both arm64 and x86_64 architectures. Configure cmake with -DCMAKE_OSX_ARCHITECTURES=arm64 to target Apple silicon, or -DCMAKE_OSX_ARCHITECTURES=x86_64 to target Intel. If you wish to create a universal binary, you should use lipo to combine the x86_64 and arm64 binaries. For example: lipo -create -output universal_app x86_app arm_app

You SHOULD NOT build for both architectures simultaneously via -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64". aws-crt-cpp has not been tested in this configuration. aws-crt-cpp's cmake configuration scripts are known to get confused by this, and will not enable optimizations that would benefit an independent arm64 or x86_64 build.

OpenSSL and LibCrypto (Unix only)

If your application uses OpenSSL, configure with -DUSE_OPENSSL=ON.

aws-crt-cpp does not use OpenSSL for TLS. On Apple and Windows devices, the OS's default TLS library is used. On Unix devices, s2n-tls is used. But s2n-tls uses libcrypto, the cryptography math library bundled with OpenSSL. To simplify the build process, the source code for s2n-tls and libcrypto are included as git submodules and built along with aws-crt-cpp. But if your application is also loading the system installation of OpenSSL (i.e. your application uses libcurl which uses libssl which uses libcrypto) there may be crashes as the application tries to use two different versions of libcrypto at once.

Setting -DUSE_OPENSSL=ON will cause aws-crt-cpp to link against your system's existing libcrypto, instead of building its own copy.

You can ignore all this on Windows and Apple platforms, where aws-crt-cpp uses the OS's default libraries for TLS and cryptography math.

Dependencies?

There are no non-OS dependencies that AWS does not own, maintain, and ship.

Common Usage

To do anything with IO, you'll need to create a few objects that will be used by the rest of the library.

For example:

    Aws::Crt::LoadErrorStrings();

Will load error strings for debugging purposes. Since the C libraries use error codes, this will allow you to print the corresponding error string for each error code.

    Aws::Crt::ApiHandle apiHandle;

This performs one-time static initialization of the library. You'll need it to do anything, so don't forget to create one.

    Aws::Crt::Io::EventLoopGroup eventLoopGroup(<number of threads you want>);

To use any of our APIs that perform IO you'll need at least one event-loop. An event-loop group is a collection of event-loops that protocol implementations will load balance across. If you won't have very many connections (say, more than 100 or so), then you most likely only want 1 thread. In this case, you want to pass a single instance of this to every client or server implementation of a protocol you use in your application. In some advanced use cases, you may want to reserve a thread for different types of IO tasks. In that case, you can have an instance of this class for each reservation.

     Aws::Crt::Io::TlsContextOptions tlsCtxOptions =
        Aws::Crt::Io::TlsContextOptions::InitClientWithMtls(certificatePath.c_str(), keyPath.c_str());
    /*
     * If we have a custom CA, set that up here.
     */
    if (!caFile.empty())
    {
        tlsCtxOptions.OverrideDefaultTrustStore(nullptr, caFile.c_str());
    }

    uint32_t port = 8883;
    if (Io::TlsContextOptions::IsAlpnSupported())
    {
        /*
        * Use ALPN to negotiate the mqtt protocol on a normal
        * TLS port if possible.
        */
        tlsCtxOptions.SetAlpnList("x-amzn-mqtt-ca");
        port = 443;
    }

    Aws::Crt::Io::TlsContext tlsCtx(tlsCtxOptions, Io::TlsMode::CLIENT);

If you plan on using TLS, you will need a TlsContext. These are NOT CHEAP, so use as few as possible to perform your task. If you're in client mode and not doing anything fancy (e.g. mutual TLS), then you can likely get away with using a single instance for the entire application.

Aws::Crt::Io::ClientBootstrap bootstrap(eventLoopGroup);

Lastly, you will need a client or server bootstrap to use a client or server protocol implementation. Since everything is non-blocking and event driven, this handles most of the "callback hell" inherent in the design. Assuming you aren't partitioning threads for particular use-cases, you can have a single instance of this that you pass to multiple clients.

Mac-Only TLS Behavior

Please note that on Mac, once a private key is used with a certificate, that certificate-key pair is imported into the Mac Keychain. All subsequent uses of that certificate will use the stored private key and ignore anything passed in programmatically. Beginning in v0.8.10, when a stored private key from the Keychain is used, the following will be logged at the "info" log level:

static: certificate has an existing certificate-key pair that was previously imported into the Keychain.  Using key from Keychain instead of the one provided.

License

This library is licensed under the Apache 2.0 License.

aws-crt-cpp's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-crt-cpp's Issues

Cannot build s2n when -DUSE_OPENSSL=ON

With command bellow

AWS_IOT_SDK_FOLDERNAME="aws-iot-device-sdk-cpp"
AWS_IOT_SDK_URL="https://github.com/aws/aws-iot-device-sdk-cpp-v2.git"
AWS_IOT_SDK_VER="1.18.6"

  rm -rf awsbuild &&
  git clone --recursive ${AWS_IOT_SDK_URL} --branch v${AWS_IOT_SDK_VER} ${AWS_IOT_SDK_FOLDERNAME} &&
  mkdir awsbuild &&
  cd awsbuild &&
  cmake -DUSE_OPENSSL=ON \
      -DCMAKE_INSTALL_PREFIX="$(pwd)/../dependencies/aws-iot-device-sdk" \
      -Dcrypto_LIBRARY="$(pwd)/../openssl/libcrypto.a" \
      -Dcrypto_INCLUDE_DIR="$(pwd)/../openssl/include" \
      -DOPENSSL_CRYPTO_LIBRARY="$(pwd)/../openssl/libcrypto.a" \
      -DOPENSSL_INCLUDE_DIR="$(pwd)/../openssl/include" \
      -DCMAKE_BUILD_TYPE="Release" ../${AWS_IOT_SDK_FOLDERNAME} &&
  cmake --build . --target install

I am getting this error:

-- madvise() support detected
-- clone() support detected
-- Found crypto: /home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/awsbuild/../dependencies/openssl/libcrypto.a  
-- LibCrypto Include Dir: /home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/dependencies/openssl/include
-- LibCrypto Shared Lib:  crypto_SHARED_LIBRARY-NOTFOUND
-- LibCrypto Static Lib:  crypto_STATIC_LIBRARY-NOTFOUND
Using libcrypto from the cmake path.
CMake Error at /home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/awsbuild/CMakeFiles/CMakeTmp/CMakeLists.txt:18 (add_executable):
  Target "cmTC_23663" links to target "AWS::crypto" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at crt/aws-crt-cpp/crt/s2n/CMakeLists.txt:496 (try_compile):
  Failed to generate test project build system.


-- Configuring incomplete, errors occurred!
See also "/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/awsbuild/CMakeFiles/CMakeOutput.log".
See also "/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/awsbuild/CMakeFiles/CMakeError.log".

MSVC Linker error

Describe the bug

I am trying to compile the library, using CMake and MSVC, but I am getting a linker error:

example/mqttclient/mqttclient.exe 
cmd.exe /C "cd . && "C:\Program Files\JetBrains\CLion 2021.3.3\bin\cmake\win\x64\bin\cmake.exe" -E vs_link_exe --intdir=example\mqttclient\CMakeFiles\mqttclient.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100203~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100203~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\LLVM\bin\lld-link.exe /nologo @CMakeFiles\mqttclient.rsp  /out:example\mqttclient\mqttclient.exe /implib:example\mqttclient\mqttclient.lib /pdb:example\mqttclient\mqttclient.pdb /version:0.0 /machine:X86 /debug /INCREMENTAL /subsystem:console  && cd ."
LINK Pass 1: command "C:\PROGRA~1\LLVM\bin\lld-link.exe /nologo @CMakeFiles\mqttclient.rsp /out:example\mqttclient\mqttclient.exe /implib:example\mqttclient\mqttclient.lib /pdb:example\mqttclient\mqttclient.pdb /version:0.0 /machine:X86 /debug /INCREMENTAL /subsystem:console /MANIFEST /MANIFESTFILE:example\mqttclient\CMakeFiles\mqttclient.dir/intermediate.manifest example\mqttclient\CMakeFiles\mqttclient.dir/manifest.res" failed (exit code 1) with the following output:
lld-link: warning: aws-c-common.lib(allocator.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(allocator.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(allocator.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(allocator.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(error.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(error.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(error.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(thread_shared.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: _fwrite (defined in libucrtd.lib(fwrite.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: __errno (defined in libucrtd.lib(errno.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(logging.c.obj): locally defined symbol imported: _fclose (defined in libucrtd.lib(fclose.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(byte_buf.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(encoding.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(assert.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(assert.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(assert.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(assert.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(array_list.c.obj): locally defined symbol imported: _qsort (defined in libucrtd.lib(qsort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(array_list.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(hash_table.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(common.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(common.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(common.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(json.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(mutex.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(condition_variable.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(clock.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(condition_variable.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(thread.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(string.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_writer.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_writer.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_writer.c.obj): locally defined symbol imported: _fwrite (defined in libucrtd.lib(fwrite.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_writer.c.obj): locally defined symbol imported: __errno (defined in libucrtd.lib(errno.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_writer.c.obj): locally defined symbol imported: _fclose (defined in libucrtd.lib(fclose.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_formatter.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_formatter.c.obj): locally defined symbol imported: ___stdio_common_vsnprintf_s (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(log_channel.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: _fseek (defined in libucrtd.lib(fseek.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __errno (defined in libucrtd.lib(errno.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: _fclose (defined in libucrtd.lib(fclose.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: _ftell (defined in libucrtd.lib(ftell.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: _fread (defined in libucrtd.lib(fread.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cpuid.c.obj): locally defined symbol imported: _getenv (defined in libucrtd.lib(getenv.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cpuid.c.obj): locally defined symbol imported: _atoi (defined in libucrtd.lib(atox.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cpuid.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(encoding_avx2.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(system_info.c.obj): locally defined symbol imported: _fflush (defined in libucrtd.lib(fflush.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(system_info.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(system_info.c.obj): locally defined symbol imported: ___acrt_iob_func (defined in libucrtd.lib(_file.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(system_info.c.obj): locally defined symbol imported: ___stdio_common_vfprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(ref_count.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(ref_count.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(task_scheduler.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _malloc (defined in libucrtd.lib(malloc.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _free (defined in libucrtd.lib(free.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _realloc (defined in libucrtd.lib(realloc.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _strncmp (defined in libucrtd.lib(strncmp.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _strtod (defined in libucrtd.lib(strtod.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: __dclass (defined in libucrtd.lib(nan.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: ___stdio_common_vsscanf (defined in libucrtd.lib(input.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cJSON.c.obj): locally defined symbol imported: _tolower (defined in libucrtd.lib(tolower_toupper.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(date_time.c.obj): locally defined symbol imported: _strtol (defined in libucrtd.lib(strtox.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(date_time.c.obj): locally defined symbol imported: _strftime (defined in libucrtd.lib(strftime.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(date_time.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(date_time.c.obj): locally defined symbol imported: _tolower (defined in libucrtd.lib(tolower_toupper.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __wfopen_s (defined in libucrtd.lib(fopen.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: _wcsnlen (defined in libucrtd.lib(strnlen.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __fseeki64 (defined in libucrtd.lib(fseek.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __errno (defined in libucrtd.lib(errno.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __fileno (defined in libucrtd.lib(fileno.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: __get_osfhandle (defined in libucrtd.lib(osfinfo.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(file.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(device_random.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(lru_cache.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cache.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(cpuid.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(device_random.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(device_random.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(statistics.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(priority_queue.c.obj): locally defined symbol imported: _abort (defined in libucrtd.lib(abort.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(priority_queue.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(uri.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(uri.c.obj): locally defined symbol imported: _atoi (defined in libucrtd.lib(atox.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(environment.c.obj): locally defined symbol imported: _getenv (defined in libucrtd.lib(getenv.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(environment.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(time.c.obj): locally defined symbol imported: __localtime64_s (defined in libucrtd.lib(localtime.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(time.c.obj): locally defined symbol imported: __gmtime64_s (defined in libucrtd.lib(gmtime.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(time.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(uuid.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(uuid.c.obj): locally defined symbol imported: ___stdio_common_vsscanf (defined in libucrtd.lib(input.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(linked_hash_table.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(rw_lock.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: warning: aws-c-common.lib(fifo_cache.c.obj): locally defined symbol imported: ___stdio_common_vsprintf (defined in libucrtd.lib(output.obj)) [LNK4217]
lld-link: error: undefined symbol: __declspec(dllimport) __aligned_malloc
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\allocator.c:59
>>>               aws-c-common.lib(allocator.c.obj):(_s_default_malloc)

lld-link: error: undefined symbol: __declspec(dllimport) __aligned_free
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\allocator.c:70
>>>               aws-c-common.lib(allocator.c.obj):(_s_default_free)

lld-link: error: undefined symbol: __declspec(dllimport) __aligned_realloc
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\allocator.c:96
>>>               aws-c-common.lib(allocator.c.obj):(_s_default_realloc)

lld-link: error: undefined symbol: __except_handler4_common
>>> referenced by d:\a01\_work\38\s\src\vctools\crt\vcstartup\src\eh\i386\chandler4gs.c:94
>>>               msvcrtd.lib(chandler4gs.obj):(__except_handler4)

lld-link: error: undefined symbol: __declspec(dllimport) _modf
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\date_time.c:191
>>>               aws-c-common.lib(date_time.c.obj):(_aws_date_time_init_epoch_secs)

lld-link: error: undefined symbol: __declspec(dllimport) __mktime64
>>> referenced by C:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt\time.h:515
>>>               aws-c-common.lib(date_time.c.obj):(_mktime)

lld-link: error: undefined symbol: __declspec(dllimport) __putenv_s
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\windows\environment.c:41
>>>               aws-c-common.lib(environment.c.obj):(_aws_set_environment_value)
>>> referenced by C:\Users\andrio01\Documents\git\drivers\build\_deps\aws-crt-cpp-src\crt\aws-c-common\source\windows\environment.c:51
>>>               aws-c-common.lib(environment.c.obj):(_aws_unset_environment_value)

lld-link: error: undefined symbol: __declspec(dllimport) __mkgmtime64
>>> referenced by C:\Program Files (x86)\Windows Kits\10\include\10.0.20348.0\ucrt\time.h:507
>>>               aws-c-common.lib(time.c.obj):(__mkgmtime)
ninja: build stopped: subcommand failed.

I set the variables:
set(BUILD_SHARED_LIBS OFF)
set(AWS_STATIC_MSVC_RUNTIME_LIBRARY ON)
set(AWS_WARNINGS_ARE_ERRORS OFF)

And my executable is also has the compiler option /MTd

I am not sure how to proceed. Can anyone give any pointers?

Expected Behavior

To compile

Current Behavior

Linker error described above

Reproduction Steps

Build the project with CMake

Possible Solution

No response

Additional Information/Context

No response

aws-crt-cpp version used

0.23.1

Compiler and version used

clang-cl 13.0.1

Operating System and version

Windows 10 x64

MqttClientConnectionConfigBuilder issue

I come from repo https://github.com/aws/aws-iot-device-sdk-cpp-v2 which indeed uses the present repo.

I have created my own code based on this sample.

When trying to test my code using google test, I tested my whole class IotServer several times within the same test file, which implies the establishment of the whole process one and again (setup credentials, setup mqtt client, new connection, etc, and the the shutdown process):

I might have a problem with function.

Aws::Iot::MqttClientConnectionConfigBuilder::Build()

This is the piece of code in which I call that function:

boost::optional<Aws::Iot::MqttClientConnectionConfig> IotServer::setup_credentials(
    const std::string &certificate,
    const std::string &key,
    const std::string &cafile,
    const std::string &endpoint)
{
    RCLCPP_INFO(logger_, "Trying to setup credentials");
    Aws::Iot::MqttClientConnectionConfigBuilder builder(
        certificate.c_str(),
        key.c_str()
    );
    builder.WithCertificateAuthority(cafile.c_str());
    builder.WithEndpoint(endpoint.c_str());

    boost::optional<Aws::Iot::MqttClientConnectionConfig> clientConfig = builder.Build();
    if (!clientConfig.get()) {
        RCLCPP_ERROR(
            logger_,
            "Client Configuration initialization failed with error %s",
            Aws::Crt::ErrorDebugString(clientConfig.get().LastError()));
        return boost::none;
    } else {
        return clientConfig;
    }
}

When calling my class constructor for the first time (which uses the function setup_credentials), everything works correctly and the connection is correctly established. After that I make shutdown actions (disconnect,...)

Within the same test file I call the my class constructor for the second time (with same conditions, same certificate, etc) and it fails with error AWS_ERROR_SUCCESS.

Moreover I was able to get logs from the Aws::Crt::ApiHandle

[INFO] [2021-01-21T08:32:59Z] [00007fa73cd68440] [event-loop] - id=0x557567d5e6a0: Initializing edge-triggered epoll
[INFO] [2021-01-21T08:32:59Z] [00007fa73cd68440] [event-loop] - id=0x557567d5e6a0: Using eventfd for cross-thread notifications.
[TRACE] [2021-01-21T08:32:59Z] [00007fa73cd68440] [event-loop] - id=0x557567d5e6a0: eventfd descriptor 20.
[INFO] [2021-01-21T08:32:59Z] [00007fa73cd68440] [event-loop] - id=0x557567d5e6a0: Starting event-loop thread.
[INFO] [2021-01-21T08:32:59Z] [00007fa73cd68440] [dns] - id=0x5575680e5230: Initializing default host resolver with 1 max host entries.
[INFO] [2021-01-21T08:32:59Z] [00007fa73cd68440] [channel-bootstrap] - id=0x557567d5f310: Initializing client bootstrap with event-loop group 0x557567d5e630
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [mqtt-client] - client=0x557567d5f4a0: Initalizing MQTT client
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [channel-bootstrap] - id=0x557567d5f310: releasing bootstrap reference
[INFO] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: main loop started
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: subscribing to events on fd 20
[INFO] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: default timeout 100000, and max events to process per tick 100
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: waiting for a maximum of 100000 ms
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [tls-handler] - ctx: Certificate and key have been set, setting them up now.
[ERROR] [2021-01-21T08:32:59Z] [00007fa73cd68440] [tls-handler] - ctx: configuration error size mismatch (Error encountered in /home/sr-azu/repo/aws/aws-iot-device-sdk-cpp-v2/crt/aws-crt-cpp/crt/s2n/crypto/s2n_rsa.c line 94)
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [mqtt-client] - client=0x557567d5f4a0: Cleaning up MQTT client
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [channel-bootstrap] - id=0x557567d5f310: releasing bootstrap reference
[DEBUG] [2021-01-21T08:32:59Z] [00007fa73cd68440] [channel-bootstrap] - id=0x557567d5f310: destroying
[INFO] [2021-01-21T08:32:59Z] [00007fa72d9a5700] [event-loop] - id=0x557567d5e6a0: Destroying event_loop
[INFO] [2021-01-21T08:32:59Z] [00007fa72d9a5700] [event-loop] - id=0x557567d5e6a0: Stopping event-loop thread.
[TRACE] [2021-01-21T08:32:59Z] [00007fa72d9a5700] [event-loop] - id=0x557567d5e6a0: Scheduling task 0x557567d5ec58 cross-thread for timestamp 0
[TRACE] [2021-01-21T08:32:59Z] [00007fa72d9a5700] [event-loop] - id=0x557567d5e6a0: Waking up event-loop thread
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: wake up with 1 events to process.
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: activity on fd 20, invoking handler.
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: notified of cross-thread tasks to schedule
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: processing cross-thread tasks
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: task 0x557567d5ec58 pulled to event-loop, scheduling now.
[DEBUG] [2021-01-21T08:32:59Z] [00007fa735318700] [task-scheduler] - id=0x557567d5ec58: Scheduling epoll_event_loop_stop task for immediate execution
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: running scheduled tasks.
[DEBUG] [2021-01-21T08:32:59Z] [00007fa735318700] [task-scheduler] - id=0x557567d5ec58: Running epoll_event_loop_stop task with <Running> status
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: no more scheduled tasks using default timeout.
[DEBUG] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: exiting main loop
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: un-subscribing from events on fd 20
[TRACE] [2021-01-21T08:32:59Z] [00007fa735318700] [event-loop] - id=0x557567d5e6a0: scheduling task 0x7fa71c000e60 in-thread for timestamp 0
[DEBUG] [2021-01-21T08:32:59Z] [00007fa735318700] [task-scheduler] - id=0x7fa71c000e60: Scheduling epoll_event_loop_unsubscribe_cleanup task for immediate execution
[DEBUG] [2021-01-21T08:32:59Z] [00007fa72d9a5700] [task-scheduler] - id=0x7fa71c000e60: Running epoll_event_loop_unsubscribe_cleanup task with <Canceled> status

May you have any hint on the issue?

I know it seems to be an issue related to s2n, but it is true that this potential issue is not tested so I think it can be a contribution for present repo.

Please let me know if this repo is not the one which I should open present repo.

Crash in TlsContext::NewConnectionOptions() due to dereferencing a null pointer

We've observed in some cases the underlying std::shared_ptr<aws_tls_ctx> m_ctx member of TlsContext may be storing a null when TlsContext::NewConnectionOptions() is called by MqttClient::NewConnection(), which will cause the application crash. If in

return TlsConnectionOptions(m_ctx.get(), m_ctx->alloc);
the member variable m_ctx is storing null pointer, accessing m_ctx->alloc will crash in most compilers.

Issues when building with cmake 3.16

I was trying to compile this project as a shared library on Arch Linux with cmake 3.16 installed and ran into some complications along the way.
I realize that none of the issues below are from this repository directly, however as the issue spans over multiple repositories and I wanted to keep things coherent this seemed as good a place as any to create the issue.
I'd be happy to open issues on the respective repositories, if needed.

Here's the rundown:
My environment:

 $ uname -a
Linux workstation 5.4.5-arch1-1 #1 SMP PREEMPT Wed, 18 Dec 2019 19:48:51 +0000 x86_64 GNU/Linux
 $ cmake --version
cmake version 3.16.1

Steps used to build:

cd aws-crt-cpp/;
mkdir build install;
cd build;
cmake -DCMAKE_INSTALL_PREFIX="$PWD/../install" -DBUILD_SHARED_LIBS=ON  -DBUILD_DEPS=ON ../;
cmake --build . --target install;

This builds just fine, however it leaves the cmake-configs in an awkard state. Trying to include the package in another project yields the following:

CMake Error at aws-crt-cpp/install/lib/aws-c-compression/cmake/aws-c-compression-config.cmake:6 (include):
  include could not find load file:

    <path-to-project>/includes-aws-crt-cpp/aws-crt-cpp/install/lib/aws-c-compression/cmake/shared/aws-crt-cpp-targets.cmake

This is because the template for generating the config file for those packages uses @CMAKE_PROJECT_NAME@ for building the path to the targets file. The behavior of this variable seems to have changed at v3.12 (compared to the previous entry v3.11) now giving the topmost project's name. This is true for all submodules except aws-c-io and s2n, where the variable has been changed to @PROJECT_NAME@ in commits awslabs/aws-c-io@250484c and aws/s2n-tls@e50cf9c respectively.

When building this project through awslabs/aws-iot-device-sdk-cpp-v2, I additionally had to deal with this:

CMake Error at jobs/CMakeLists.txt:97 (target_link_libraries):
  Error evaluating generator expression:

    $<TARGET_PROPERTY:LibCrypto::Crypto,INTERFACE_INCLUDE_DIRECTORIES>

  Target "LibCrypto::Crypto" not found.

The error seems to originate from an oddity in how the TARGET_PROPERTY computation expression is looking up symbols. The problem and a workaround is described in this issue on the cmake gitlab. Again this didn't happen in the ubuntu container (cmake v3.10), however i couldn't find a documented change in how TARGET_PROPERTY operates, so I can't pinpoint where the regression happened. While the issue remains open, the described workaround resolves the problem for me:

# s2n/CMakeLists.txt
-target_include_directories(${PROJECT_NAME} PUBLIC $<TARGET_PROPERTY:LibCrypto::Crypto,INTERFACE_INCLUDE_DIRECTORIES>)
+get_target_property(LIBCRYPTO_INTERFACE_INCLUDE_DIRECTORIES LibCrypto::Crypto INTERFACE_INCLUDE_DIRECTORIES)
+target_include_directories(${PROJECT_NAME} PUBLIC "${LIBCRYPTO_INTERFACE_INCLUDE_DIRECTORIES}")

Now I didn't have much to do with cmake before trying to integrate with awslabs/aws-iot-device-sdk-cpp-v2, so this has been somewhat of a learning experience for me and I'm not sure if the steps i took are correct. However I would be happy to open up pull requests at the respective repositories if it helps anyone.

Mqtt::ReturnCode Bug in sample

Hi @graebm / @JonathanHenson ,

the recent updates,

  1. The Mqtt::ConnectionState enum has been removed from the latest version of the SDK
  2. Introduced Mqtt::ReturnCode

If you look into this sample code both,

  1. In onConnectionCompleted callback lambda it is treating if there is no errorCode means connection is successful and just printing the connection completed message with returnCode
  2. In onResumed callback lambda it is not even checking the returnCode and just printing the message
  3. In both cases it is assuming that the returnCode is always gives succes but If we look into the mqtt.h file there is a possibility of getting returnCode with different errors(We tested it we got 4/5 sometimes)
  4. Our further business logic depends on this initial mqtt connection status and connection resume status, So could you please look into this.
/* Result of a connect request [MQTT-3.2.2.3]. */
enum aws_mqtt_connect_return_code {
    AWS_MQTT_CONNECT_ACCEPTED,
    AWS_MQTT_CONNECT_UNACCEPTABLE_PROTOCOL_VERSION,
    AWS_MQTT_CONNECT_IDENTIFIER_REJECTED,
    AWS_MQTT_CONNECT_SERVER_UNAVAILABLE,
    AWS_MQTT_CONNECT_BAD_USERNAME_OR_PASSWORD,
    AWS_MQTT_CONNECT_NOT_AUTHORIZED,
    /* reserved = 6 - 255 */
};
    /*
     * This will execute when an mqtt connect has completed or failed.
     */
    auto onConnectionCompleted = [&](Mqtt::MqttConnection &, int errorCode, Mqtt::ReturnCode returnCode, bool) {
        if (errorCode)
        {
            fprintf(stdout, "Connection failed with error %s\n", ErrorDebugString(errorCode));
            std::lock_guard<std::mutex> lockGuard(mutex);
            connectionSucceeded = false;
        }
        else
        {
            fprintf(stdout, "Connection completed with return code %d\n", returnCode);
            connectionSucceeded = true;
        }
        {
            std::lock_guard<std::mutex> lockGuard(mutex);
            connectionCompleted = true;
        }
        conditionVariable.notify_one();
    };

auto onResumed = [&](Mqtt::MqttConnection &, Mqtt::ReturnCode, bool) { fprintf(stdout, "Connection resumed\n"); };

Thank you.

Regards,
Ravichandhra Palla.

CMake Error at CMakeLists.txt:317 (aws_add_sanitizers): Unknown CMake command "aws_add_sanitizers"

Describe the bug

Configure fails:

===>  Configuring for aws-crt-cpp-0.24.0
===>  Performing out-of-source build
/bin/mkdir -p /wrkdirs/usr/ports/devel/aws-crt-cpp/work/.build
-- AWS CRT C++ 0.24.0
-- The CXX compiler identification is Clang 14.0.5
-- The C compiler identification is Clang 14.0.5
<snip>
-- aws-c-cal found target: AWS::crypto
-- crypto Include Dir: /usr/include
-- aws-c-cal found target: AWS::crypto
-- crypto Include Dir: /usr/include
-- S2N found target: AWS::crypto
-- crypto Include Dir: /usr/include
-- S2N found target: AWS::crypto
-- crypto Include Dir: /usr/include
-- aws-c-cal found target: AWS::crypto
-- crypto Include Dir: /usr/include
CMake Error at CMakeLists.txt:317 (aws_add_sanitizers):
  Unknown CMake command "aws_add_sanitizers".


-- Configuring incomplete, errors occurred!
*** Error code 1

Configure fails:

Expected Behavior

configure ok

Current Behavior

configure fails related to CMAkeLists: 317

aws_add_sanitizers(${PROJECT_NAME})

Reproduction Steps

cmake

Possible Solution

No response

Additional Information/Context

No response

aws-crt-cpp version used

0.24.0

Compiler and version used

clang 14.0.5

Operating System and version

FreeBSD 13.2 amd64

Program hangs with Aws::Iot::MqttClient:NewConnection()

Describe the bug

It is likely that this is user error rather than a bug. If so, please educate me on what I am doing wrong here.

Here's my simplified code:

// Some code adopted from https://github.com/aws/aws-iot-device-sdk-cpp-v2/blob/main/samples/pub_sub/basic_pub_sub/main.cpp

#include <aws/crt/Api.h>
#include <aws/iot/MqttClient.h>
#include <iostream>

#define DEBUG_PRINT std::cout << __FILE__ << ":" << __LINE__ << "\n" << std::flush;

class AwsIotMqttHandler
{
    public:

        typedef struct
        {
            Aws::Crt::String certificateFilepath;
            Aws::Crt::String privateKeyFilepath;
            Aws::Crt::String rootCertificateFilepath;
            Aws::Crt::String endpoint;
        } connectionParameters_t;

        AwsIotMqttHandler();

        void InitializeConnection(connectionParameters_t connectionParameters);

    protected:

        std::shared_ptr<Aws::Crt::Mqtt::MqttConnection> m_connection;
};

AwsIotMqttHandler::AwsIotMqttHandler()
{
}

void AwsIotMqttHandler::InitializeConnection(connectionParameters_t connectionParameters)
{
    // Do some global initialization for the API
    Aws::Crt::ApiHandle apiHandle;

    // Create the MQTT builder and populate it with connection parameters
    auto clientConfigBuilder =
        Aws::Iot::MqttClientConnectionConfigBuilder(connectionParameters.certificateFilepath.c_str(),
                                                    connectionParameters.privateKeyFilepath.c_str());
    clientConfigBuilder.WithEndpoint(connectionParameters.endpoint);
    clientConfigBuilder.WithCertificateAuthority(connectionParameters.rootCertificateFilepath.c_str());
    DEBUG_PRINT

    // Create the MQTT connection from the MQTT builder
    Aws::Iot::MqttClientConnectionConfig clientConfig = clientConfigBuilder.Build();
    if (!clientConfig)
    {
        std::cerr << std::string("Client configuration initialization failed with error: ")
                        + std::string(Aws::Crt::ErrorDebugString(clientConfig.LastError()));
        return;
    }
    DEBUG_PRINT
    Aws::Iot::MqttClient client = Aws::Iot::MqttClient();
    m_connection = client.NewConnection(clientConfig);
    DEBUG_PRINT
    if (m_connection != nullptr && !*m_connection)
    {
        std::cerr << std::string("MQTT connection creation failed with error: ")
                        + std::string(Aws::Crt::ErrorDebugString(m_connection->LastError()));
        return;
    }
    DEBUG_PRINT
}

int main(int argc, char** argv)
{
    std::cout << "\nStarting: " << argv[0] << "\n";

    DEBUG_PRINT

    AwsIotMqttHandler::connectionParameters_t connectionParameters;
    connectionParameters.certificateFilepath     = "/root/gateway/aws_keys/certificate.pem.crt";
    connectionParameters.privateKeyFilepath      = "/root/gateway/aws_keys/private.pem.key";
    connectionParameters.rootCertificateFilepath = "/root/gateway/aws_keys/AmazonRootCA1.pem";
    connectionParameters.endpoint                = "a1ivqb4fk63pz7-ats.iot.us-east-1.amazonaws.com";

    AwsIotMqttHandler awsIotMqttHandler;
    awsIotMqttHandler.InitializeConnection(connectionParameters);

    DEBUG_PRINT

    return 0;
}

Expected Behavior

Print


Starting: ./aws_mqtt_connection_hang.exe
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:72
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:45
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:55
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:58
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:65
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:83

and exit. This is what happens if I comment out the m_connection = client.NewConnection(clientConfig); line.

Current Behavior

Prints


Starting: ./aws_mqtt_connection_hang.exe
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:72
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:45
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:55
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:58
/root/gateway/experiments/aws_mqtt_connection_hang.cpp:65

(no /root/gateway/experiments/aws_mqtt_connection_hang.cpp:83 line) and and does not exit. Three threads are left running:
aws_mqtt_connec
AwsEventLoop 1
AwsEventLoop 2

Reproduction Steps

  1. Change the filepaths and endpoint in the main function.
  2. Build the code under g++-13 c++20.
  3. Run the code.

Possible Solution

No response

Additional Information/Context

No response

aws-crt-cpp version used

AWS CRT C++ 0.24.3

Compiler and version used

g++-13 (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0

Operating System and version

Ubuntu 22.04.2 LTS

Compilation failing on Centos 6 with GCC 7.2.0

Hello AWS CRT,
I have been attempting to update our AWS SDK to version 1.9.278 and have installed the aws-crt-cpp using the script supplied by that project. The build of the AWS SDK fails when it tries to build aws-c-common on my development VM.

I attempted to build just that project standalone and hit the same problem so I'd appreciate some help. Here's what I have:
Centos 6 VM (we build using our product's oldest supported platform)
GCC 7.2.0

I have tried an out of source build of just the aws-c-common library and I get the same error. My cmake configuration line is as follows:
cmake -G "Unix Makefiles" -DCMAKE_C_COMPILER=/tools64/v2.5.0/gcc-7.2.0/bin/gcc ../aws-c-common
This completes successfully with the following output:
aws-crt-cpp-cmake-configure-output.txt

Trying to build the library then gives the following errors (related to avx and avx2) I think.
aws-crt-cpp-make errors.txt

According to the cpuinfo on the centos VM avx and avx2 are supported:
$> grep avx2 /proc/cpuinfo

Results in ..
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good xtopology unfair_spinlock eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch xsaveopt invpcid_single ssbd pti ibrs ibpb stibp fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f rdseed adx avx512cd flush_l1d

Any help appreciated. If I'e missed any essential information from this post please let me know and I'll add it.
Regards,
P

Double Free / Segfault Issues

I found that cleanup inside gtest seems to have some issues. This code snippet demonstrates the double free issue - I have also had segfaults, but I have been unable to reproduce this with a minimal example.

TEST(JsonCleanupTest, testNoDoubleFree) {
  Aws::Crt::JsonObject obj;
  Aws::Crt::Vector<Aws::Crt::JsonObject> inner_array;
  for (int i = 0; i < 2000; i++) {
    Aws::Crt::JsonObject f;
    f.AsDouble((double)i);
    inner_array.push_back(f);
  }
  obj.WithArray("array", inner_array);
}

CMake Deprecation Warnings

Describe the bug

I am using v0.26.4 and getting many warnings like this when building:

CMake Deprecation Warning at libraries/third_party/aws-crt-cpp/crt/aws-c-sdkutils/CMakeLists.txt:4 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

The fix is given the last two lines there.

Expected Behavior

No warnings when building

Current Behavior

See above

Reproduction Steps

Build with the library

Possible Solution

No response

Additional Information/Context

No response

aws-crt-cpp version used

0.26.4

Compiler and version used

gcc version 13.2.0 (Ubuntu 13.2.0-4ubuntu3)

Operating System and version

Ubuntu 23.10

Encountered AWS_ERROR_NO_PERMISSION when I tried to use CRT in AWS CLI to sync files from local to S3

Describe the bug

I am trying to use aws s3 sync to backup files from my servers to S3. To utilize the bandwidth available, I configure the CLI tool to with preferred_transfer_client = crt. From initial testing, the transfer speed can go up to 600MB/s so I decide to adopt this client for the actual backup job.

I ran some command similar to the following:

aws s3 sync <some-readonly-file> s3://backup-store

However, I saw the following error:

upload failed: <some-readonly-file> to s3://backup-store/<some-readonly-file> 43 (AWS_ERROR_NO_PERMISSION): User does not have permission to perform the requested action.

I then re-run the cmd with debug mode on and saw the following detailed error log:

[ERROR] [2023-07-04T03:23:54Z] [00007f2a05976b80] [common-io] - static: Failed to open file. path:'/data/<some-readonly-file>' mode:'r+b' errno:13 aws-error:43(AWS_ERROR_NO_PERMISSION)

I suspect it is due to the codes in https://github.com/awslabs/aws-c-io/blob/1f9a085027c5e0f3e0c0e2a9fbe685f12af8105d/source/stream.c#L290-L293 and the permission of my file, which looks like the following:

ll <some-readonly-file>
-r--r--r-- 1 root root 11236 Jul  2 01:56 <some-readonly-file>

Expected Behavior

I expect the file to be able to upload as if I use the default transfer client, even though I have only read access to the files.

Current Behavior

Error AWS_ERROR_NO_PERMISSION is encountered.

Reproduction Steps

  • This reproduction steps assume a default AWS profile with suitable S3 access has been configured.
touch readonly-empty-file
chmod 0444 readonly-empty-file

aws configure set default.s3.preferred_transfer_client crt
aws s3 cp readonly-empty-file s3://<some writable bucket>

Possible Solution

Open the file to be uploaded with rb access instead of r+b access.

Additional Information/Context

I hope that I do not open the issue in the wrong repository.

aws-crt-cpp version used

Unknown

Compiler and version used

Unknown

Operating System and version

Ubuntu 22.04.2 LTS

Build fails with reference to AWS_MQTT5_COTABT_MANUAL

Describe the bug

Build fails with

In file included from /root/gateway/libraries/third_party/aws-crt-cpp/include/aws/crt/Api.h:9,
                 from /root/gateway/libraries/third_party/aws-crt-cpp/source/ImdsClient.cpp:8:
/root/gateway/libraries/third_party/aws-crt-cpp/include/aws/crt/mqtt/Mqtt5Client.h:56:26: error: ‘AWS_MQTT5_COTABT_MANUAL’ was not declared in this scope; did you mean ‘AWS_MQTT5_COTABT_LRU’?
   56 |                 Manual = AWS_MQTT5_COTABT_MANUAL,
      |                          ^~~~~~~~~~~~~~~~~~~~~~~
      |                          AWS_MQTT5_COTABT_LRU

Expected Behavior

Build not failing

Current Behavior

Build fails

Reproduction Steps

Build including Mqtt5Client

Possible Solution

I am guessing AWS_MQTT5_COTABT_MANUAL should be AWS_MQTT5_COTABT_USER

Additional Information/Context

No response

aws-crt-cpp version used

0.26.4

Compiler and version used

gcc version 13.2.0 (Ubuntu 13.2.0-4ubuntu3)

Operating System and version

Ubuntu 23.10

Support for OS versions prior to Windows 8.1

I see there is a runtime check for Windows 8.1 in aws-crt-cpp to decide whether ALPN is available or not. I decided to try out how the SDK works out in my Windows 7 environment and noticed the following issues:

  • When building on Windows 10 with Visual Studio 2017 I had to add the following setting in CMakeLists.txt, otherwise the call to CertCreateCertificateChainEngine() would fail:

    add_definitions(-DWINVER=0x0601 -D_WIN32_WINNT=0x0601)
    

    Is this the correct approach? It would be nice to be able to avoid modifying the CMakeLists.txt file.

  • When trying to use the resulting binary, the connection to port 8883 of the AWS IoT server was disconnected during the TLS handshake. I checked the following 36 hosts resolving to the IoT DNS name and found some of them not supporting TLS 1.0:

    TLS 1 host
    no 3.248.37.235:8883
    yes 3.248.143.162:8883
    no 18.202.47.175:8883
    no 34.246.240.147:8883
    no 34.247.250.12:8883
    yes 34.248.23.176:8883
    no 34.248.215.173:8883
    no 34.248.219.217:8883
    yes 34.249.33.19:8883
    no 34.249.201.54:8883
    yes 34.251.110.255:8883
    yes 34.251.148.129:8883
    yes 34.253.253.202:8883
    no 34.254.167.26:8883
    no 52.17.220.6:8883
    yes 52.18.188.24:8883
    no 52.19.57.158:8883
    yes 52.19.161.237:8883
    no 52.31.90.196:8883
    yes 52.49.241.208:8883
    no 52.49.244.154:8883
    yes 52.51.157.5:8883
    no 52.208.29.67:8883
    yes 52.208.146.89:8883
    no 52.213.243.33:8883
    yes 52.214.225.79:8883
    no 52.215.86.27:8883
    yes 52.215.100.50:8883
    yes 54.72.137.20:8883
    yes 54.76.92.180:8883
    no 54.154.36.110:8883
    no 54.194.73.139:8883
    yes 54.194.108.62:8883
    no 63.32.148.190:8883
    yes 108.128.37.111:8883
    yes 108.129.67.76:8883

    Should all servers support TLS 1.0?

aws-c-io tests fail to link for gcc-linaro-6.3.1 x86-64 arm-linux toolchain

Hi there guys.

We're using the MQTT client in our IOT product that runs on a specific hardware.
After I managed to build this project as a stand alone library on my development machine for x86-64 Intel CPU, I tried to compile this project using our IOT toolchain (gcc-linaro-6.3.1 x86_64_arm-linux)

s2n project fails to be compiled with the error message

[100%] Linking C executable aws-c-io-tests
/connector-build-deps/lib/libs2n.a(s2n_random.c.o): In function `s2n_rand_init':
s2n_random.c:(.text+0x1b8): undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
tests/CMakeFiles/aws-c-io-tests.dir/build.make:618: recipe for target 'tests/aws-c-io-tests' failed
make[5]: *** [tests/aws-c-io-tests] Error 1
CMakeFiles/Makefile2:1018: recipe for target 'tests/CMakeFiles/aws-c-io-tests.dir/all' failed
make[4]: *** [tests/CMakeFiles/aws-c-io-tests.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make[3]: *** [all] Error 2
CMakeFiles/AwsCIO.dir/build.make:111: recipe for target 'build/src/AwsCIO-stamp/AwsCIO-build' failed
make[2]: *** [build/src/AwsCIO-stamp/AwsCIO-build] Error 2
CMakeFiles/Makefile2:179: recipe for target 'CMakeFiles/AwsCIO.dir/all' failed
make[1]: *** [CMakeFiles/AwsCIO.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
CMake Error at CMakeLists.txt:80 (message):
  Failed to build aws crt libraries.


Link error

It seems like the aws-c-http library isnt getting the aws-c-compression dependency properly.
This linker error does not happen with commit 792a4b

/home/kevin/deviceagentcpp-install/lib/libaws-c-http.a(hpack.c.o): In function `aws_hpack_context_new':
hpack.c:(.text+0x111a): undefined reference to `aws_huffman_encoder_init'
hpack.c:(.text+0x112d): undefined reference to `aws_huffman_decoder_init'
/home/kevin/deviceagentcpp-install/lib/libaws-c-http.a(hpack.c.o): In function `aws_hpack_encode_string':
hpack.c:(.text+0x1638): undefined reference to `aws_huffman_encode'
collect2: error: ld returned 1 exit status
iot_layer/tests/CMakeFiles/test-iot_layer.dir/build.make:137: recipe for target 'iot_layer/tests/test-iot_layer' failed
make[2]: *** [iot_layer/tests/test-iot_layer] Error 1
CMakeFiles/Makefile2:1727: recipe for target 'iot_layer/tests/CMakeFiles/test-iot_layer.dir/all' failed
make[1]: *** [iot_layer/tests/CMakeFiles/test-iot_layer.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

Sample code failure

samples/mqtt_pub_sub fails to compile due to recent changes in the bootstrap class.

warning: implicit declaration of function ‘SSL_CTX_set_security_level' -DUSE_OPENSSL=OFF

Hello,
as i am not able build libraries with -DUSE_OPENSSL=ON. I am trying to do it with -DUSE_OPENSSL=OFF. This way your libraries can be built and linked into application. The issue is that i have two more modules which are using openssl. So i have conflicts in linking time. So i built paho.mqtt.c module with crypto libraries from aws and i am getting this warning which i think shouldn't appear (it is not appearing with my custom openssl build).

steps

AWS_IOT_SDK_FOLDERNAME="aws-iot-device-sdk-cpp"
AWS_IOT_SDK_URL="https://github.com/aws/aws-iot-device-sdk-cpp-v2.git"
AWS_IOT_SDK_VER="1.18.6"

MQTT_C_FOLDERNAME="paho_mqtt_c"
MQTT_C_URL="[email protected]:eclipse/paho.mqtt.c.git"
MQTT_C_VER="1.3.10"

git clone --recursive ${MQTT_C_URL} --branch v${MQTT_C_VER} ${MQTT_C_FOLDERNAME} &&
    cd ${MQTT_C_FOLDERNAME} &&
    cmake -Bbuild -H. -DPAHO_ENABLE_TESTING=OFF -DPAHO_BUILD_STATIC=ON -DPAHO_WITH_SSL=ON -DPAHO_BUILD_SHARED=OFF \
        -DPAHO_HIGH_PERFORMANCE=ON \
        -DOPENSSL_ROOT_DIR="$(pwd)/../dependencies/${AWS_IOT_SDK_FOLDERNAME}" \
        -DCMAKE_INSTALL_PREFIX="$(pwd)/../dependencies/pahomqttc" &&
    cmake --build build/ --target install

output:

 85%] Building C object src/CMakeFiles/common_ssl_obj_static.dir/WebSocket.c.o
[ 87%] Building C object src/CMakeFiles/common_ssl_obj_static.dir/Proxy.c.o
[ 87%] Built target common_ssl_obj_static
[ 89%] Building C object src/CMakeFiles/paho-mqtt3cs-static.dir/MQTTClient.c.o
[ 91%] Building C object src/CMakeFiles/paho-mqtt3cs-static.dir/SSLSocket.c.o
/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/paho_mqtt_c/src/SSLSocket.c: In function ‘SSLSocket_createContext’:
/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/paho_mqtt_c/src/SSLSocket.c:597:9: warning: implicit declaration of function ‘SSL_CTX_set_security_level’; did you mean ‘SSL_CTX_set_verify_depth’? [-Wimplicit-function-declaration]
  597 |         SSL_CTX_set_security_level(net->ctx, 1);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |         SSL_CTX_set_verify_depth
[ 92%] Linking C static library libpaho-mqtt3cs.a
[ 92%] Built target paho-mqtt3cs-static
[ 94%] Building C object src/CMakeFiles/paho-mqtt3as-static.dir/MQTTAsync.c.o
[ 96%] Building C object src/CMakeFiles/paho-mqtt3as-static.dir/MQTTAsyncUtils.c.o
[ 98%] Building C object src/CMakeFiles/paho-mqtt3as-static.dir/SSLSocket.c.o
/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/paho_mqtt_c/src/SSLSocket.c: In function ‘SSLSocket_createContext’:
/home/acrios_cervennka/kseftiky/aeler/atlas-pilot-fw/Firmware/containers/aws-iotsdk/paho_mqtt_c/src/SSLSocket.c:597:9: warning: implicit declaration of function ‘SSL_CTX_set_security_level’; did you mean ‘SSL_CTX_set_verify_depth’? [-Wimplicit-function-declaration]
  597 |         SSL_CTX_set_security_level(net->ctx, 1);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |         SSL_CTX_set_verify_depth
[100%] Linking C static library libpaho-mqtt3as.a
[100%] Built target paho-mqtt3as-static

TLS negotiation error using

Using Ubuntu16 (x64) and raspberry pi:
connection_manager branch fails tls negotiation with greengrass core on a cisco RV110W router.

Compilation failure in Windows 10 using Visual studio 2019

Hello,

I compiled the Medical Imaging API thus have to compile the CRT repository as part of the AWS C++ Sdk.

I've hit a few compilation errors because of wrong Windows header includes:

#pki_utils.h:

#ifdef _WIN32
/* It's ok to include external headers because this is a PRIVATE header file
 * (it is usually a crime to include windows.h from header file) */
#    include <Windows.h>
#    include <schannel.h>

_WIN32 is defined as WIN32_
Resharper also suggested to include schannel.h. Without it, it doesn't compile.

#iocp_event_loop.c
Order of includes:

#include <Windows.h>
#include <schannel.h>

Flags I'm using:

  • BUILD_DEPS
  • BUILD_TESTING
  • CMAKE_CONFIGURATION_TYPES = Debug;Release;MinSizeRel;RelWithDebInfo
  • ENABLE_NET_TESTS
  • MQTT_WITH_WEBSOCKETS
  • USE_CPU_EXTENSIONS
  • USE_IO_COMLETION_PORTS
  • USE_OPENSSL
  • WINDOWS_KERNAL_LIB = Kernal32

Thank you.

Build Issue with M1 MacOS and GCC

Describe the bug

I am trying to compile on my MacOS 14.3.1 with GCC 13.2.0 but ran into errors.

Expected Behavior

build successfully

Current Behavior

I get the Issue:

/ghRepos/aws-crt-cpp/crt/aws-c-cal/source/darwin/commoncrypto_aes.c: In function 's_cc_crypto_gcm_finalize':
/ghRepos/aws-crt-cpp/crt/aws-c-cal/source/darwin/commoncrypto_aes.c:371:9: warning: implicit declaration of function '__builtin_available'; did you mean '__builtin_scalbl'? [-Wimplicit-function-declaration]
  371 |     if (__builtin_available(macOS 10.13, iOS 11.0, *)) {
      |         ^~~~~~~~~~~~~~~~~~~
      |         __builtin_scalbl
/ghRepos/aws-crt-cpp/crt/aws-c-cal/source/darwin/commoncrypto_aes.c:371:29: error: 'macOS' undeclared (first use in this function)
  371 |     if (__builtin_available(macOS 10.13, iOS 11.0, *)) {
      |                             ^~~~~
/ghRepos/aws-crt-cpp/crt/aws-c-cal/source/darwin/commoncrypto_aes.c:371:29: note: each undeclared identifier is reported only once for each function it appears in
/ghRepos/aws-crt-cpp/crt/aws-c-cal/source/darwin/commoncrypto_aes.c:371:34: error: expected ')' before numeric constant
  371 |     if (__builtin_available(macOS 10.13, iOS 11.0, *)) {
      |                            ~     ^~~~~~
      |                                  )

Reproduction Steps

steps performed:

git clone --recursive https://github.com/awslabs/aws-crt-cpp.git 
mkdir build && cd build
cmake -DCMAKE_C_COMPILER=/opt/homebrew/bin/gcc-13 -DCMAKE_CXX_COMPILER=/opt/homebrew/bin/g++-13 -DCMAKE_OSX_ARCHITECTURES=arm64 ..
make

Possible Solution

No response

Additional Information/Context

I was able to build with apple clang 15.0.0, but I need to work with gcc

aws-crt-cpp version used

latest: 4d9e36d

Compiler and version used

gcc version 13.2.0 (Homebrew GCC 13.2.0)

Operating System and version

macOS Sonoma 14.3.1

Can't build this repo as a standalone lib

I'm trying to build this library as a dependency for a project that uses the MQTT client.

neither the instructions in aws-iot-device-sdk-cpp-v2 or the instructions here work for me..
the "script" I use to build this project is:

mkdir build								;
git clone [email protected]:awslabs/aws-crt-cpp.git			;
cd build								;
cmake -DBUILD_DEPS=ON -DCMAKE_INSTALL_PREFIX="." ../aws-crt-cpp		;
make									;

output attached. CMake complains about missing packages.
I'm lost on this, any help?

make_output.txt

random segfault in s3 crt cpp client when uploading

I encounter some random segfault within s3 crt cpp client when uploading files (in parallel, to the same bucket), this seems related to name resolving but I'm not sure.
about 10% of my large batches encounter that.
not (until now) at the begining of the process but after a lot of files have been uploaded (maybe 20 to 100)

what could help apart the stacktrace shown below ?

thanks in advance.

(gdb) where
#0  0x00007f2a8984c6bc in aws_hash_string (item=0x6865765f6c766600)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/hash_table.c:965
#1  0x00007f2a8984c05f in s_hash_for (key=0x6865765f6c766600, state=0x55a8c5b833a0)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/hash_table.c:48
#2  aws_hash_table_find (map=map@entry=0x55a8c5b83118, key=0x6865765f6c766600, p_elem=p_elem@entry=0x7f2a85622388)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/hash_table.c:402
#3  0x00007f2a897932c2 in default_get_host_address_count (host_resolver=<optimized out>, host_name=<optimized out>, flags=1)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-io/source/host_resolver.c:1516
#4  0x00007f2a89b89be0 in aws_s3_client_update_meta_requests_threaded (client=client@entry=0x55a8c5c9af20)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-s3/source/s3_client.c:1217
#5  0x00007f2a89b8a584 in s_s3_client_process_work_default (client=0x55a8c5c9af20)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-s3/source/s3_client.c:1073
#6  0x00007f2a89850932 in s_run_all (scheduler=scheduler@entry=0x55a8c5b7ea30, current_time=1990735528099,
    status=status@entry=AWS_TASK_STATUS_RUN_READY)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/task_scheduler.c:249
#7  0x00007f2a89850e47 in aws_task_scheduler_run_all (scheduler=scheduler@entry=0x55a8c5b7ea30, current_time=<optimized out>)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/task_scheduler.c:188
#8  0x00007f2a89797a74 in s_main_loop (args=<optimized out>)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-io/source/linux/epoll_event_loop.c:635
#9  0x00007f2a8984ff57 in thread_fn (arg=0x55a8c5b7aa20)
    at /builds/anonimized/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-common/source/posix/thread.c:137
#10 0x00007f2a889deea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#11 0x00007f2a88ceddef in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

JsonObject Boolean issue

Hello,

I noticed today that set/get functionality of Aws::Crt::JsonObject does not work as expected.

This happens e.g. with following example when I try to replace existing value with new. Value does not change.

bool Configuration::setBoolValue(const Aws::Crt::String key, bool value)
{
    *m_configJson = m_configJson->WithBool(key, value);
    return m_configJson->View().GetBool(key));
}

With Intergers, Strings etc this works fine.

aws-crt-cpp-mqtt-pub-sub v0.5.5 hangs

Hi. How I compile it:

m -rf aws-crt-cpp
git clone --branch v0.5.5 https://github.com/awslabs/aws-crt-cpp.git

B="$PWD/B"
rm -rf "$B"

rm -rf aws-crt-cpp-build
mkdir aws-crt-cpp-build
cd aws-crt-cpp-build

cmake -DCMAKE_INSTALL_PREFIX="$B" -DCMAKE_PREFIX_PATH="$B" -DBUILD_SHARED_LIBS=ON -DBUILD_DEPS=ON ../aws-crt-cpp
cmake --build . --target install

Run:

./aws-crt-cpp-mqtt-pub-sub --endpoint "xxx.amazonaws.com" --cert "xxx" --key "xxx" --topic "xxx" --ca_file "xxx" --client_id "xxx"

Stdout:

Connecting...
Connection completed with return code 0
Connection interrupted with error aws-c-common: AWS_ERROR_SUCCESS, Success.
Connection resumed
Connection interrupted with error aws-c-common: AWS_ERROR_SUCCESS, Success.
Connection resumed
Connection interrupted with error aws-c-common: AWS_ERROR_SUCCESS, Success.
Connection resumed
...

It should ask to enter a text, but never does that. Because it has hanged in the subscription code here:

        // HANGS
        //
        conditionVariable.wait(uniqueLock);

        // NEVER EXECUTED
        while (true)
        {
            String input;
            fprintf(
                stdout,
                "Enter the message you want to publish to topic %s and press enter. Enter 'exit' to exit this "

FYI The Python sample (from here: https://github.com/aws/aws-iot-device-sdk-python) works fine with the same data/certs/keys/client-id/topic etc.

0.19.5: error: unknown type name 'aws_mqtt_connection_operation_statistics'

0.19.5 fails to build on FreeBSD:

Seems related to "Mqtt311 operation statistics support". Also, same error with 0.19.6

[ 91% 33/34] /usr/bin/c++ -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_C
RT_CPP_EXPORTS -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_MQTT_WITH_WEBSOC
KETS -DAWS_S3_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_USE_KQUEUE -DCJSON_HIDE_SYMBOLS -Daws_crt_cpp_EXPORTS -I/wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/inc
lude -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing   -DNDEBUG -fPIC -DS2N_STACKTRACE -DS2N_CPUID_
AVAILABLE -fPIC -DS2N_FALL_THROUGH_SUPPORTED -DS2N___RESTRICT__SUPPORTED -DS2N_MADVISE_SUPPORTED -DS2N_MINHERIT_SUPPORTED -DS2N_LIBCRYPTO_SUPPORTS_EVP_MD5_SHA1_HASH -DS2N_LIBCRYPTO_SUPPORTS_E
VP_RC4 -DS2N_LIBCRYPTO_SUPPORTS_EVP_MD_CTX_SET_PKEY_CTX -std=gnu++11 -MD -MT CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o -MF CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.
o.d -o CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o -c /wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/source/mqtt/MqttClient.cpp
FAILED: CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o
/usr/bin/c++ -DAWS_AUTH_USE_IMPORT_EXPORT -DAWS_CAL_USE_IMPORT_EXPORT -DAWS_CHECKSUMS_USE_IMPORT_EXPORT -DAWS_COMMON_USE_IMPORT_EXPORT -DAWS_COMPRESSION_USE_IMPORT_EXPORT -DAWS_CRT_CPP_EXPORT
S -DAWS_CRT_CPP_USE_IMPORT_EXPORT -DAWS_EVENT_STREAM_USE_IMPORT_EXPORT -DAWS_HTTP_USE_IMPORT_EXPORT -DAWS_IO_USE_IMPORT_EXPORT -DAWS_MQTT_USE_IMPORT_EXPORT -DAWS_MQTT_WITH_WEBSOCKETS -DAWS_S3
_USE_IMPORT_EXPORT -DAWS_SDKUTILS_USE_IMPORT_EXPORT -DAWS_USE_KQUEUE -DCJSON_HIDE_SYMBOLS -Daws_crt_cpp_EXPORTS -I/wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/include -isystem
 /usr/local/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing   -DNDEBUG -fPIC -DS2N_STACKTRACE -DS2N_CPUID_AVAILABLE -fP
IC -DS2N_FALL_THROUGH_SUPPORTED -DS2N___RESTRICT__SUPPORTED -DS2N_MADVISE_SUPPORTED -DS2N_MINHERIT_SUPPORTED -DS2N_LIBCRYPTO_SUPPORTS_EVP_MD5_SHA1_HASH -DS2N_LIBCRYPTO_SUPPORTS_EVP_RC4 -DS2N_LIBCRYPTO_SUPPORTS_EVP_MD_CTX_SET_PKEY_CTX -std=gnu++11 -MD -MT CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o -MF CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o.d -o CMakeFiles/aws-crt-cpp.dir/source/mqtt/MqttClient.cpp.o -c /wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/source/mqtt/MqttClient.cpp
/wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/source/mqtt/MqttClient.cpp:703:17: error: unknown type name 'aws_mqtt_connection_operation_statistics'; did you mean 'aws_mqtt5_client_operation_statistics'?
                aws_mqtt_connection_operation_statistics m_operationStatisticsNative = {0, 0, 0, 0};
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                aws_mqtt5_client_operation_statistics
/usr/local/include/aws/mqtt/v5/mqtt5_client.h:445:8: note: 'aws_mqtt5_client_operation_statistics' declared here
struct aws_mqtt5_client_operation_statistics {
       ^
/wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/source/mqtt/MqttClient.cpp:706:21: error: use of undeclared identifier 'aws_mqtt_client_connection_get_stats'; did you mean 'aws_mqtt_client_connection_connect'?
                    aws_mqtt_client_connection_get_stats(m_underlyingConnection, &m_operationStatisticsNative);
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                    aws_mqtt_client_connection_connect
/usr/local/include/aws/mqtt/client.h:401:5: note: 'aws_mqtt_client_connection_connect' declared here
int aws_mqtt_client_connection_connect(
    ^
/wrkdirs/usr/ports/devel/aws-crt-cpp/work/aws-crt-cpp-0.19.5/source/mqtt/MqttClient.cpp:706:82: error: cannot initialize a parameter of type 'const struct aws_mqtt_connection_options *' with
an rvalue of type 'aws_mqtt5_client_operation_statistics *'
                    aws_mqtt_client_connection_get_stats(m_underlyingConnection, &m_operationStatisticsNative);
                                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/aws/mqtt/client.h:403:47: note: passing argument to parameter 'connection_options' here
    const struct aws_mqtt_connection_options *connection_options);
                                              ^
3 errors generated.

Thanks

Access violation in TlsConnectionOptions member functions

Our code looks like this:

tls_ctx_opt = Io::TlsContextOptions::InitClientWithMtls(
	ByteCursorFromCString(cert),
	ByteCursorFromCString(key));
Io::TlsContext tls_ctx{tls_ctx_opt, Io::TlsMode::CLIENT};
Io::TlsConnectionOptions tls_opt = tls_ctx.NewConnectionOptions();
auto cur = ByteCursorFromCString(conn_opt.HostName.c_str());
tls_opt.SetServerName(cur);

If the tls_ctx constructor fails, the call tls_opt.SetServerName() causes an access violation for accessing the uninitialized member Io::TlsConnectionOptions::m_allocator.

What appears to happen is that when we call tls_ctx.NewConnectionOptions(), the function isValid() will be false, so we get an empty TlsConnectionOptions() object:

TlsConnectionOptions TlsContext::NewConnectionOptions() const noexcept
{
  if (!isValid())
  {
    AWS_LOGF_ERROR(
      AWS_LS_IO_TLS, "Trying to call TlsContext::NewConnectionOptions from an invalid TlsContext.");
    return TlsConnectionOptions();
  }
  return TlsConnectionOptions(m_ctx.get(), m_ctx->alloc);
}

This allocates the tls_opt into state where its m_isInit=false, m_lastError=AWS_ERROR_SUCCESS and m_allocator=<uninitialized random value>. When we then call tls_opt.SetServerName(), it gives the invalid m_allocator pointer to aws_tls_connection_options_set_server_name, which dereferences it.

Submodules not pinned to version. Mismatch between aws-c-io and vcpkg

On Feb 4th there was a dependency introduced in the aws-c-io. Anything after tag/v.0.7.3 (Jan 29th) it pulls that dependency from Feb 4th and since vcpkg core is not in sync cmake breaks with

C:\projects-2020\deps\include\aws\io\io.h(100): error C3861: 'AWS_ERROR_ENUM_BEGIN_RANGE': identifier not found

where error.h file comes from -IC:\projects-2020\vcpkg\installed\x64-windows\include

Please confirm.

AWS_ERROR_MQTT_UNEXPECTED_HANGUP in sample mqtt/basic_pub_sub

Hi,
I got the same error as described here: #80

Connection failed with error libaws-c-mqtt: AWS_ERROR_MQTT_UNEXPECTED_HANGUP, The connection was closed unexpectedly.

I learned that I have to quote the paths to the cert/key. And I reference the CA in form of a Amazon_Root_CA_1.pem. Also I learned form the above-mentioned thread that I have to add iot:* to the policy.

What other preconditions are necessary to get it working.

Thanks for your help.

Crash after aws-crt-cpp uninitialized

A crash happens in version v0.10.7 when aws-crt-cpp is uninitialized and then the DLL using the SDK is unloaded. This does not happen in v0.10.2.

The issue can be reproduced by modifying the basic_pub_sub sample to be a DLL with the following code change:

diff --git a/samples/mqtt/basic_pub_sub/main.cpp b/samples/mqtt/basic_pub_sub/main.cpp
index 09ecd27..eb3337c 100644
--- a/samples/mqtt/basic_pub_sub/main.cpp
+++ b/samples/mqtt/basic_pub_sub/main.cpp
@@ -78,6 +78,7 @@ char *s_getCmdOption(char **begin, char **end, const String &option)
     return 0;
 }
 
+extern "C" __declspec(dllexport)
 int main(int argc, char *argv[])
 {
 

and changing CMakeLists.txt to build the file as a DLL:

add_library(basicpubsub SHARED main.cpp)

Then use a simple runner to load and run the sample code:

#include <stdio.h>
#include <windows.h>
typedef int (WINAPI *fn_main_t)(int argc, char *argv[]);

int main(int argc, char *argv[])
{
    HMODULE dll = LoadLibraryW(L"basicpubsub.dll");
    fn_main_t fn_main = (fn_main_t)GetProcAddress(dll, "main");
    int ret = fn_main(argc, argv);
    BOOL status = FreeLibrary(dll);
    Sleep(60*1000);
    return ret;
}

When running this using version v0.10.7 of the SDK, and typing "exit" to the input prompt, the following crash happens as soon as the DLL is unloaded and the main thread is in the Sleep() call, when the SDK tries to execute code from the unloaded DLL:

(43d8.9358): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
<Unloaded_basicpubsub.dll>+0x17845e:
00007ffd`ed59845e ??              ???

After loading the DLL back to memory the call stack can be viewed:

0:004> lmvm basicpubsub.dll
start             end                 module name

Unloaded modules:
00007ffd`ed420000 00007ffd`ed71c000   basicpubsub.dll
    Timestamp: Thu Oct  8 18:36:53 2020 (5F7F3215)
    Checksum:  00000000
    ImageSize:  002FC000
0:004> .reload basicpubsub.dll=00007ffd`ed420000,002FC000
0:004> kP
 # Child-SP          RetAddr           Call Site
00 000000e0`52dfe920 00007ffd`ed6dd8a0 basicpubsub!aws_thread_launch(
			struct aws_thread * thread = 0x000000e0`52dfe998, 
			<function> * func = 0x00007ffd`ed5351a0, 
			void * arg = 0x0000029a`94529dc0, 
			struct aws_thread_options * options = 0x000000e0`52dfe9d8)+0xce [crt\aws-c-common\source\windows\thread.c @ 107]
01 000000e0`52dfe928 00000000`00000001 basicpubsub!default_allocator
02 000000e0`52dfe930 00000000`00000020 0x1
03 000000e0`52dfe938 cccccccc`cccccccc 0x20
04 000000e0`52dfe940 cccccccc`00000000 0xcccccccc`cccccccc
05 000000e0`52dfe948 000000e0`52dfe9b0 0xcccccccc`00000000
06 000000e0`52dfe950 00000000`00000000 0x000000e0`52dfe9b0

The stack isn't very sensible, but the top frame shows that a thread is executing in aws_thread_launch().

It seems that at the end of basicpubsub!main the destructors didn't clean up all the SDK resources.

This is hurting us badly, since version v0.10.2 has other problems (too long Windows paths, TLS timeout issues, ...) and can't be used either.

Support custom keychain on macOS

In aws/aws-iot-device-sdk-cpp-v2#163 I suggested adding support for using a custom keychain instead of the default login keychain for use-cases like daemons using the AWS IoT SDK. It was suggested that I'd instead change the default keychain for the account to the private keychain, or just a PKCS#12 file which appeared to not use a keychain.

As commented in the ticket, now when doing the daemon integration, we wound that changing the default keychain of the daemon account appeared to cause some other issues, and using PKCS#12 appears to also rely on the default keychain. To avoid dealing with the default keychain, I'd wish to add support for optionally using a custom private keychain for the IoT TLS connections only.

This issue covers the changes for aws-crt-cpp to support the changes in awslabs/aws-c-io#340

UUID::ToString() always returns an empty string

I was wondering why my strings in one of the aws samples were empty and looked into UUID::ToString(). It seems like the data is written do the buffer of the returned string, but the string is never informed that its length is not 0 anymore:

        String UUID::ToString() const
        {
            String uuidStr;
            uuidStr.reserve(AWS_UUID_STR_LEN);

            auto outBuf = ByteBufFromEmptyArray(reinterpret_cast<const uint8_t *>(uuidStr.data()), uuidStr.capacity());
            aws_uuid_to_str(&m_uuid, &outBuf);
            return uuidStr;
        }

Segmentation Fault Using Library

I'm attempting to build a dynamic library that depends on the aws-crt-cpp library. It compiles fine but when I try to test it, it either crashes or I get random corrupted data back when I try to access any variables from aws-crt-cpp.

Specifically, I'm trying to create an instance of the Aws::Iot::WebsocketConfig struct. I'm not sure if its an issue with aws-crt-cpp, but I can't seem to resolve it. It seems like an issue with the memory alignment or allocation.

Has anyone had any issues with using dynamic libraries, or is there anything special I need to do when building my dynamic library? Any help would be appreciated.

#if when #ifdef should be used - inconsistent with AWS SDK C++

The include headers contain a mix of #if _MSC_VER and #ifdef _MSC_VER. When compiled with strict warning-as-error settings, this produces the error '_MSC_VER' is not defined, evaluates to 0. The problem cannot be solved by defining _MSC_VER=0 in the build system, because this would cause the #ifdef statements to produce false-positives. The #ifs should be replaced with #ifdefs. This occurs with at least one other symbol in addition to _MSC_VER.

Warning: 'slot' variable may be used uninitialized

aws-common-runtime/s2n/utils/s2n_map.c: In function 's2n_map_embiggen':

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c:125:21: error: 'slot' may be used uninitialized in this function [-Werror=maybe-uninitialized]

 while(map->table[slot].key.size) {

                 ^

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c:121:14: note: 'slot' was declared here

 uint32_t slot;

          ^~~~

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c: In function 's2n_map_add':

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c:125:21: error: 'slot' may be used uninitialized in this function [-Werror=maybe-uninitialized]

 while(map->table[slot].key.size) {

                 ^

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c: In function 's2n_map_put':

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c:157:21: error: 'slot' may be used uninitialized in this function [-Werror=maybe-uninitialized]

 while(map->table[slot].key.size) {

                 ^

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c: In function 's2n_map_lookup':

aws-crt-cpp/aws-common-runtime/s2n/utils/s2n_map.c:207:16: error: 'slot' may be used uninitialized in this function [-Werror=maybe-uninitialized]

         if (slot == initial_slot) {

Git version history and tag problem

I mentioned this in awslabs/aws-c-mqtt#135 (comment), but thought to create an issue to the repository which is affected.

It seems to me that existing tags v0.8.4 and v0.8.5 have been changed to point to different commits recently. My previous clone shows:

* commit 393433e9fcfe48ab09c02a23577724f96dcdee5f (HEAD, tag: v0.8.5)
| Author: Bret Ambrose <[email protected]>
| Date:   Mon Jun 15 11:31:40 2020 -0700
* commit 421a69a81fb73b4c9917ffb3d095c2c65e9e973d (tag: v0.8.4)
| Author: Justin Boswell <[email protected]>
| Date:   Mon Jun 8 15:02:10 2020 -0700

My current clone shows:

* commit 107f24676a9f99bc5c5e61abe61b69044117c8fe (tag: v0.8.4)
| Author: GitHub Actions <[email protected]>
| Date:   Tue Jun 23 01:55:40 2020 +0000
|
|     Updated version to v0.8.4
|
* commit 8c45c499e10eb2ea49fa56698c156e9bb9583f4c (origin/update_dep)
| Author: Dengke Tang <[email protected]>
| Date:   Mon Jun 22 18:55:15 2020 -0700
* commit fb751d0ea75401423c07acd4737fd0b7a7fc2bdc (origin/master, origin/HEAD, master)
| Author: Justin Boswell <[email protected]>
| Date:   Thu Jun 18 09:09:15 2020 -0700
| * commit 31fd668ef26af0ae0ee1ee84c01f7466b023e67f (tag: v0.8.5)
| | Author: GitHub Actions <[email protected]>
| | Date:   Wed Jun 17 23:34:12 2020 +0000
| |
| |     Updated version to v0.8.5
| |
| * commit c2dfd35805f25e28ec933a29e3a2bf77a22e5afa
|/  Author: Justin Boswell <[email protected]>
|   Date:   Wed Jun 17 23:33:37 2020 +0000
* commit 9fda1e3309a4de37c2f052c5b1d531537c70b8a3

The new tags don't look reasonable and existing tags shouldn't be reassigned in any case. Is this some automation running amok?

[ClientBootstrap]: segfault at program exit

Summary

This issue happens regardless of whether ClientBootstrap::EnableBlockingShutdown() has been called, and it is caused by returning from ~ClientBootstrap() while asynchronous shutdown is still in progress.

In production, we are repeatedly getting the same backtrace:

FATAL: Received signal 11 (Segmentation fault)
Backtrace (most recent call first)
#6	 <?> at 0x564a57ceca28 in Aws::Utils::Logging::s_aws_logger_redirect_log()
#5	 <?> at 0x564a57d4bb2b in aws_default_dns_resolve
#4	 <?> at 0x564a57d4dfed in resolver_thread_fn
#3	 <?> at 0x564a57d5a117 in thread_fn
#2	 <?> at 0x564a5753f32c in thread_metrics_start_routine
#1	 <?> at 0x7f04dcfb86db in start_thread

Details

The orderly shutdown of the client bootstrap is invoked by calling CleanupCrt(), followed by shutting down CRT logging (see
aws/aws-sdk-cpp#1995).

// aws core/source/Globals.cpp
    void CleanupCrt()
    {
        Aws::SetDefaultClientBootstrap(nullptr);
        // ...
    }

The s_client_bootstrap_destroy_impl invokes the "blocking shutdown" callback right after the aws_event_loop_group_release and aws_host_resolver_release calls:

static void s_client_bootstrap_destroy_impl(struct aws_client_bootstrap *bootstrap) {
    AWS_ASSERT(bootstrap);
    AWS_LOGF_DEBUG(AWS_LS_IO_CHANNEL_BOOTSTRAP, "id=%p: destroying", (void *)bootstrap);
    aws_client_bootstrap_shutdown_complete_fn *on_shutdown_complete = bootstrap->on_shutdown_complete;
    void *user_data = bootstrap->user_data;

    aws_event_loop_group_release(bootstrap->event_loop_group);
    aws_host_resolver_release(bootstrap->host_resolver);

    if (on_shutdown_complete) {
        on_shutdown_complete(user_data);
    }
}

However, both the event_loop_group and the host_resolver perform asynchronous shutdown, i.e. their "destroy" functions merely trigger the shutdown of threads that are still running; they do not await thread exit.

The problem now arises that the event_loop_group and clearly the host_resolver still issue calls to the CRT logging system.
We can not afford to shut down CRT logging (via Aws::Utils::Logging::ShutdownCRTLogging()) while these threads are still exiting,
otherwise the above segfault will result.

Possible fix

Populate the shutdown callback functions of both the contained event_loop_group and host_resolver.
Have them set a flag/counter to indicate that the shutdown callbacks (at thread exit) have been invoked.

For good measure, also have the counter set in the shutdown of the aws_client_bootstrap.

In the destructor of ClientBootstrap, await, via condition variable, all 3 shutdowns.

Ignore the "blocking shutdown" boolean variable (due to the above described problems).
Use a timeout for the condition variable, in case an exiting thread failed to invoke the final callback.
This is to avoid a hanging program at end (the current solution uses a std::promise, which would block indefinitely if set_value was not called for an unexpected reason).

Presigned URL with MQTT client

Hi,

I'm trying to create a mqtt connection using a presigned url return from a subscription query to AWS appsync.

When I try to connect I get the following error:
aws-c-io: AWS_IO_DNS_INVALID_NAME, Host name was invalid for dns resolution.

The presigned url looks like this:
wss://iotendpoint-ats.iot.us-west-2.amazonaws.com/mqtt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AAAAAAAAAAAA%2F20200203%2Fus-west-2%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20200203T190240Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature>&X-Amz-Security-Token=<token>

Simplified code:

Aws::Crt::Mqtt::MqttClient *mqttClient = new Aws::Crt::Mqtt::MqttClient(bootstrap);

std::shared_ptr<Aws::Crt::Mqtt::MqttConnection> mqttConnection = mqttClient->NewConnection(url.c_str(), 443, Aws::Crt::Io::SocketOptions(), true);

First, I'm wondering if I should be able to use a presigned url with the MqttClient in the Crt library? If so, is there something here I'm doing wrong?

cross compile for arm embedded linux

Hi,
my environment:

  • Ubuntu 16.04 VM
  • emb. Linux cross compile toolchain with openssl + libcrypto libs available
  • emb. Linux provides script which sets all paths to target compile toolchain, so one can use cmake / make and gcc,... for arm target is automatically set

when in the directory to build, I do:

cmake -DBUILD_DEPS=ON ../.

but get

Target "LibCrypto::Crypto" not found.

when deps are going to be built

tried to alter CMakeLists.txt and add link_directories with paths to the emb.linux libs, no success

Any ideas highly appreciated ...

source/auth/Credentials.cpp fpermissive errors

When compiling from the latest release on gcc 7.3, compiling Credentials.cpp produces the following error:

source/auth/Credentials.cpp:293:93: error: invalid conversion from ‘const aws_tls_connection_options*’ to ‘aws_tls_connection_options*’ [-fpermissive]
                     proxy_options.tls_options = proxy_config.TlsOptions->GetUnderlyingHandle();

Hopefully, it's a simple fix! :)

mqtt sample: AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE

Although I successfully managed to connect and publish to aws iot on desktop ubuntu, I fail doing the same on my embedded linux target (same cert / key files):

tried:

aws-crt-cpp-mqtt-pub-sub --endpoint SECRET-ats.iot.eu-west-1.amazonaws.com --cert "/data/certificate.pem.crt" --key "/data/private.pem.key" --topic "testsdkv2"

get:

Connection failed with error aws-c-io: AWS_IO_TLS_ERROR_NEGOTIATION_FAILURE, TLS (SSL) negotiation failed

Is there an issue with finding the libcrypt or something?

I'm able to ping the iot endpoint, policy allows iot actions...

Wrong version number

The version number which is included in the MQTT connection information is wrong.

It seems the version number has been correct in 7 releases out of 36 and incorrect in the rest 29 releases:

release tag version number
v0.3.1 v0.3.0
v0.3.2 v0.3.0
v0.4.0 v0.3.0
v0.4.1 v0.3.0
v0.4.2 v0.3.0
v0.4.4 v0.3.0
v0.4.5 v0.3.0
v0.4.6 v0.3.0
v0.5.0 v0.3.0
v0.5.1 v0.3.0
v0.5.2 v0.3.0
v0.5.3 v0.3.0
v0.5.4 v0.3.0
v0.5.5 v0.3.0
v0.6.0 v0.5.6
v0.6.1 v0.5.6
v0.6.3 v0.6.2
v0.6.4 v0.6.2
v0.6.6 v0.6.5
v0.6.7 v0.6.5
v0.6.9 v0.6.8
v0.6.10 v0.6.8
v0.7.2 v0.7.1
v0.7.4 v0.7.3
v0.7.5 v0.7.3
v0.7.6 v0.7.3
v0.8.1 v0.7.3
v0.8.3 v0.8.2
v0.8.4 v0.8.2

Maybe the version number should be generated automatically?

Crash at aws_http_library_init() when no network is available

I tracked down the cause for the crash in issue awslabs/aws-c-http#320.

When our application tries to initialize and notices there is no network connection, it uninitializes and tries again later. When it does that, it crashes.

This is because a recently made seemingly unmotivated change in commit BYO_CRYPTO:

          ApiHandle::ApiHandle() noexcept : m_logger(), m_shutdownBehavior(ApiHandleShutdownBehavior::Blocking)
          {
-             s_initApi(DefaultAllocator());
+             s_initApi(g_allocator);
          }

This made ApiHandle() to initialize its initial state from its initial state, basically doing g_allocator = g_allocator. Since g_allocator is set to nullptr in ~ApiHandle() this meant that suddenly ApiHandle() doesn't use the default allocator after the 1st use. Instead it makes the SDK crash.

g++-11 compile error (Ubuntu 20.04 x86_64)

Opened originally on aws-cpp-sdk #1635.
When compiling the cpp sdk I get these errors on the crt-cpp:

/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.c:92:44: error: argument 1 of type 'int16_t[256]' {aka 'short int[256]'} with mismatched bound [-Werror=array-parameter=]
   92 | void PQCLEAN_KYBER51290S_CLEAN_ntt(int16_t poly[256]) {
      |                                    ~~~~~~~~^~~~~~~~~
In file included from /root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.c:1:
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.h:9:45: note: previously declared as 'int16_t *' {aka 'short int *'}
    9 | void PQCLEAN_KYBER51290S_CLEAN_ntt(int16_t *poly);
      |                                    ~~~~~~~~~^~~~
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.c:116:47: error: argument 1 of type 'int16_t[256]' {aka 'short int[256]'} with mismatched bound [-Werror=array-parameter=]
  116 | void PQCLEAN_KYBER51290S_CLEAN_invntt(int16_t poly[256]) {
      |                                       ~~~~~~~~^~~~~~~~~
In file included from /root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.c:1:
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/s2n/pq-crypto/kyber_90s_r2/ntt.h:10:48: note: previously declared as 'int16_t *' {aka 'short int *'}
   10 | void PQCLEAN_KYBER51290S_CLEAN_invntt(int16_t *poly);
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-cal/source/unix/openssl_platform_init.c:386:43: error: the comparison will always evaluate as 'false' for the address of 's_locking_fn' will never be NULL [-Werror=address]
  386 |         if (CRYPTO_get_locking_callback() == s_locking_fn) {
      |                                           ^~
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-cal/source/unix/openssl_platform_init.c: In function 'aws_cal_platform_clean_up':
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-cal/source/unix/openssl_platform_init.c:402:39: error: the comparison will always evaluate as 'false' for the address of 's_locking_fn' will never be NULL [-Werror=address]
  402 |     if (CRYPTO_get_locking_callback() == s_locking_fn) {
      |                                       ^~
/root/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-cal/source/unix/openssl_platform_init.c:411:34: error: the comparison will always evaluate as 'false' for the address of 's_id_fn' will never be NULL [-Werror=address]
  411 |     if (CRYPTO_get_id_callback() == s_id_fn) {
      |  

Memory leak at JsonObject::NewArray(Vector<JsonObject> &&objectsToMove)

Describe the bug

There is a memory leak at JsonObject::NewArray(Vector &&objectsToMove)

In the loop, objects are supposed to be moved. Nevertheless, there's a aws_json_value_duplicate call made for the object and then the old m_value is just assigned to nullptr (without calling aws_json_value_destroy). Seems like a copy paste error from the method above in the file?

Expected Behavior

No object/memory leaks when C++ move semantics is used.

Current Behavior

Consider a code:

JsonObject object;
...
object.WithArray("whatever", std::move(object_vector));

This calls JsonObject::NewArray(Vector &&objectsToMove), creating unnecessary copies of all the objects but forgetting to free the old ones.

Reproduction Steps

If you need, I can provide full example, but this seems quite clear even without it?

Possible Solution

It seems obvious that aws_json_value_duplicate should not be called with rvalue objectsToMove.

Additional Information/Context

No response

aws-crt-cpp version used

main branch

Compiler and version used

gcc

Operating System and version

ubuntu linux

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.