Hi, I have a simple node application that uses <code class="notransl

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.github

As you have seen from <a class="issue-link js-issue-link" data-error-text="Failed to l

Cannot run pkg on windows,about awslabs/aws-crt-nodejs

Comments (19)

bretambrose commented on September 24, 2024

To the best of my knowledge, the CRT doesn't require any external DLL beyond the msvc runtime that it was built with. That being said, the aws-crt js layer does expect the native library to be in dist/bin/<os>_<arch>/aws-crt-nodejs.node, which is not the path I see in your output.

from aws-crt-nodejs.

Goldsmith42 commented on September 24, 2024

I have the same problem. To try and debug it, I've set up a very simple project that consists of a single index.js file and has pkg and aws-crt as its only direct dependencies. This is what src\index.js contains:

console.log('Version', process.version);
const sdk = require('aws-crt');
console.log('Loaded sdk', !!sdk);

Running node src\index.js produces the following output:

Version v12.22.2
Loaded sdk true

While trying the generated executable does this:

Version v12.22.2
pkg/prelude/bootstrap.js:1697
      throw error;
      ^

Error: A dynamic link library (DLL) initialization routine failed.
\\?\C:\Users\zlatko\Code\test-aws-crt-pack\node_modules\aws-crt\dist\bin\win32-x64\aws-crt-nodejs.node
    at process.dlopen (pkg/prelude/bootstrap.js:2108:30)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1057:18)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1676:31)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\native\binding.js:55:19)
    at Module._compile (pkg/prelude/bootstrap.js:1751:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)

If I copy aws-crt-nodejs.node to the same folder as the executable, the change is noted but it's the same error:

Version v12.22.2
pkg/prelude/bootstrap.js:1697
      throw error;
      ^

Error: A dynamic link library (DLL) initialization routine failed.
\\?\C:\Users\zlatko\Code\test-aws-crt-pack\node_modules\aws-crt\dist\bin\win32-x64\aws-crt-nodejs.node
    at process.dlopen (pkg/prelude/bootstrap.js:2108:30)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1057:18)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1676:31)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\native\binding.js:55:19)
    at Module._compile (pkg/prelude/bootstrap.js:1751:22)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)

If I add the module as an asset in the pkg section of my package.json file:

"pkg": {
    "scripts": "src/**/*.js",
    "assets": ["node_modules\\aws-crt\\dist\\bin\\win32-x64\\aws-crt-nodejs.node"],
    "targets": [
      "node12-win-x64"
    ],
    "outputPath": "bin"
}

the error is slightly different:

C:\Users\zlatko\AppData\Local\Temp\20beb37a61fc8df24a4fc7b95a53d714c783bbef0f57dbfa90b6ccc76895f2ca\aws-crt-nodejs.node
    at tryImporting (pkg/prelude/bootstrap.js:2029:37)
    at tryImporting (pkg/prelude/bootstrap.js:2083:16)
    at process.dlopen (pkg/prelude/bootstrap.js:2106:7)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1057:18)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at Module.require (pkg/prelude/bootstrap.js:1676:31)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\native\binding.js:55:19)

If I remove the asset and move the .exe file to another location, I get this:

Error: AWS CRT binary not present in any of the following locations:                                                            C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\bin\native\aws-crt-nodejs                                       C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\bin\win32-x64\aws-crt-nodejs                                at Object.<anonymous> (C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\native\binding.js:60:11)                 at Module._compile (pkg/prelude/bootstrap.js:1751:22)                                                                   at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)                                               at Module.load (internal/modules/cjs/loader.js:863:32)                                                                  at Function.Module._load (internal/modules/cjs/loader.js:708:14)                                                        at Module.require (internal/modules/cjs/loader.js:887:19)                                                               at Module.require (pkg/prelude/bootstrap.js:1676:31)                                                                    at require (internal/modules/cjs/helpers.js:74:18)                                                                      at Object.<anonymous> (C:\snapshot\test-aws-crt-pack\node_modules\aws-crt\dist\native\crt.js:22:35)                     at Module._compile (pkg/prelude/bootstrap.js:1751:22)

(which is in line with what I expect)

Seeing how I've had no trouble with pkg and other binary modules (specifically, sqlite3 is fine as long as you provide the .node file), my initial assumption would be that this has something to do with aws-crt, but I don't know if there's anything that I can do to change how it tries to load itself.

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

@Goldsmith42 I will log a support ticket with AWS. Let's see what they say. This is the sample repo.
https://github.com/zhex900/aws-crt-win-pkg

Something to do with this vercel/pkg#305?

from aws-crt-nodejs.

bretambrose commented on September 24, 2024

I believe the native dll on Windows is built with MSVC 2015, so it requires the associated C runtime package to be installed (https://www.microsoft.com/en-us/download/details.aspx?id=48145)

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

Please see what I have installed.

By the way, I can build aws-crt in windows. So all the C/C++ runtime packages should be already there.

from aws-crt-nodejs.

TingDaoK commented on September 24, 2024

https://github.com/vercel/pkg#native-addons this maybe related. They will treat .node file as asset.
vercel/pkg#837 (comment). They have the same issue.

I guess you need to config pkg to build from source. https://github.com/vercel/pkg#config or dig more into how they resolve the native addon issue from pkg side.

from aws-crt-nodejs.

TingDaoK commented on September 24, 2024

I also tested on mac and linux. Seems like it only works for Linux.
For mac, we will have a

dyld: lazy symbol binding failed: Symbol not found: _napi_module_register
  Referenced from: /Users/dengket/project/test_folder/node_modules/aws-crt/dist/bin/darwin-x64/aws-crt-nodejs.node
  Expected in: flat namespace

So, something is wrong when they link the aws-crt-nodejs.node.
On windows, seems like they treat the .node as DLL and link into it.
My best guess is there is a linking issue.

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

Thank you for your efforts. Any suggestions?

from aws-crt-nodejs.

TingDaoK commented on September 24, 2024

As you have seen from vercel/pkg#305 (comment). I think it's very likely some linker issue between pkg and cmakejs. I tried to dig into the root cause from their package, but it's somehow out of my scope and not finding something related. I don't really have good suggestion here...

from aws-crt-nodejs.

whtlnv commented on September 24, 2024

I'm facing the same issue only on Windows. Compiling an empty electron app that includes aws-crt will work perfectly on OSX but in Windows it will crash on launch with:

App threw an error during load
Error: A dynamic link library (DLL) initialization routine failed.
\\?\C:\Users\Administrator\active\electron-quick-start\aws-crt-nodejs\dist\bin\win32-ia32\aws-crt-nodejs.node
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1846)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1138:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:1846)
    at Module.load (internal/modules/cjs/loader.js:935:32)
    at Module._load (internal/modules/cjs/loader.js:776:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12913)
    at Module.require (internal/modules/cjs/loader.js:959:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (C:\Users\Administrator\active\electron-quick-start\aws-crt-nodejs\dist\native\binding.js:55:19)
    at Module._compile (internal/modules/cjs/loader.js:1078:30)

cmake-js is configured in the (electron) application:

  "cmake-js": {
    "runtime": "electron",
    "runtimeVersion": "13.1.9",
    "arch": "ia32"
  }

And cmake does use this configuration during install:

info TOOL Using Visual Studio 15 2017 generator.
info CMD CONFIGURE
info RUN cmake "C:\Users\Administrator\active\electron-quick-start\aws-crt-nodejs" --no-warn-unused-cli 
-G"Visual Studio 15 2017" -DCMAKE_JS_VERSION="6.1.0" 
-DCMAKE_BUILD_TYPE="Release" 
-DCMAKE_RUNTIME_OUTPUT_DIRECTORY="C:\Users\Administrator\active\electron-quick-start\aws-crt-nodejs\build" 
-DCMAKE_JS_INC="C:\Users\Administrator\.cmake-js\electron-ia32\v13.1.9\include\node" 
-DCMAKE_JS_SRC="C:/Users/Administrator/active/electron-quick-start/aws-crt-nodejs/node_modules/cmake-js/lib/cpp/win_delay_load_hook.cc" 
-DNODE_RUNTIME="electron" 
-DNODE_RUNTIMEVERSION="13.1.9" 
-DNODE_ARCH="ia32" 
-DCMAKE_JS_LIB="C:\Users\Administrator\.cmake-js\electron-ia32\v13.1.9\node.lib" 
-DCMAKE_EXPORT_COMPILE_COMMANDS="true" 
-DCMAKE_JS_PLATFORM="win32" 
-DCMAKE_JS_ARCH="ia32" 
-DBUILD_TESTING="OFF" 
-DCMAKE_INSTALL_PREFIX="crt/install" 
-DCMAKE_PREFIX_PATH="crt/install" 
-DCMAKE_SHARED_LINKER_FLAGS="/DELAYLOAD:NODE.EXE /SAFESEH:NO"
Not searching for unused variables given on the command line.
-- The C compiler identification is MSVC 19.16.27045.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x86/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Using submodule dependencies
-- CMake 3.21.1

Out of desperation, I tried removing modules from the source project such as http and auth. It compiled correctly but got the same error once the electron app starts.

Any ideas where to go from here?

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

@whtlnv It is good to see another person having the same problem.

Unfortunately, I am at a dead end. Maybe it will work if the native modules are build using node-gyp not cmake. But I no idea how to do that.

For now, my only option is to deploy on windows with node run time. So I need to deploy with my code and run npm start

I don't really like it. But I don't know any other way to get my code to run in windows.

from aws-crt-nodejs.

whtlnv commented on September 24, 2024

@zhex900 The alternatives we are exploring is to use websockets instead, so that we can only import the browser libraries that do not require the native module. This requires to authenticate using identity pools instead of X.509.

This will be a patch, as it is highly desirable for us to use the native module, since we already have 3 products deployed using it.

What our team might be trying in the future:

Debug the windows app during the crash to get more information.
Compile the module using gyp (we also have limited experience using it, and aren't super excited to dig deeper into this lib)
Compile the component modules (io, auth, mqtt, common, etc) on their own and analyze the behavior
WASM? this seems like a long shot, but it'll be a super cool if possible.

Would you mind going into a little more detail about your solution? Are you launching an electron app and a node app?

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

Hi, I am building a digital sign app using electron. For aws iot to work on windows I need to run a node app for aws iot and electron. Electron communicate with the node app through ipc sockets. The node app share the aws-iot logic. So in Linux and Mac electron directly uses aws-iot code. But Windows electron depends on the ipc sockets for aws-iot communication. So the windows node app is a small wrapper. Personally I do like it. But this is the only way I can use aws iot with electron on windows. My current problem is deployment. Since I cannot package the node app into exe. I have to deploy thr code and run npm start. I use pm2 too. I thought about using aws iot in the browser. But using identity pool is a bad solution for me.

…

On Sun, 15 Aug 2021 at 1:47 pm, whtlnv ***@***.***> wrote: @zhex900 <https://github.com/zhex900> The alternatives we are exploring is to use websockets instead, so that we can only import the browser libraries that do not require the native module. This requires to authenticate using identity pools instead of X.509. This will be a patch, as it is highly desirable for us to use the native module, since we already have 3 products deployed using it. What our team might be trying in the future: - Debug the windows app during the crash to get more information. - Compile the module using gyp (we also have limited experience using it, and aren't super excited to dig deeper into this lib) - Compile the component modules (io, auth, mqtt, common, etc) on their own and analyze the behavior - WASM? this seems like a long shot, but it'll be a super cool if possible. Would you mind going into a little more detail about your solution? Are you launching an electron app *and* a node app? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#227 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABJBVFP7JDW23JKNNPDBOALT442ENANCNFSM47FT2ZLQ> .

from aws-crt-nodejs.

whtlnv commented on September 24, 2024

@zhex900 we managed to connect to IoT Core without aws-crt, same module for OSX and Windows, running on Electron, using MQTT.js authenticating the devices with X.509 certificates generated from IoT Core. I hope it can work for you too.

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

That is good idea! How did you provision the device cert? Did you use MQTT.js too?
I am using fleet provisioning.

How did you register the IoT Thing and provision the device cert?

Looks like it can be done. I will try it.

    async publishCreateKeysAndCertificate(
        request: model.CreateKeysAndCertificateRequest,
        qos: mqtt.QoS)
        : Promise<mqtt.MqttRequest> {

        let topic: string = "$aws/certificates/create/json";
        return this.connection.publish(topic, JSON.stringify(request), qos);
    }

from aws-crt-nodejs.

whtlnv commented on September 24, 2024

In our case, things are provisioned on demand using IotCore.createKeysAndCertificate(), which returns:

certificatePem
keyPair.PublicKey
keyPair.PrivateKey

All of this happens in a lambda, and then sent to the device. The thing stores this securely along with its id, then on MQTT.js on the device, we do something like:

import mqtt, { IClientOptions } from 'mqtt';

const AWS_ENPOINT = 'foo.iot.us-east-1.amazonaws.com';
const AWS_PORT = 8883;
const THING = {
  thingId: '0001',
  pem: '-----BEGIN CERTIFICATE----- blah -----END CERTIFICATE-----',
  privateKey: '-----BEGIN RSA PRIVATE KEY----- bleh -----BEGIN RSA PRIVATE KEY-----',
  ca: '-----BEGIN CERTIFICATE----- blih -----END CERTIFICATE-----'
};

export function connect() {
  const options: IClientOptions = {
    clientId: THING.thingId,

    protocol: 'mqtt' as const,
    host: AWS_ENPOINT,
    port: AWS_PORT,

    key: THING.privateKey,
    cert: THING.pem,
    ca: THING.ca,

    keepalive: 10,
    // will: {},
  };

  const client = mqtt.connect(AWS_ENPOINT, options);
}

The only difference with aws-crt is that all messages will be emitted under the same message event, you'll have to filter by topic by hand.

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

Did you run this in side electron?

I can get mqtt.js working by itself. But it is placed inside electron I get this error

TypeError: nextTick is not a function
    at Object.generate [as writeToStream] (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:133432:5)
    at sendPacket (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:134636:27)
    at MqttClient._setupStream (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:134908:3)
    at new MqttClient (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:134817:8)
    at Object.connect (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:136427:16)
    at Object.connectAsync (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:71305:25)
    at /Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:201238:59
    at step (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:201206:23)
    at Object.next (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:201187:53)
    at fulfilled (/Users/jake/Code/pam-kiosk-app/packages/kiosk-app/.webpack/main/index.js:201178:58)

from aws-crt-nodejs.

whtlnv commented on September 24, 2024

Here's a proof of concept in an empty Electron project

from aws-crt-nodejs.

zhex900 commented on September 24, 2024

Thank you. I am switching to mqtt.js

from aws-crt-nodejs.

Cannot run pkg on windows about aws-crt-nodejs HOT 19 CLOSED

Comments (19)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent