Giter VIP home page Giter VIP logo

node-nanomsg's Introduction

nanomsg for node

Build Status Build status

install:

npm install nanomsg

This is the default way of installing node-nanomsg. Behind the scenes, nanomsg library will be downloaded, built and statically linked, providing the simplest and easiest way to start working.

Another option, if you have a more complex environment/project, and you may already have nanomsg installed on your system, or you want to target a specific version when using it from node:

npm install nanomsg --use_system_libnanomsg=true

This has the following prerequisites:

  • nanomsg must be installed. See https://github.com/nanomsg/nanomsg
  • pkg-config must be installed
  • only tested on linux, so other platforms are explicitly excluded at the moment (any help testing other OS is welcome)

check it out:

var nano = require('nanomsg');

var pub = nano.socket('pub');
var sub = nano.socket('sub');

var addr = 'tcp://127.0.0.1:7789'
pub.bind(addr);
sub.connect(addr);

sub.on('data', function (buf) {
  console.log(String(buf));
  pub.close();
  sub.close();
});

setTimeout(function () {
  pub.send("Hello from nanomsg!");
}, 100);

API

nano.socket(type, [options,])

Starts a new socket. The nanomsg socket can bind or connect to multiple heterogeneous endpoints as well as shutdown any of these established links.

options

  • 'raw' (Boolean, default: false): determines the domain of the socket. AF_SP, the default, creates a standard full-blown SP socket. AF_SP_RAW family sockets operate over internal network protocols and interfaces. Raw sockets omit the end-to-end functionality found in AF_SP sockets and thus can be used to implement intermediary devices in SP topologies, see nanomsg docs or consult your man page entry socket(2) for more info.
//ex. starting raw sockets
nano.socket('bus', { raw: true } );

socket.shutdown(address)

(Function, param: String): Removes an endpoint established by calls to bind() or connect(). The nanomsg library will try to deliver any outstanding outbound messages to the endpoint.

socket.shutdown('tcp://127.0.0.1:5555');

socket.bind(address)

(Function, param: String): Adds a local endpoint to the socket. The endpoint can be then used by other applications to connect.

bind() (or connect()) may be called multiple times on the same socket thus allowing the socket to communicate with multiple heterogeneous endpoints.

socket.bind('tcp://::1:5555');

recommend checking your machine's ifconfig first before using IPv6. ipconfig on windows.

socket.connect(address)

(Function, param: String): Adds a remote endpoint to the socket. The nanomsg library would then try to connect to the specified remote endpoint.

connect() (as well as bind()) may be called multiple times on the same socket thus allowing the socket to communicate with multiple heterogeneous endpoints.

socket.connect('tcp://127.0.0.1:5555');

When connecting over remote TCP allow 100ms or more depending on round trip time for the operation to complete.

socket.close()

(Function, param: Function): Closes the socket. Any buffered inbound messages that were not yet received by the application will be discarded. The nanomsg library will try to deliver any outstanding outbound messages.

nanomsg transports and the endpoint address string

(String)

Endpoint address strings consist of two parts as follows: transport://address. The transport specifies the underlying transport protocol to use. The meaning of the address part is specific to the underlying transport protocol.

  • TCP transport mechanism: 'tcp://127.0.0.1:65000' When binding a TCP socket, address of the form tcp://interface:port should be used. Port is the TCP port number to use. Interface is either: IPv4 or IPv6 address of a local network interface, or DNS name of the remote box.
  • WebSocket transport mechanism: 'ws://127.0.0.1:64999' Implemented on top of TCP, a WebSocket address of the form ws://interface:port should be used. Port is the TCP port number to use. Interface is either: IPv4 or IPv6 address of a local network interface, or DNS name of the remote box. When calling either bind() or connect(), omitting the port defaults to the RFC 6455 default port 80 for HTTP. See examples/ws for basic implementation over the browser.
  • in-process transport mechanism: 'inproc://bar' The inproc transport allows messages between threads or modules inside a process. In-process address is an arbitrary case-sensitive string preceded by inproc:// protocol specifier. All in-process addresses are visible from any module within the process. They are not visible from outside of the process. The overall buffer size for an inproc connection is determined by rcvbuf socket option on the receiving end of the connection. sndbuf is ignored. In addition to the buffer, one message of arbitrary size will fit into the buffer. That way, even messages larger than the buffer can be transfered via inproc connection.
  • inter-process transport mechanism: 'ipc:///tmp/foo.ipc' The ipc transport allows for sending messages between processes within a single box. The nanomsg implementation uses native IPC mechanism provided by the local operating system and the IPC addresses are thus OS-specific. On POSIX-compliant systems, UNIX domain sockets are used and IPC addresses are file references. Note that both relative (ipc://test.ipc) and absolute (ipc:///tmp/test.ipc) paths may be used. Also note that access rights on the IPC files must be set in such a way that the appropriate applications can actually use them. On Windows, named pipes are used for IPC. The Windows IPC address is an arbitrary case-insensitive string containing any character except for backslash: internally, address ipc://test means that named pipe \\.\pipe\test will be used.

sending and receiving: writable and readable

socket.send(msg)

(Function, param: String or Buffer): send a message.

socket.send('hello from nanømsg!');

send(msg) is automatically invoked during Writable consumption of some other Readable stream. In that case a Writable's pipe() method can be used to transmit across readable data sources. See example for more detail. The flow of data distributes to endpoint(s) determined by the particular socket type.

var source = require('fs').createReadStream('filename.ext');

source.pipe(socket); //sends each chunk as a msg to socket's particular endpoint

socket.on('data', callback)

(Function, param order: String, Function): The Readable stream's on() function is an event listener that emits 'data' events. To receive messages, pass the string 'data' followed a callback containing a single data parameter.

// the default inbound message is a node buffer
// setEncoding sets the message type, use utf8 to receive strings instead.
socket.setEncoding('utf8');

socket.on('data', function (msg) {
  console.log(msg); //'hello from nanømsg!'
});

The readable stream's data event is automatically invoked when piped to a Writable or Transform consumer stream. See example for more detail. Here msgprocessor is a transform you could pipe to a writable or the next transform:

var through = require('through');

var msgprocessor = through(function(msg){
  var str = msg; //'hello from nanømsg'
  this.queue(str + ' and cheers!');
});

socket.pipe(msgprocessor); //msg transformed to: 'hello from nanømsg and cheers!'

subscription api

socket.chan(Array)

(Function, param: Array of Strings, default: ['']): Allows for sub sockets to filter messages based on a prefix. Not applicable to non sub sockets.

By default, all sub sockets are subscribed to the '' channel. Once you opt in to filtering on a channel, you are unsubscribed from ''.

socket.rmchan(String)

(Function, param: String): Allows for sub sockets to remove channel filters. Not applicable to non sub sockets. This function is variadic; you can pass multiple strings and all will be unfiltered.

If you unsubscribe from the default channel, '', without subscribing to any new channels, your sub socket will stop receiving messages.

sockopt api

socket.tcpnodelay(boolean)

(Function, param: Boolean, default: false): When set, disables Nagle’s algorithm. It also disables delaying of TCP acknowledgments. Using this option improves latency at the expense of throughput.

Pass no parameter for current tcp nodelay setting.

//default
console.log(socket.tcpnodelay()); //tcp nodelay: off

socket.tcpnodelay(true); //disabling Nagle's algorithm

console.log(socket.tcpnodelay()); //tcp nodelay: on

socket.maxttl(hops)

(Function, param: Number, default: 8): Sets the maximum number of "hops" a message can go through before it is dropped. Each time the message is received (for example via the nn_device(3) function) counts as a single hop. This provides a form of protection against inadvertent loops.

Pass no parameter for the socket's maxttl hop count.

socket.maxttl(4);
console.log(socket.maxttl()); // 4

socket.linger(duration)

(Function, param: Number, default: 1000): Specifies how long the socket should try to send pending outbound messages after socket.close() or socket.shutdown() is called, in milliseconds.

Pass no parameter for the linger duration.

Note: linger was deprecated upstream. As of node-nanomsge > v3.3.0 (libnanomsg 1.1.0), this value no longer has any meaning, and will always read back 0.

socket.linger(5000);
console.log(socket.linger()); //5000

socket.sndbuf(size)

(Function, param: Number, size in bytes, default: 128kB): Size of the send buffer, in bytes. To prevent blocking for messages larger than the buffer, exactly one message may be buffered in addition to the data in the send buffer.

Pass no parameter for the socket's send buffer size.

socket.sndbuf(131072);
console.log(socket.sndbuf()); // 131072

socket.rcvbuf(size)

(Function, param: Number, size in bytes, default: 128kB): Size of the receive buffer, in bytes. To prevent blocking for messages larger than the buffer, exactly one message may be buffered in addition to the data in the receive buffer.

Pass no parameter for the socket's receive buffer size.

socket.rcvbuf(20480);
console.log(socket.rcvbuf()); // 20480

socket.sndtimeo(duration)

(Function, param: Number, default: -1): The timeout for send operation on the socket, in milliseconds.

Pass no parameter for the socket's send timeout.

socket.sndtimeo(200);
console.log(socket.sndtimeo()); // 200

socket.rcvtimeo(duration)

(Function, param: Number, default: -1): The timeout for recv operation on the socket, in milliseconds.

Pass no parameter for the socket's recv timeout.

socket.rcvtimeo(50);
console.log(socket.rcvtimeo()); // 50

socket.reconn(duration)

(Function, param: Number, default: 100): For connection-based transports such as TCP, this option specifies how long to wait, in milliseconds, when connection is broken before trying to re-establish it. Note that actual reconnect interval may be randomized to some extent to prevent severe reconnection storms.

Pass no parameter for the socket's reconnect interval.

socket.reconn(600);
console.log(socket.reconn()); // 600

socket.maxreconn(duration)

(Function, param: Number, default: 0): Only to be used in addition to socket.reconn(). maxreconn() specifies maximum reconnection interval. On each reconnect attempt, the previous interval is doubled until maxreconn is reached. Value of zero means that no exponential backoff is performed and reconnect interval is based only on reconn. If maxreconn is less than reconn, it is ignored.

Pass no parameter for the socket's maxreconn interval.

socket.maxreconn(60000);
console.log(socket.maxreconn()); // 60000

socket.sndprio(priority)

(Function, param: Number, default: 8): Sets outbound priority for endpoints subsequently added to the socket.

This option has no effect on socket types that send messages to all the peers. However, if the socket type sends each message to a single peer (or a limited set of peers), peers with high priority take precedence over peers with low priority.

Highest priority is 1, lowest is 16. Pass no parameter for the socket's current outbound priority.

socket.sndprio(2);
console.log(socket.sndprio()); // 2

socket.rcvprio(priority)

(Function, param: Number, default: 8): Sets inbound priority for endpoints subsequently added to the socket.

This option has no effect on socket types that are not able to receive messages.

When receiving a message, messages from peer with higher priority are received before messages from peer with lower priority.

Highest priority is 1, lowest is 16. Pass no parameter for the socket's current inbound priority.

socket.rcvprio(10);
console.log(socket.rcvprio()); // 10

socket.ipv6(boolean)

(Function, param: Boolean, default: false): Allows for the use of IPv6 addresses to bind or connect to.

By default, nanomsg only works with IPv4 addresses, and support for IPv6 addresses must explicitly be enabled.

If enabled, both IPv4 and IPv6 addresses can be used.

socket.ipv6(true);
console.log(socket.ipv6()); // true

socket.rcvmaxsize(size)

(Function, param: Number, size in bytes, default: 1024kB): Maximum message size that can be received, in bytes. Negative value means that the received size is limited only by available addressable memory.

Pass no parameter for the socket's maximum receive buffer size.

socket.rcvmaxsize(10000000);
console.log(socket.rcvmaxsize()); // 10000000

socket.wsopt(str)

(Function, param: String, Websocket msg frame format, default: 'binary'): This option may be set to type 'text' or 'binary'. This string value determines whether data msgs are sent as WebSocket text frames, or binary frames, per RFC 6455. Text frames should contain only valid UTF-8 text in their payload, or they will be rejected. Binary frames may contain any data. Not all WebSocket implementations support binary frames. The default is to send binary frames.

Pass no parameter for the socket's frame format.

socket.wsopt('text');
console.log(socket.wsopt()); // 'text'

If you are implementing nanomsg websockets in the browser, please carefully review the spec: https://raw.githubusercontent.com/nanomsg/nanomsg/master/rfc/sp-websocket-mapping-01.txt

socket.dontwait(boolean)

(Function, param: Boolean, default: true, except PUSH sockets): Sets the NN_DONTWAIT flag, specifying that the operation should be performed in non-blocking mode,

  • true for non-blocking mode
  • false for blocking mode

Pass no parameter for the socket's current mode.

socket.dontwait(false);
console.log(socket.dontwait()); // false

// or set when socket is started:
require('nanomsg').socket('pub', { dontwait: false });

test

$ git clone https://github.com/nickdesaulniers/node-nanomsg.git nano
$ cd nano && git submodule update --init

# now you can build the project and run the test suite:
$ make && make check

# or perhaps you'd prefer to use the npm commands instead:
$ npm i
$ npm t

# let's say you switch to another version of node/iojs, you might want to run:
$ make clean && make && make check

# for the super deluxe make clean, rebuild, and test suite:
$ make full

Note: you must git submodule update --init to initialize the nanomsg repository.

test - when node-nanomsg is being installed the optional way (dynamically linking to libnanomsg)

# you can build the project and run the test suite:
$ make use_system_libnanomsg && make check

# or perhaps you'd prefer to use the npm commands instead:
$ npm i --use_system_libnanomsg=true
$ npm t

# let's say you switch to another version of node/iojs, you might want to run:
$ make clean && make use_system_libnanomsg && make check

# for the super deluxe make clean, rebuild, and test suite:
$ make use_system_libnanomsg-full

performance

run benchmarks:

$ make perf

for more info how to do that and your own custom comparisons check out: running benchmarks

and if you want you can also run:

$ make bench

:)

contributing

Issues and pull requests welcome!

contributors

a0000778GitHub/a0000778-
Ant SkeltonGitHub/blowback-
Adam BiroGitHub/sirudog-
Benjamin ByholmGitHub/kkoopa-
Bent CardanGitHub/reqsharkTwitter/@rekshark
Deepak PrabhakaraGitHub/deepakprabhakaraTwitter/@deepakprab
Flynn JoffrayGitHub/nucleardreamerTwitter/@nucleardreamer
m-ohuchiGitHub/m-ohuchi-
Michele ComignanoGitHub/comick-
Nick DesaulniersGitHub/nickdesaulniersTwitter/@LostOracle
Tim Cameron RyanGitHub/tcrTwitter/@timcameronryan
Trygve LieGitHub/trygve-lieTwitter/@trygve_lie

formatting

C/C++

Please run clang-format -style=Mozilla -i <file> on all C/C++ code.

JS

WIP

license

MIT

memory leak hunting

npm i --asan=true

node-nanomsg's People

Contributors

a0000778 avatar alimousazy avatar bartel-c8 avatar blowback avatar comick avatar daraghking avatar deepakprabhakara avatar den1zk avatar dependabot[bot] avatar farwayer avatar kkoopa avatar nickdesaulniers avatar nucleardreamer avatar reqshark avatar tcr avatar trygve-lie avatar vvscode avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-nanomsg's Issues

npm install nanomsg build failure

I actually already have nanomsg built locally on my machine, is there a way to install the package and point it at my .dll?

Windows 8, 64-bit

C:\Users\karbarcca\node_modules\nanomsg>node "C:\Users\karbarcca\nodejs\node_mod
ules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild

Building the projects in this solution one at a time. To enable parallel build,
please add the "/m" switch.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.Cpp.InvalidPlatform
.Targets(23,7): error MSB8007: The Platform for project 'nanomsg.vcxproj' is in
valid.  Platform='x64'. You may be seeing this message because you are trying t
o build a project without a solution file, and have specified a non-default Pla
tform that doesn't exist for this project. [C:\Users\karbarcca\node_modules\nan
omsg\build\nanomsg.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\karbarcca\nodejs\node_module
s\npm\node_modules\node-gyp\lib\build.js:267:23)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:807:
12)
gyp ERR! System Windows_NT 6.2.9200
gyp ERR! command "node" "C:\\Users\\karbarcca\\nodejs\\node_modules\\npm\\node_m
odules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\karbarcca\node_modules\nanomsg
gyp ERR! node -v v0.10.28
gyp ERR! node-gyp -v v0.13.0
gyp ERR! not ok
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the nanomsg package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls nanomsg
npm ERR! There is likely additional logging output above.

npm ERR! System Windows_NT 6.2.9200
npm ERR! command "C:\\Users\\karbarcca\\nodejs\\\\node.exe" "C:\\Users\\karbarcc
a\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "nanomsg"
npm ERR! cwd C:\Users\karbarcca
npm ERR! node -v v0.10.28
npm ERR! npm -v 1.4.9
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\Users\karbarcca\npm-debug.log
npm ERR! not ok code 0

C:\Users\karbarcca>

Asynchronous REPREQ

I came across this thread that addresses this issue at the library level :
http://nanomsg.freelists.narkive.com/UpDQxh1T/asynchronous-repreq

which starts with the question :

Say I get a request that I can't fulfill until data becomes available a few
seconds later. I'd like the worker to be able to hold onto the request and
process others in the meantime. The worker then might get triggered via
another socket (maybe a bus type broadcast to all workers) that new data is
available and then the outstanding request can maybe be fulfilled.

And I guess I'm re-asking the same question : I was hoping to find that REPREQ would occur with http(req,res) kind of per-client asynchronous in a node setting.

If not (and I don't see where it's implemented currently), of the three suggested solutions is preferable? ::

a) Continually keeping a new socket('REP') waiting for an extra client('REQ'), while asynchronously handing off work as it arrives

b) Using different nanomsg services (e.g. a PULL socket for the server for incoming requests, with each being tagged with a client-provided ID, so that clients can SUB to a feed of all the replies and watch for their ID)

c) Building something at the sendmsg, recvmsg level (which may be too down-and-dirty to build for just the node implementation)

I'm thinking that (a) could be a 'fairly simple' addition to this module, or something built as a thin layer on to of it.

Any thoughts?

Support "raw" socket type in createSocket constructor

I'll create a PR for this shortly:

function createSocket(type, opts) {
    var domain = (opts || {}).raw ? nn.AF_SP_RAW : nn.AF_SP;
    switch(type) {
        case 'req': return new Socket(type, nn.Socket(domain, nn.NN_REQ), true, true);
        ...

Question: Any way to guarantee pub.send delivery?

I'd like to publish a message with a command line app and exit once I'm sure the message has been sent.

Although it appears pub.send is synchronous, it seems there are times when messages are lost due to latency issues. I've tried to increase the the linger value, but the only thing that seems to reliably send is if I sleep for a second before exiting.

It seems to me that although pub.send appear synchronous, it's really asynchronous. Is that the case?

buffer error: kMaxLength

running the test suite on osx and linux i get a buffer triggered assertion crap out on survey.js.

also wondering why tape is zero for zero when t.plan(4) :

ok survey.js ........................ 0/0
TAP version 13
Assertion failed: (length <= kMaxLength), function New, 
file ../src/node_buffer.cc, line 145.

How to catch nanomsg timeout ?

Hi !

Since many days, i try to catch nanomsg timeout.

I set rcvtimeo and sndtimeo at 100, and tyed to catch it with socket.on("error") and socket.on("timeout") but that never occurs.

Can I have some help please.

Thx.

Support dynamic linking of libnanomsg

It looks like the node-Nanomsg is pulling its own version of the C source for nanomsg, which his a pain as the nanomsg lib is updated.

My use of Nanomsg is to integrate processes written in different languages (C, C++, JS) and I would like to stick to one version of nanomsg across my system -- hence it would be better if the lib would use the c-nanomsg version installed on the local system and build against that rather than pull a specific version tag of c-nanomsg a version which is being outdated pretty quickly.

unable to install

make: *** No rule to make target `Release/obj.target/nanomsg/deps/nanomsg/src/core/ep.o', needed by `Release/nanomsg.a'.

IPv6 support

Hi,

I am trying to connect to "tcp://[fe80::3e15:c2ff:feb9:67c2]:7777" but getting an error message.
From what I googled, it seems that nanomsg is coming with IPv6 disabled...
Because this project is using a fork of nanomsg, is it possible to enable it ?

PUB/SUB doesn't work with multiple subscribers

Tried a minor modification of the pubsub example to use multiple subscribers, and I'm getting a random assortment of weird and undesirable outcomes, like no output at all, output from only one subscriber, or 'protocol not available' errors.

This is what I tried:

var nano = require('../');

var pub = nano.socket('pub');
var sub1 = nano.socket('sub');
var sub2 = nano.socket('sub');
var sub3 = nano.socket('sub');

var addr = 'tcp://127.0.0.1:7789'
pub.bind(addr);
sub1.connect(addr);
sub2.connect(addr);
sub3.connect(addr);

sub1.on('message', function (buf) {
    console.log("sub1 got: %s", buf.toString());
});

sub2.on('message', function (buf) {
    console.log("sub2 got: %s", buf.toString());
});

sub3.on('message', function (buf) {
    console.log("sub3 got: %s", buf.toString());
});

setTimeout(function () {
    console.log("PUBLISHING...");
    pub.send("Hello from nanomsg!");
}, 100);

I'd expect to see three responses, one from each of the subscribers?

[vanity api] Include a .survey method

Proposing this as an idea, though I haven't thought it through too much. A .survey method for surveyor sockets could collect values and return them at once. A .surveyStream method could create a stream that emits buffers of responses, then closes when the timeout occurs.

Example code:

Socket.prototype.survey = function (buf, callback) {
    var responses = [];
    function listener (buf) {
        responses.push(buf);
    }
    this.once('survey-timeout', function () {
        this.removeListener('message', listener);
        callback(responses);
    })
    this.on('message', listener)
    this.send(buf);
}

s.survey(new Buffer('question?'), function (answers) {
    console.log(answers);
});

zero-copy

zero-copy becomes increasingly useful as msg size grows.

for people not familiar with zero-copy or interested in more background info, here's a nice article explaining what it is and why it's important.

libnanomsg supports zero-copy msg transport for both send and recv operations. Ok so when and where do we want to be doing zero-copy in nanomsg? I think according to the lib's creator Martin Sustrik, probably where: large starts around 512kB, half a megabyte.

now since we need to enable zero-copy, I guess the only question is where and how in node-nanomsg. Right off the bat, I think a JS style sockopt with a Boolean would be a great way.

at first i was thinking about an opt available to send, doing something like:

socket.send(msg, *opt*)

but translating that to the EventEmitter's onmessage callback is awkward, since we're not doing explicit recv operations in node-nanomsg. so that's why I was thinking maybe:

socket.zerocopy(true)

Current master will not compile on iojs 2.x

iojs 2.0.0 was just released with quite a bit of large and breaking changes in it. Sadly it seems like master will not compile with this version of iojs:

  CC(target) Release/obj.target/nanomsg/deps/nanomsg/src/protocols/pubsub/xsub.o
  CC(target) Release/obj.target/nanomsg/deps/nanomsg/src/protocols/reqrep/rep.o
  CC(target) Release/obj.target/nanomsg/deps/nanomsg/src/protocols/reqrep/req.o
make: *** No rule to make target `Release/obj.target/nanomsg/deps/nanomsg/src/protocols/reqrep/task.o', needed by `Release/obj.target/nanomsg.a'.  Stop.
make: Leaving directory `/home/trygve/Dev/node-nanomsg/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:269:23)
gyp ERR! stack     at emitTwo (events.js:87:13)
gyp ERR! stack     at ChildProcess.emit (events.js:169:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1009:12)
gyp ERR! System Linux 3.13.0-51-generic
gyp ERR! command "/usr/local/bin/iojs" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/trygve/Dev/node-nanomsg
gyp ERR! node -v v2.0.0
gyp ERR! node-gyp -v v1.0.3
gyp ERR! not ok 

Events emitted?

Inspecting the code I spot those events that will be emitted:

  • survey-timeout
  • message
  • error
  • close
  1. Are there more?
  2. How can I be notified for connect event?
  3. What would be the road to go if I want to implement more events?

Bad file descriptor [9] (../deps/nanomsg/src/aio/poller_epoll.inc:107)

For some odd nanomsg crashes when used within a Docker container throwing an error Bad file descriptor [9] (../deps/nanomsg/src/aio/poller_epoll.inc:107). TCP ports are exposed correctly.

Any ideas as to why this might be happening? To see the crash, take any of the examples, dockerize and run.

REQ not sending ETIMEDOUT

According to @gdamore on nanomsg gitter channel:

REQ will return ETIMEDOUT on recv() if the timeout expires.

So far I've tried every configuration possible and req.on("timeout", function() { ... }) never fires.

streams2 api?

Hi,

Have you thought about a streams api for nanomsg?

I'd like to be able to implement nanomsg with some sort of backpressure support if my downstream node stuff, for example, is too busy to actually process PULL messages coming up from my nanomsg pipeline. The EventEmitter callback-style isn't really ideal since I assume it will just keep accepting messages as long as I'm still listening and has the potential to just queue up callbacks in memory, if, say, I'm inserting messages into the db on the backend and the db insert can't keep up.

I'm looking at implementing one myself if you haven't thought about it already, or if you have and have any pointers or thoughts, would love to hear them!

Best,
Jen

getting double message callback on BUS socket

Do not know if this is the problem of my code or not but I seems to be getting double messages every time I call a send on a BUS socket:

var nano = require('nanomsg'),
argv = require('yargs').argv;

var bus = nano.socket('bus');

var addr = 'ipc:///tmp/' + argv.node;
var p1 = 'ipc:///tmp/' + argv.p1;
var p2 = 'ipc:///tmp/' + argv.p2;

console.log('binding to', addr);
bus.bind(addr);

console.log('connecting to', p1);
bus.connect(p1);
//console.log('connecting to', p2);
//bus.connect(p2);

bus.on('message', function (buf) {
console.log(buf.toString());
});

var seq = 0;
setInterval(function () {
seq++;
console.log('sending...', seq);
bus.send(argv.node + " " + seq);
}, 5000);

////////////////////////
node nano-test.js --node n0 --p1 n1
node nano-test.js --node n1 --p1 n0

///////////////////////
n1 2
n1 2
sending... 1
n1 3
n1 3
sending... 2
n1 4
n1 4
sending... 3
n1 5
n1 5

Behavior of PUSH PULL

REQREP - when there are no consumers and the producer sent messages, if a consumer connects, it will receive each message slowly, per reconnecting configuration

PUSHPULL - when there are no consumers and the producer sent messages, if a consumer connects, it will receive all pending messages instantly

Is this behavior normal? Also, I see that if I leave the producer enabled for hours and a consumer connects, it will still get all those messages instantly. What's the limit?

Pub/Sub on a particular topic?

I'm having trouble subscribing/publishing to a particular socket. Whatever I do, my subscriber ends up receiving all messages.

Is this not implemented?

// subscriber.js
var nano = require('nanomsg');

var sub = nano.socket('sub');
var nn = nano._bindings;

var addr = 'tcp://127.0.0.1:7789'
sub.setsockopt(nn.NN_SUB, nn.NN_SUB_SUBSCRIBE, 'my.topic');
sub.connect(addr);

sub.on('message', function (buf) {
    console.log(buf.toString());
});

// publisher.js
var nano = require('nanomsg');
var pub = nano.socket('pub');
var addr = 'tcp://127.0.0.1:7789'
pub.bind(addr);

setInterval(function () {
    pub.send("my.topic This is a topic");
    pub.send("my.dog My dog's name is Spot");
}, 1000);

merge nanomsg.iojs

Looking to adopt some of the API changes from nanomsg.iojs and possibly custom event model. cc @reqshark see also reqshark/nanomsg.iojs#19

@reqshark you now have commit bits. For anything that changes existing APIs, let's use PRs and code reviews, but otherwise feel free to commit to master for smaller changes.

remove cpp macros

Let's get rid of macros. While they shorten the amount of code needed to type, they make me feel uncomfortable.

npm install nanomsg fails now

I installed nanomsg 0.0.0 several days ago on a machine and it worked. There have been recent commits to the package so I attempted to upgrade to 0.2.0, but it doesn't install via npm correctly on that machine or a fresh machine. I am on Ubuntu 12.04 64bit. It fails during the add-on layer shim portion of the make script. Here is the terminal log with error messages:

sudo npm install nanomsg
npm http GET https://registry.npmjs.org/nanomsg
npm http 200 https://registry.npmjs.org/nanomsg
npm http GET https://registry.npmjs.org/nanomsg/-/nanomsg-0.1.0.tgz
npm http 200 https://registry.npmjs.org/nanomsg/-/nanomsg-0.1.0.tgz
npm http GET https://registry.npmjs.org/bindings/1.1.1
npm http 200 https://registry.npmjs.org/bindings/1.1.1
npm http GET https://registry.npmjs.org/bindings/-/bindings-1.1.1.tgz
npm http 200 https://registry.npmjs.org/bindings/-/bindings-1.1.1.tgz

[email protected] install /media/truecrypt8/dev/working/bench/nanomsg2/node_modules/nanomsg/node_modules/addon-layer
node-gyp rebuild

make: Entering directory /media/truecrypt8/dev/working/bench/nanomsg2/node_modules/nanomsg/node_modules/addon-layer/build' CXX(target) Release/obj.target/addon-layer/src/shim.o ../src/shim.cc: In function ‘void shim::shim_module_preinit()’: ../src/shim.cc:1919:46: error: ‘RTLD_SELF’ was not declared in this scope make: *** [Release/obj.target/addon-layer/src/shim.o] Error 1 make: Leaving directory/media/truecrypt8/dev/working/bench/nanomsg2/node_modules/nanomsg/node_modules/addon-layer/build'
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:797:12)
gyp ERR! System Linux 3.8.0-37-generic
gyp ERR! command "node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /media/truecrypt8/dev/working/bench/nanomsg2/node_modules/nanomsg/node_modules/addon-layer
gyp ERR! node -v v0.10.26
gyp ERR! node-gyp -v v0.12.2
gyp ERR! not ok
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the addon-layer package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get their info via:
npm ERR! npm owner ls addon-layer
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 3.8.0-37-generic
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "nanomsg"
npm ERR! cwd /media/truecrypt8/dev/working/bench/nanomsg2
npm ERR! node -v v0.10.26
npm ERR! npm -v 1.4.3
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /media/truecrypt8/dev/working/bench/nanomsg2/npm-debug.log
npm ERR! not ok code 0

unique prototypes for unique socket types

this one cropped up during streams development in #53

just an idea that might help binding duplex stream pull sockets with built-in back-pressure.

I wonder if we can grab most of the general Socket class, like all the sockopt functions, and other properties and bring them over to a few type specific prototypes.

In order to keep prototype inheritance boilerplate a bit cleaner, I copied the io.js util.inherits() core module function and made it cooler than school: https://github.com/reqshark/utildot

What if we had a generalized Socket with all the shared prototype methods and just pulled from a few child constructors with type-specific support?

remove unnecessary requires

all of the tests seem to include should, but dont seem to use it at all (unless importing should has side effects, looks like yes, not sure we use it still).

1.0 release

With PR: #65, were ready for a major version release. We should add docs about the new pub sub channel support, and run npm version major.

update socket.shutdown

@nickdesaulniers, we should probably implement this method the way we say it's implemented.

socket.shutdown(address)

call shutdown(str) w/ an address string to detach the socket connected/bound to that address.

makes way more sense than trying to cope with obscure return values: libnanomsg's how integer returned by low level calls to nn_bind() or nn_connect().

I think we just need to tuck that weird implementation detail into a new object property on the JS socket and use the delete operator to clear it when shutdown() gets called. No biggie...

I'll PR that one after lunch tomorrow and we won't have to worry about updating the docs :)

The proposed tcp.js test does not test multiple binds on same endpoint

This is because the TCP transport signals SIGABRT when a multiple bind is attempted, which as far as I am aware cannot be usefully trapped in a node test harness*.

The inproc.js test does test for this failure condition, but this is possible because libnanomsg's INPROC transport returns an error, rather than aborting.

In other words, it is an inconsistency in libnanomsg transport implementions.

Since the test cannot successfully be run without dumping core, it has been elided.

[* I'm happy to be proven wrong, but as far as I know a SIGABRT handler will only be honoured if it does not return, i.e. with a longjmp() in C - I don't know how this can be realised in node such that the test harness regains control. ]

Fail to compile with io.js

This module currently fail to compile with io.js 1.0.x:

  CC(target) Release/obj.target/nanomsg/deps/nanomsg/src/utils/wire.o
  AR(target) Release/obj.target/nanomsg.a
  COPY Release/nanomsg.a
  CXX(target) Release/obj.target/node_nanomsg/src/node_nanomsg.o
In file included from ../src/node_pointer.h:6:0,
                 from ../src/node_nanomsg.cc:3:
../node_modules/nan/nan.h:481:19: error: ‘NanNew’ declared as an ‘inline’ variable
       v8::String::ExternalAsciiStringResource *resource) {
                   ^
../node_modules/nan/nan.h:481:19: error: ‘always_inline’ attribute ignored [-Werror=attributes]
../node_modules/nan/nan.h:481:19: error: ‘v8::Local<v8::String> NanNew’ redeclared as different kind of symbol
../node_modules/nan/nan.h:475:36: error: previous declaration of ‘v8::Local<v8::String> NanNew(v8::String::ExternalStringResource*)’
   NAN_INLINE v8::Local<v8::String> NanNew(
                                    ^
../node_modules/nan/nan.h:481:7: error: ‘ExternalAsciiStringResource’ is not a member of ‘v8::String’
       v8::String::ExternalAsciiStringResource *resource) {
       ^
../node_modules/nan/nan.h:481:48: error: ‘resource’ was not declared in this scope
       v8::String::ExternalAsciiStringResource *resource) {
                                                ^
../node_modules/nan/nan.h: In function ‘bool _NanGetExternalParts(v8::Handle<v8::Value>, const char**, size_t*)’:
../node_modules/nan/nan.h:2103:12: error: ‘class v8::String’ has no member named ‘IsExternalAscii’
   if (str->IsExternalAscii()) {
            ^
../node_modules/nan/nan.h:2104:11: error: ‘ExternalAsciiStringResource’ in ‘class v8::String’ does not name a type
     const v8::String::ExternalAsciiStringResource* ext;
           ^
../node_modules/nan/nan.h:2105:5: error: ‘ext’ was not declared in this scope
     ext = str->GetExternalAsciiStringResource();
     ^
../node_modules/nan/nan.h:2105:16: error: ‘class v8::String’ has no member named ‘GetExternalAsciiStringResource’
     ext = str->GetExternalAsciiStringResource();
                ^
cc1plus: all warnings being treated as errors
make: *** [Release/obj.target/node_nanomsg/src/node_nanomsg.o] Error 1
make: Leaving directory `/home/trygve/Dev/node-nanomsg/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack     at ChildProcess.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1038:12)
gyp ERR! System Linux 3.13.0-44-generic
gyp ERR! command "node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/trygve/Dev/node-nanomsg
gyp ERR! node -v v1.0.3
gyp ERR! node-gyp -v v1.0.2
gyp ERR! not ok 

npm ERR! Linux 3.13.0-44-generic
npm ERR! argv "/usr/local/bin/iojs" "/usr/local/bin/npm" "install"
npm ERR! node v1.0.3
npm ERR! npm  v2.2.0
npm ERR! code ELIFECYCLE
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! This is most likely a problem with the nanomsg package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls nanomsg
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/trygve/Dev/node-nanomsg/npm-debug.log

bad test case

The test send.js says this on Travis in Node 0.10:

/home/travis/build/nickdesaulniers/node-nanomsg/node_modules/tape/index.js:75
throw err
^
TypeError: Object hello has no method 'equals'
at Socket. (/home/travis/build/nickdesaulniers/node-nanomsg/test/send.js:87:17)
at Socket.emit (events.js:95:17)
at Socket._receive (/home/travis/build/nickdesaulniers/node-nanomsg/lib/index.js:184:10)
at Socket. (/home/travis/build/nickdesaulniers/node-nanomsg/lib/index.js:198:30)

I'm pretty sure that's from this: https://github.com/nickdesaulniers/node-nanomsg/blob/master/test/send.js#L87 when I removed should.

Blocking on incoming message

I'm using nanomsg in a node.js app and had to insert a rcvtimeout option to deal with a blocking operation in nanomsg that was causing all of nodejs to become unresponsive:

``var subscriber = nano.socket('sub');
subscriber.rcvtimeo(50);

subscriber.connect( "tcp://127.0.0.1:5556" )
subscriber.on('message', function(data) {} )``

It's using socket.io and all browsers saw a "stalled" socket.io library download. Interestingly, when the debug console running nodejs was resized, it all became alive again.

With the receive timeout the issue no longer appears.

In this case, the issue only appeared when node was already running and the server with the bound pub socket would come up later. If node was killed, socket.io failed and the browsers could load. If node is brought back up, it works for the remainder of the session. So it's something related to initializing the sub socket when the pub is not there and then bringing the pub up later.

measuring performance

@comick, thanks again for that contribution fb0b057 to our perf scripts today! looks great! 👍

I think it would be nice if we could measure/compare performance of node.js strings as well as buffers.

either allowing for an additional option or maybe just adding some different scripts altogether

comparing the latency and throughput of messages against these two types of input we regularly handle in our node programs would determine the right way to structure input to our topologies.

But also in particular: inform us whether and to what extent #56 impacts the module, and other perf oriented additions we add in the future.

thoughts?

can't call any nanomsg methods after nn_term()

In libnanomsg's world, calling nn_term() is pretty final, it shuts down all the sockets and moves the whole library into a zombie state. Subsequent nanomsg calls are liable to fail in strange and peculiar ways.

Normally this is not a problem, because calling nn_term() is called at process exit, and that's all she wrote.

However, we want to unit test the nano.term() method -- there's a test case for it in test/term.js.

Currently it is only by sheer luck that this works: the npm test command (tape test/*.js) seems to pick term.js second last, which is lucky (and the test that follows it is reasonably innocuous).

Try this and you'll see what I mean:

mv test/term.js test/aardvark.js
npm test

Now, I could arrange that term.js is run last by mucking about with test/index.js or tape or the npm test definition, no doubt, and all would be fine.

Except that I'm currently testing an implementation of nn_device(), and the only way to shut down nn_device() is to call nn_term().

This means I would have two tests that both need to shut down the library. Obviously they can't both be last, which causes a bit of a problem.

As I see it, the options are:

  1. separate tape runs for each test that requires nn_term()
  2. don't test for shutdown of any nn_device()s created during tests
  3. some sort of tape/node magic that allows me to reinit the library or start a new process

Anyone got any ideas?

symbol.js tests broken

recent work may have broken the symbol.js test and symbols.js test. I might need @tcr 's help to debug this one. Probably related to #50 .

nanomsg 0.6

forked nanommsg is at 0.5 version(feb 26). When sould nanomsg be upgraded?

NN_RCVPRIO symbol not exported

this is an issue with the nn binding in our javascript.

for the new rcvprio option function, we're having to go:

/**
 * generic socket-level NN_SOL_SOCKET options
 */
var sol = {
  linger          : nn.NN_LINGER,
  sndbuf          : nn.NN_SNDBUF,
  rcvbuf          : nn.NN_RCVBUF,
  sndtimeo        : nn.NN_SNDTIMEO,
  rcvtimeo        : nn.NN_RCVTIMEO,
  reconn          : nn.NN_RECONNECT_IVL,
  maxreconn       : nn.NN_RECONNECT_IVL_MAX,
  sndprio         : nn.NN_SNDPRIO,
  rcvprio         : 9,
  tcpnodelay      : nn.NN_TCP_NODELAY,
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.