fent / clusterhub Goto Github PK

View Code? Open in Web Editor NEW

133.0 9.0 7.0 107 KB

Sync data in your cluster applications.

License: MIT License

JavaScript 99.03% Shell 0.97%

node cluster

clusterhub's Introduction

clusterhub

An attempt at giving multi process node programs a simple and efficient way to share data.

Usage

const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
const hub = require('clusterhub');

if (cluster.isMaster) {
  // Fork workers.
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

} else {
  hub.on('event', (data) => {
    // do something with `data`
  });

  // emit event to all workers
  hub.emit('event', { foo: 'bar' });
}

Features

Efficient event emitter system. Clusterhub will not send an event to a process that isn't listening for it. Events from the same process of a listener will be emitted synchronously.
In process database. Each hub has its own instance of a redis-like database powered by EventVat.

Motive

Node.js is a perfect candidate to developing Date Intensive Real-time Applications. Load balancing in these applications can become complicated when having to share data between processes.

A remote database can be an easy solution for this, but it's not the most optimal. Communicating with a local process is several times faster than opening remote requests from a database. And even if the database is hosted locally, the overhead of communicating with yet another program is lessened.

Note that this module is experimental. It currently works by using a process's internal messaging system.

Made with Clusterhub

socket.io-clusterhub - Sync data between multi-process socket.io applications.
clusterchat - A multi-process chat that shows off socket.io-clusterhub.

API

hub.createHub(id)

Clusterhub already comes with a default global hub. But you can use this if you want to create more.

Hub#destroy()

Call to disable hub from emitting and receiving remote messages/commands.

Additionally, all functions from the regular EventEmitter are included. Plus a couple of extras.

Hub#emitLocal(event, ...args)

Emit an event only to the current process.

Hub#emitRemote(event, ...args)

Emit an event only to other worker processes and master. Or only to workers if the current process is the master.

hub.on('remotehello', () => {
  // Hello from another process.
});

hub.emitRemote('remotehello', { hello: 'there' });

All functions from EventVat are included as well. Their returned value can be accessed by providing a callback as the last argument. Or optionally by its returned value if called by the master.

worker process

hub.set('foo', 'bar', () => {
  hub.get('foo', (val) => {
    console.log(val === 'bar'); // true
  });
});

master process

let returnedVal = hub.incr('foo', (val) => {
  // Can be given a callback for consistency.
  console.log(val === 1); // true
});

// But since it's the master process it has direct access to the database.
console.log(returnedVal === 1); // true

Install

npm install clusterhub

Tests

Tests are written with mocha

npm test

clusterhub's People

Contributors

Stargazers

Watchers

Forkers

ashbrener rameshrr web5design liukun lywzx yolao durielz

clusterhub's Issues

An in-range update of mocha is breaking the build 🚨

Version 4.0.0 of mocha just got published.

Branch	Build failing 🚨
Dependency	mocha
Current Version	3.5.3
Type	devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As mocha is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Status Details

❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Release Notes

v4.0.0

4.0.0 / 2017-10-02

You might want to read this before filing a new bug! 😝

💥 Breaking Changes

For more info, please read this article.

Compatibility

#3016: Drop support for unmaintained versions of Node.js (@boneskull):
- 0.10.x
- 0.11.x
- 0.12.x
- iojs (any)
- 5.x.x
#2979: Drop support for non-ES5-compliant browsers (@boneskull):
- IE7
- IE8
- PhantomJS 1.x
#2615: Drop Bower support; old versions (3.x, etc.) will remain available (@ScottFreeCode, @boneskull)

Default Behavior

#2879: By default, Mocha will no longer force the process to exit once all tests complete. This means any test code (or code under test) which would normally prevent node from exiting will do so when run in Mocha. Supply the --exit flag to revert to pre-v4.0.0 behavior (@ScottFreeCode, @boneskull)

Reporter Output

#2095: Remove stdout: prefix from browser reporter logs (@skeggse)
#2295: Add separator in "unified diff" output (@olsonpm)
#2686: Print failure message when --forbid-pending or --forbid-only is specified (@ScottFreeCode)
#2814: Indent contexts for better readability when reporting failures (@charlierudolph)

👎 Deprecations

#2493: The --compilers command-line option is now soft-deprecated and will emit a warning on STDERR. Read this for more info and workarounds (@ScottFreeCode, @boneskull)

🎉 Enhancements

#2628: Allow override of default test suite name in XUnit reporter (@ngeor)

📖 Documentation

#3020: Link to CLA in README.md and CONTRIBUTING.md (@skeggse)

🔩 Other

#2890: Speed up build by (re-)consolidating SauceLabs tests (@boneskull)

Commits

The new version differs by 48 commits.

d69bf14 Release v4.0.0
171b9f9 pfix "prepublishOnly" potential portability problem
60e39d9 Update link to wiki (GitHub at the leading --)
804f9d5 Update link because GitHub ate the leading --
3326c23 update CHANGELOG for v4.0.0 [ci skip]
6dd9252 add link to wiki on --compilers deprecation
96318e1 Deprecate --compilers
92beda9 drop bower support
58a4c6a remove unused .npmignore
7af6611 kill Date#toISOString shim
43501a2 reduce noise about slow tests; make a few tests faster, etc.
fa228e9 update --exit / --no-exit integration test for new default behavior
3fdd3ff Switch default from forced exit to no-exit
c5d69e0 add integration tests for --exit/--no-exit
3a7f8dc enhance runMochaJSON() helper by returning the subprocess instance

There are 48 commits in total.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.

Your Greenkeeper Bot 🌴

`emit` `emitLocal` and `emitRemote` should accept callback

In order to simulate remote function call:

// worker
hub.emit('masterFunc', args, function() {
  hub.on('masterFuncReturn', function(result) {
    console.log(result);
  });
});

// master
hub.on('masterFunc', function(args) {
  hub.emit('masterFuncReturn', masterFunc(args));
});

HUB instances.

Hi,

I'd just like to confirm, HUB instances are made one per process in cluster as well as there is a separate database per process?

[bug] Error: channel closed

At first I've hope that this module is still supported by you.

This error is thrown when clusterhub tried to call child.worker.send and worker.send in fork.js and globals.js and - of course - crash whole program.

You can try to reproduce this by killing and spawning cluster workers instances.

My resolution is add simple try / catch for these lines, but probably you will find some better approach for this.

Operations should return `this` to allow chain-able function call

For example:

hub.emit('event', msg).once('anotherEvent', function(anotherMsg) {
  console.log(anotherMsg);
});

Version 10 of node.js has been released

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

Added the new Node.js version to your .travis.yml
The new Node.js version is in-range for the engines in 1 of your package.json files, so that was left alone

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

More information on this issue

Greenkeeper has checked the engines key in any package.json file, the .nvmrc file, and the .travis.yml file, if present.

engines was only updated if it defined a single version, not a range.
.nvmrc was updated to Node.js 10
.travis.yml was only changed if there was a root-level node_js that didn’t already include Node.js 10, such as node or lts/*. In this case, the new version was appended to the list. We didn’t touch job or matrix configurations because these tend to be quite specific and complex, and it’s difficult to infer what the intentions were.

For many simpler .travis.yml configurations, this PR should suffice as-is, but depending on what you’re doing it may require additional work or may not be applicable at all. We’re also aware that you may have good reasons to not update to Node.js 10, which is why this was sent as an issue and not a pull request. Feel free to delete it without comment, I’m a humble robot and won’t feel rejected 🤖

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

Your Greenkeeper Bot 🌴

Cannot read property 'getsockname' of undefined

By some reasons http server throw error when I use clusterhub in application.
node -v / 10.0.3

TypeError: Cannot read property 'getsockname' of undefined
at net.js:1049:23
at Object.4:1 (cluster.js:586:5)
at handleResponse (cluster.js:170:41)

var cluster = require('cluster');
var numCPUs = require('os').cpus().length;
var hub = require('clusterhub');

if (cluster.isMaster) {
    // Fork workers.
    var cpuCount = require('os').cpus().length;

    // Create a worker for each CPU
    for (var i = 0; i < cpuCount; i += 1) {
        cluster.fork();
    }

} else {
    hub.on('event', function(data) {
        console.log(data);
    });

    // emit event to all workers
    hub.emit('event', { foo: 'bar' });

    var http = require('http');

    http.createServer(function (req, res) {
        res.writeHead(200, {'Content-Type': 'text/plain'});
        res.end('okay');
    }).listen(3000);
}

An in-range update of mocha is breaking the build 🚨

The devDependency mocha was updated from `6.0.1` to `6.0.2`.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

mocha is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

❌ continuous-integration/appveyor/branch: AppVeyor build failed (Details).
✅ continuous-integration/travis-ci/push: The Travis CI build passed (Details).

Release Notes for v6.0.2

6.0.2 / 2019-02-25

🐛 Fixes

Two more regressions fixed:

#3768: Test file paths no longer dropped from mocha.opts (@boneskull)
#3767: --require does not break on module names that look like certain node flags (@boneskull)

Commits

The new version differs by 6 commits.

00a895f Release v6.0.2
1edce76 update CHANGELOG for v6.0.2 [ci skip]
347e9db fix broken positional arguments in config; ensure positional args are unique; closes #3763
9e31e9d fix handling of bareword args matching node flags; closes #3761
6535965 Update "karma-browserify" to eliminate Karma middleware warning (#3762)
37febb6 improve issue template. (#3411)

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

Your Greenkeeper Bot 🌴

fent / clusterhub Goto Github PK

clusterhub's Introduction

clusterhub

Usage

Features

Motive

Made with Clusterhub

API

hub.createHub(id)

Hub#destroy()

Hub#emitLocal(event, ...args)

Hub#emitRemote(event, ...args)

worker process

master process

Install

Tests

clusterhub's People

Contributors

Stargazers

Watchers

Forkers

clusterhub's Issues

Version 4.0.0 of mocha just got published.

4.0.0 / 2017-10-02

💥 Breaking Changes

Compatibility

Default Behavior

Reporter Output

👎 Deprecations

🎉 Enhancements

📖 Documentation

🔩 Other

Version 10 of Node.js (code name Dubnium) has been released! 🎊

The devDependency mocha was updated from 6.0.1 to 6.0.2.

6.0.2 / 2019-02-25

🐛 Fixes

Recommend Projects

Recommend Topics

Recommend Org

The devDependency mocha was updated from `6.0.1` to `6.0.2`.