Giter VIP home page Giter VIP logo

metarhia / noroutine Goto Github PK

View Code? Open in Web Editor NEW
120.0 15.0 11.0 152 KB

Goroutine analogue for Node.js, spreads I/O-bound routine calls to utilize thread pool (worker_threads) using balancer with event loop utilization. ๐ŸŒฑ

Home Page: https://metarhia.com

License: MIT License

JavaScript 100.00%
multithreading parallel balancer metarhia concurrency threads workers routine goroutine

noroutine's Introduction

Node Routine (noroutine)

ci status snyk npm version npm downloads/month npm downloads license

Goroutine analogue for Node.js, spreads I/O-bound routine (tasks) to utilize thread pool with worker_threads using balancer with event loop utilization (see perf_hooks API).

Usage

Install: npm install noroutine

const noroutine = require('noroutine');
const module1 = require('./module1.js');
const module2 = require('./module2.js');
noroutine.init({ modules: [module1, module2] });

(async () => {
  const res1 = await module1.method1('value1');
  const res2 = await module2.method2('value2');
  console.log({ res1, res2 });
})();

Initialization options

noroutine.init({
  modules: [module1, module2],
  pool: 5, // number of workers in thread pool
  wait: 2000, // maximum delay to wait for a free thread
  timeout: 5000, // maximum timeout for executing a functions
  monitoring: 5000, // event loop utilization monitoring interval
});

License & Contributors

Copyright (c) 2021-2024 Metarhia contributors. Noroutine is MIT licensed.
Noroutine is a part of Metarhia technology stack.

noroutine's People

Contributors

cleverid avatar saroamirkhanyan avatar tshemsedinov avatar wansmer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

noroutine's Issues

Relative path in noroutine.js

Describe the bug
In file 'noroutine.js' on line 102 path for './lib/worker.js' is relative. This leads to error: trying find this file in working directory instead of package folder.

Example:

node:internal/event_target:646
  process.nextTick(() => { throw err; });
                           ^
Error: Cannot find module '/Users/wansmer/projects/code/noroutine-sh/lib/worker.js'
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12)
    at MessagePort.<anonymous> (node:internal/main/worker_thread:187:24)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:460:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28)
Emitted 'error' event on Worker instance at:
    at Worker.[kOnErrorMessage] (node:internal/worker:298:10)
    at Worker.[kOnMessage] (node:internal/worker:309:37)
    at MessagePort.<anonymous> (node:internal/worker:210:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:460:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

To Reproduce
Install noroutine as a dependency to your project. Try run code from example from README.md

Desktop (please complete the following information):

  • OS: MacOs 11.6
  • Node.js version 16.6.1
  • noroutine version 0.0.2

Debug mode: disable multi-threading

For debug mode we can add parameter debug: boolean

noroutine.init({
  modules: [module1, module2],
  debug: true,
});

this will disable multi-threading and ballancing.

Restart failed workers

Terminate worker on uncaughtException and unhandledRejection with code 1 and restart worker from master thread

Use default values for options

Default values:

{
  pool: 5, // number of workers in thread pool
  wait: 2000, // maximum delay to wait for a free thread
  timeout: 5000, // maximum timeout for executing a functions
  monitoring: 5000, // event loop utilization monitoring interval
}

so options for norutine.init will be not required

Balancing not working as expected

const noroutine = require('noroutine');
const module1 = require('./module1.js');
const module2 = require('./module2.js);
noroutine.init({ modules: [module1, module2] });

(async () => {
  const res11 = await module1.method1('value1');
  const res12 = await module1.method2('value2');
  const res22 = await module2.method2('value21');
})();

actual: run one by one in different workers
expected: more suitable example

(async () => {
  const res11 = module1.method1('value1');
  const res12 = module1.method2('value2');
  const res22 = module2.method2('value21');

  await res11;
  await res12;
  await res22;
  // or Promise.all
})();

actual: functions run one by one in first worker 
expected: functions run in parallel 

worker utilization sample for http requests for default options:
actual:
1 -----__________-----
2 _____-----__________
3 __________-----_____
4 ____________________

expected:

1 -----------_------_-
2 _----_--------------
3 _-------------_-----
4 --------_-----------

Pass function as a parameter

Is your feature request related to a problem? Please describe.

Now methods of modules can work with with primitives and objects only, bun no with a functions.

Code below throw Error, because postMessage no work with function:

(async () => {
  const res = await module1.method1(x => x ** x);
  console.log({ res });
})();

Describe the solution you'd like

There an idea of implementation with vm: https://github.com/Wansmer/noroutine/pull/1

But there is open questions:

  1. Is a good approach to create new sandbox for every higher order function?
  2. Is need disable modules like os and fs for safety or user must make decision himself? (set flags in options or generate and pass own context, for example);
  3. If 'dangerous' code not expected (from end user), maybe eval in this case will be better and lighter solution instead vm.Script?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.