gulp-multi-thread-task's People
Forkers
nhkyxxgulp-multi-thread-task's Issues
handlers[message.type] is not a function
Any idea why these happens?
$ npx gulp lint:js2
[04:23:20] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:20] Starting 'lint:js2'...
[04:23:20] spawning 4 worker
[04:23:22] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:22] Starting 'lint:js2'...
[04:23:22] 'lint:js2' errored after 2 s
[04:23:22] TypeError: handlers[message.type] is not a function
at Worker.<anonymous> (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/index.js:30:29)
at Worker.emit (events.js:326:22)
at Worker.EventEmitter.emit (domain.js:506:15)
at ChildProcess.<anonymous> (internal/cluster/worker.js:32:12)
at ChildProcess.emit (events.js:314:20)
at ChildProcess.EventEmitter.emit (domain.js:506:15)
at emit (internal/child_process.js:876:12)
at processTicksAndRejections (internal/process/task_queues.js:85:21)
[04:23:22] Using gulpfile ~/src/projectName/gulpfile.js[04:23:22] Starting 'lint:js2'...
[04:23:22] The following tasks did not complete: lint:js2
[04:23:22] Did you forget to signal async completion?
[04:23:22] The following tasks did not complete: lint:js2
[04:23:22] Did you forget to signal async completion?
[04:23:23] Using gulpfile ~/src/projectName/gulpfile.js
[04:23:23] Starting 'lint:js2'...
[04:23:23] 'lint:js2' errored after 3.54 ms
[04:23:23] Error: write EPIPE
at process.target._send (internal/child_process.js:806:20)
at process.target.send (internal/child_process.js:677:19)
at _requestFile (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/lib/message-handlers.js:75:13)
at workerMessageHandlers (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/lib/message-handlers.js:110:3)
at GulpMultiThreadTask (/home/bobby/src/projectName/node_modules/gulp-multi-thread-task/index.js:77:27)
at /home/bobby/src/projectName/gulpfile.js:81:10
at lint:js2 (/home/bobby/src/projectName/node_modules/undertaker/lib/set-task.js:13:15)
at bound (domain.js:427:14)
at runBound (domain.js:440:12)
at asyncRunner (/home/bobby/src/projectName/node_modules/async-done/index.js:55:18)
I believe the write EPIPE
errors were simply due to the gulp task ending prematurely due to the first handlers[message.type] is not a function
error.
Here's the snippet of my gulp file:
const gulp = require('gulp')
const { GulpMultiThreadTask } = require('gulp-multi-thread-task')
const eslint = require('gulp-eslint7')
// ...
function lintJs2 (done, src) {
return gulp.src(src)
.pipe(eslint())
.pipe(eslint.format('stylish'))
.pipe(eslint.failAfterError())
}
gulp.task('lint:js2', function (done, src) {
return GulpMultiThreadTask(
'lint:js2',
[
'./*.js',
'./{scripts,src}/**/*.js'
],
lintJs2,
{
concurrency: Math.min(require('os').cpus().length / 2)
}
)
})
// ...
Plugin slightly slower performance compared to regular gulp
Hi,
I tried your plugin for my gulp task that processes images, but found that it was actually slower than without it.. Not sure if I'm doing something wrong, my task is not suitable for multi-threading or that the plugin might have some issue?
Situation
I have some directories with JPG, PNG and GIF image files. They have to be converted to WebP and placed besides their original counterpart.
System
- PC: Windows 10 on quad core processor
- Node: v14.8.0
- Npm: v6.14.8
Directory structure
cdn/
pictures/
(around 120 .jpg, .png and .gif files, average filesize ~120kB)
thumbs/
(around 110 .jpg, .png and .gif files, average filesize ~10kB)
gulpfile.js
const { task, src, dest } = require('gulp');
const {GulpMultiThreadTask} = require('gulp-multi-thread-task');
const gulpimagemin = require('gulp-imagemin'); // For image handling
const imagemin_webp = require('imagemin-webp'); // Converts JPG/PNG to WebP
const imagemin_gif2webp = require('imagemin-gif2webp'); // Converts only GIF to WebP
const rename = require('gulp-rename');
function webpPicturesTask(done, srcGlob) {
return src(srcGlob, { base: 'cdn/' })
.pipe(gulpimagemin([
imagemin_webp({
lossless: false,
method: 6,
}),
imagemin_gif2webp({
lossy: false,
mixed: false,
minimize: false,
method: 6,
multiThreading: true,
})
], {
verbose: true
}))
.pipe(rename({ extname: '.webp' }))
.pipe(dest('cdn/'))
;
task('webpPictures', function(done, srcGlob) {
return GulpMultiThreadTask(
'webpPictures',
['cdn/{pictures,thumbs}/**/*.{png,jpg,jpeg,gif}'],
webpPicturesTask
);
});
Results
- Run as a normal task, it processed all 231 files in 54 seconds.
- Run in a multi-threaded set-up as above, it did the same in 65 seconds. (Using 4 workers.)
The additional time to spawn the workers at the start was very small (~1 sec), so I'm confused about the additional time it took.
Any idea what I'm doing wrong or why it is slower?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.