A task queue module for node.js. You can set several children-queue for one task queue.

At first, I wrote this module is for searching one song in 萌否收音機. And last I found that song named <the Embodiment of Scarlet Devil>.

For rembembering this and for my favorite Flandre Scarlet, I named this module Scarlet Task.

For one situation, once you want to crawl one website. If you use primitive node.js, it will like you're DDOSing that website.

So you need a task queue to help you. It will process tasks in queue one by one.

What's more, you can set that one queue has several children-queue to work concurrently.

And you can use it at any other situation that suitable.

$ npm install scarlet-task

Require the module at first and instantiate an object.

var TaskQueue = require("scarlet-task");
var taskQueue = new TaskQueue(10);

The parameter for constructor means number of children-queue. Pass no parameter for default 1 children-queue.

Define a processor function for one task. In fact, you can pass an anonymous function.

function processor(taskObject) {
	// get task object
    var task = taskObject.task;
    
    // Do something...
    // blahblah...
    
    taskQueue.taskDone(taskObject);
    console.log(taskQueue.numberOfProcessed());
};

Notice: In the processor function, you should call taskQueue.taskDone(taskObject) when you think this task is done. And then the taskQueue will process next task. The parameter taskObject is a parameter that taskQueue passed to you.

You can push task(s) at anytime.

The task object can be any type - string, number, json, etc.

var task = "it may be a url, or an object that process can do something with this task object.";
taskQueue.push(task, processor);

See more reference at test/touhou.js.

What's more, if you want to see the queue status for debuging, you can pass true to taskDone and push.

eg.

taskQueue.taskDone(taskObject, true);
taskQueue.push(task, processor, true);

You can reset the number of processed tasks as well:

taskQueue.resetNumberOfProcessed();

And you can set an after-finish function so that Scarlet will call it after a certain number of tasks finished.

taskQueue.afterFinish(20, done, false);
// this will call done() after 20 tasks done without loop (means only once unless you reset number of processed).

taskQueue.clearAfterFinish();
// You can clear after finish processor

See more reference at test/afterfinish.js.

Only me - XadillaX so far.

You can contribute your code! You're welcome.

「雖然我覺得不怎麼可能有人會關注我」