**A clean, composable way to manipulate sequences of values with asynchronous functions **
Dealing with control flow in heavily asynchronous code can be a big challange. Without any helper functions the code can easily degenerate into a christmas tree shape because of the callback of a callback of a callback syndrome.
asncFunction1(function(err, result) {
asncFunction2(function(err, result) {
asncFunction3(function(err, result) {
asncFunction4(function(err, result) {
asncFunction5(function(err, result) {
// do something useful
})
})
})
})
})
With async.js this can be written as
async.list([
asncFunction1,
asncFunction2,
asncFunction3,
asncFunction4,
asncFunction5,
]).call().end(function(err, result) {
// do something useful
});
It gets even worse if an asynchronous function has to be applied to a sequence of values. There is a nice example of this in the howtonode.org article Control Flow in Node Part III:
// Here is the async version without helpers
function loaddir(path, callback) {
fs.readdir(path, function (err, filenames) {
if (err) { callback(err); return; }
var realfiles = [];
var count = filenames.length;
filenames.forEach(function (filename) {
fs.stat(filename, function (err, stat) {
if (err) { callback(err); return; }
if (stat.isFile()) {
realfiles.push(filename);
}
count--;
if (count === 0) {
var results = [];
realfiles.forEach(function (filename) {
fs.readFile(filename, function (err, data) {
if (err) { callback(err); return; }
results.push(data);
if (results.length === realfiles.length) {
callback(null, results);
};
});
});
}
});
});
});
}
This code reads the contents of a directory, filters out all directory and returns the contens of all files as an array. Without any helpers the whole control flow management totally obscures the intent of the code. With async.js it is possible to move the control flow aspects of the code out of the function and rewrite it like this:
function loaddir(path, callback) {
async.readdir(".")
.stat()
.filter(function(file) {
return file.stat.isFile()
})
.readFile("utf8")
.filter(function(file) {
return file.data
})
.toString(callback)
}
JavaScript Array like abstraction, which supports almost all JavaScript array functions with asynchronous versions of
- filter
- map
- reduce
- forEach
- some
- every
All functions are chainable.
Most of the functions from the fs
module in node.js are exposed as chainable filters.
stat
(lstat, fstat)unlink
mkdir
/rmdir
realpath
open
/close
exists
(fromfs.exists
)chmod
readFile
/writeFile
Constructors
files
: convert a list of files into a file sequencereaddir
: create a file sequence from the files in a directorywalkfiles
: recursive file walkerglob
: Unix style file matching. Creates a file sequence of all matching files.
Utility functions to work with the file system
abspath
: convert a relative to an absolute pathcopyfile
: copy a single filecopytree
: copy whole diroctory treermtree
: remove whole direcory tree (likerm -rf
)
Generic filters:
delay
: wait a given time before returning the next valuetimeout
: If the source doesn't respond in a given time an error is returned.inspect
: print the source value as JSONprint
: print the source value as string
Constructors are the sources of async streams. They return an async.Generator
instance which generates a sequence of values.
Filters work on the sequence generated by a source generator. They can remove, add and modify values in the sequence. Filters return themselves generator instances.
Driver methods drive the actual execution of the iteration. The hole system is based on the principle of pulling the values from the end. Until any of the drivers is called no value will be generated.
- Add documentation about the fs plugin
- Add documentation about the unit testing framework
- improve test coverage of the core functionality
- create browser version
This project is tested with Travis CI
Creates a generator generating an arithmetic progression of integers.
range(i, j)
returns [i, i+1, i+2, ..., j-1] and start
defaults to 0.
When step is given, it specifies the increment (or decrement). If stop
is not given an infinite number of values if generated. This generates all positive odd numbers:
async.range(1, null, 2)
The optional construct
argument must be a sub class of async.Generator
and can be used to tell the function to return instances of the class.
Creates a generator, which returns the elements of the given array. This generates e.g. the values 1
, 2
and 3
:
async.list([1, 2, 3])
The optional construct
argument must be a sub class of async.Generator
and can be used to tell the function to return instances of the class.
Generators are implementing the 'Iterator' design pattern in an asynchronous way. It has a next(callback)
method, which passes on each call the next generated value to the given callback. Generators can be chained and all construcors and filters return new Generator
instances.
The source
is either another async.Generator
or a generator function. All filters in the generator operate on the values generated by the source.
var i=0;
var gen = new async.Generator(function(callback) {
if (i>4)
callback(async.STOP)
else
callback(null, i++) // no error
})
gen
in this example generates the sequence [0, 1, 2, 3, 4]. The first argument of the callback can be used to indicate an error the the end of the iteration, whicle the second argument is the generated value.
Special error value, which indicates the end of the iteration.
Calls the callback with the next generated value:
gen.next(function(err, value) {
//
})
The generated value is passed in the value
argument. If an error occured while computing the value the err
argument is set. In this case value the meaning of value is undefined. async.STOP
can be passed as special error value to indicate the end of the iteration.
Usually this method is only needed to write custom filters.
Applies the mapper
function to all values of the generator's source and generates the result of the mapping.
async.range(0, 3)
.map(function(item, next) {
next(null, item + 1)
})
This will add one to each incoming item and thus generate the sequence [1, 2, 3].
The mapper can also be synchronous and simply return the result. In this case the mapper must not have the next
argument.
Calls filter for each of the generator's source values. If the filter
returns false
for a passed value, the value is dropped from the sequence, otherwise the value is forwarded.
async.list([1, 2, 3, 4])
.filter(function(item, next) {
next(null, item % 2 == 0)
})
This will filter out all odd values and generate the sequence [2, 4].
Counts the generator's source values starting with 0 and skips all values with an index smaller then begin
. If the index is >= end
the sequence is stopped.
async.range(0)
.slice(1, 4)
The async.range
call generates an infinite sequence of all integers and the slice
call transforms this into the sequence [1, 2, 3] by dropping the first value and stopping the sequence after the 4th value.
Sum up the number from 1 to 5:
async.range(1, 5)
.reduce(function(previousValue, currentValue) {
return previousValue + currentValue;
})
Sum up the number from 1 to 5 but with a first 'initialValue' of 10:
async.range(1, 5)
.reduce(function(previousValue, currentValue, index, next) {
next(null, previousValue + currentValue);
}, 10)
Alias for each
.
Calls fn
for each of the source generator's values. The returned generator passes on the source values.
async.range(1, 10)
.each(function(item, next) {
console.log(item);
})
Prints the numbers from 1 to 9. Otherwise the source sequence is not changed.
Check if any of the values in the source sequence match the condition (i.e. the condition
callback returns true). The returned generator generates a one elmement sequence, which contains the boolean value of the result.
Check if the source generates an even number:
async.list([1, 8, 3, 5])
.some(function odd(item) {
return item % 2 == 0
})
This generates [true].
Check if all of the values in the source sequence match the condition (i.e. the condition
callback returns true). The returned generator generates a one elmement sequence, which contains the boolean value of the result.
Check if the source generates only even numbers:
async.list([1, 8, 3, 5])
.every(function odd(item) {
return item % 2 == 0
})
This returns [false]
Calls each of source values in sequence, which have to be functions. The functions can either be synchronous and return a value of be asynchronous and call the passed callback on completion. The generated sequence are the function's return values. The optional context
argument defines the this
context of the called functions.
async.list([
function sync() {
return "juhu"
},
function async(next) {
next(null, "kinners")
}
]).call()
This calls the two functions in sequence and generates the sequence ["juhu", "kinners"].
Concatenates a variable number of generators with this generator.
async.range(1, 4)
.concat(async.range(10, 13))
This returns a generator, which generates the values of the first range
followed by the values of the second range
: [1, 2, 3, 10, 11, 12]
Returns a generator, which generates on each next()
an arroy of the next values of this generator and each of the passed arguments. The length of the generated sequence is truncated to the lenght of the shortest input sequence. This function expects a variable list of generators as arguments.
async.range(1, 4)
.zip(async.range(10, 14))
This will generate [[1, 10], [2, 11], [3, 13]]
Performs an Array.prototype.sort on the source values and returns a generator, which generates the result as one element sequence. Since this requires reading all source elements, the source must not generate an infinite number of values. Generates the read values in sorted order.
async.list([4, 2, 3, 9])
.sort()
This generates the sequence [2, 3, 4, 9]
Performs an Array.prototype.join on the source values and returns a generator, which generates the result as one element sequence. Since this requires reading all source elements, the source must not generate an infinite number of values.
async.list([1, 4, 5])
.join(" - ")
This will generate ["1 - 4 - 5"]
Reverses the elements generated by the generator's source. Since this requires reading all source elements, the source must not generate an infinite number of values.
async.list([1, 4, 5])
.revers()
This will generate [5, 4, 1]
Perform the iteration until the generator's source either returns async.STOP
or indicates an error. The value of the last generated value is passed to the callback.
async.list([1, 8, 3, 5])
.every(function odd(item) {
return item % 2 == 0
})
.end(function(err, allEven) {
console.log("All values are even: " + allEven)
})
This will print the last generated value (the result of very
) on the console.
Perform the iteration until the generator's source either returns async.STOP
. If breakOnError
is true the iteration is stopped on the first error. Otherwise the iteration continues and all errors are collected in an error array. An array of all generated values is passed to the callback.
async.list([1, 8, 3, 5])
.map(function odd(item, next) {
next(err, item * 10)
})
.toArray(function(err, values) {
console.log(values)
})
The last callback will be called with arr set to [10, 80, 30, 50].