creationix / step Goto Github PK
View Code? Open in Web Editor NEWAn async control-flow library that makes stepping through logic easy.
License: MIT License
An async control-flow library that makes stepping through logic easy.
License: MIT License
When step throws an exception, it just throws the string, rather than an actual Error
object. This is bad because when an error is thrown, there is no stack trace to work with, so it's difficult to figure out the source of the error.
throw new Error('whatever');
The semantic versioning specification states:
Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.
However, in step's case, this isn't really accurate. At a certain point a project has been around long enough that it should be considered stable. Any future changes can be made in semver-major releases. (Also consider - especially for a smaller library like step, semi-frequent semver-major changes aren't the end of the world. They just aren't.)
Please publish 1.0.0 to npm to indicate that step is at least a somewhat mature project.
My first step is a two db queries in parallel (this.parallel)
My second step is to do something with the data from one of those queries, but not the other, which I want to preserve for the next step. I don't want to group them because they are not the same.
My third step is to render them both.
qv:
step(
function step1(){
person.findById(req.params.person, this.parallel());
blurb.find({ref:req.params.person}, this.parallel());
},
function step2(err, person, blurbs){
// refine blurb object, preserve person, and send them both to the next step
},
function step3(err, person, blurbs){
if (err){console.log(err);}
console.log(person);
console.log(blurbs);
});
I have been looking for a simple way to chain actions when looping over an array but did not found any. It could be great if we can simply do :
Step(
function() {
var serial = this.serial;
arr.forEach(function(el) {
asyncFunction(el, serial);
});
},
function end() {
// everything's done
}
);
but I did not succeeded in hacking the lib so I just wrote this function for the moment:
function arrayChain(arr, func, callback) {
var actions = arr.map(function(el) { return function() { func(el) }; });
actions.push(callback);
Step.apply(this, actions);
};
Do you think this feature is useful/possible ?
Is there a way of cancelling the step sequence? A "happy day" scenario for example might look like:
If an error happens in A, it appears you need to propagate it to F. Instead, I may prefer to just cancel all of the remaining Steps and execute a common error handling routine outside of my step sequence.
Is it possible to implement conditional control flow? I would like to skip over a few steps.
Something along these lines:
step(
function f1() { find('foo',this); },
function f2(err,foo) { if (foo) f4(foo); else return null; },
function f3(err) { create('foo',this); },
function f4(err,foo) { print(foo); }
);
If not, what is the best way to implement the code above?
Still trying to track down the exact issue, but here's a code snip:
getFileList( dir, dir, cb );
function getFileList( dir, root, cb ){
var files
, stats
, outFiles = [];
step( function(){
fs.readdir( dir, this );
}, function( err, _files ){
if( err ){ throw err; }
files = _files;
var stats = this.group();
for( var i=0, l=files.length; i<l; i++ ){
fs.stat( path.join( dir, files[i] ), stats() );
};
}, function( err, _stats ){
console.log( 'HERE1' );
console.trace();
}, function( err, files ){
console.log( 'HERE2' );
console.trace();
cb( null, files );
} );
}
from which I get (75% of the time):
HERE1
Trace:
at Function.<anonymous> (../docgen/fileList.js:42:9)
at next (../docgen/node_modules/step/lib/step.js:51:23)
at check (../docgen/node_modules/step/lib/step.js:73:14)
at ../docgen/node_modules/step/lib/step.js:86:20
at check (../docgen/node_modules/step/lib/step.js:101:9)
at ../docgen/node_modules/step/lib/step.js:118:22
HERE2
Trace:
at Function.<anonymous> (../docgen/fileList.js:45:9)
at next (../docgen/node_modules/step/lib/step.js:51:23)
at Array.check [as 0] (../docgen/node_modules/step/lib/step.js:73:14)
at EventEmitter._tickCallback (node.js:126:26)
The other 25% of the time it hangs as expected.
Ideally also provide a simple Makefile with a "check" rule, following GNU standards
I really like the way step
works, simplifying my code greatly. However when I read the code of step, I'm really skeptical about this:
function check() {
if (pending === 0) {
// When group is done, call the callback
localCallback(error, result);
}
}
process.nextTick(check2); // Ensures that check is called at least once
I believe the comment "Ensures that check is called at least once" actually means:
If
var group = this.group()
is executed but no subsequentgroup()
is called, we have to make sure the next step is called back anyway (withresult
equal to []).
That's right. But what if there are subsequent group()
calls? If this next-tick-check is done before any asynchronous callbacks (returned by subsequent group()
calls) are called, it's harmless, because pending
can't be equal to 0. However, if the next-tick-check is called last, it could be disastrous because localCallback
will consume another step function with the same result
.
The thing is, you can't assume process.nextTick
callback will be called in the very first place. This thread talks about it: https://groups.google.com/forum/?fromgroups=#!topic/nodejs/e8nLG5xrorA . From time to time process.nextTick
callback really happens as the last one.
I have managed to construct a counter-example. Firstly, to facilitate watching, I changed step.js
a little:
function check() {
console.trace('check by callback');
if (pending === 0) {
console.trace('do by callback');
// When group is done, call the callback
localCallback(error, result);
}
}
function check2() {
console.trace('check by nextTick');
if (pending === 0) {
console.trace('do by nextTick');
// When group is done, call the callback
localCallback(error, result);
}
}
process.nextTick(check2); // Ensures that check is called at least once
Then I run this test code:
var fs = require('fs'),
path = require('path'),
step = require('step');
var root = path.join(__dirname, 'test'),
files;
step(
function () {
fs.readdir(root, this);
},
function (err, _files) {
if (err) { throw err; return; }
files = _files;
var group = this.group();
files.forEach(function (file) { fs.stat(path.join(root, file), group()); });
},
function (err, stats) {
if (err) { throw err; return; }
var group = this.group();
files.filter(function (file, i) {
return stats[i].isFile();
}).forEach(function (file) { fs.readFile(path.join(__dirname, 'test', file), group()); });
},
function (err, contents) {
if (err) { throw err; return; }
console.log(contents);
}
);
When there are only one file and one directory under test
directory, I get the result, unfortunately, like this:
$ node test.js
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: do by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:100:17)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: check by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:106:15)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
Trace: do by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:108:17)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
[ { dev: 2051,
ino: 4335031,
mode: 33188,
nlink: 1,
uid: 1000,
gid: 1000,
rdev: 0,
size: 1557,
blksize: 4096,
blocks: 8,
atime: Tue Oct 16 2012 11:09:49 GMT+1300 (NZDT),
mtime: Tue Oct 16 2012 10:49:08 GMT+1300 (NZDT),
ctime: Tue Oct 16 2012 10:49:08 GMT+1300 (NZDT) },
{ dev: 2051,
ino: 4335032,
mode: 16893,
nlink: 2,
uid: 1000,
gid: 1000,
rdev: 0,
size: 4096,
blksize: 4096,
blocks: 8,
atime: Mon Oct 15 2012 17:48:52 GMT+1300 (NZDT),
mtime: Sun Oct 14 2012 09:46:50 GMT+1300 (NZDT),
ctime: Tue Oct 16 2012 10:49:16 GMT+1300 (NZDT) } ]
Trace: check by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:106:15)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
Trace: do by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:100:17)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
However, when I add one more file to the test
directory, the timing changes accordingly:
$ node test.js
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: do by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:100:17)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at Object.oncomplete (fs.js:297:15)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
Trace: check by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:98:15)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
Trace: do by callback
at check (/home/djkings/mdblog/node_modules/step/lib/step.js:100:17)
at next.group (/home/djkings/mdblog/node_modules/step/lib/step.js:127:22)
at fs.readFile (fs.js:176:14)
at Object.oncomplete (fs.js:297:15)
[ <Buffer 23 20 56 6f 64 6b 61 0a 0a 23 23 20 4e 65 6d 69 72 6f 66 66 20 44 65 6c 69 6b 61 74 0a 0a 21 5b 4e 65 6d 69 72 6f 66 66 20 44 65 6c 69 6b 61 74 5d 28 76 ...>,
<Buffer 23 20 47 69 6e 0a 0a 2e 2e 2e 74 68 65 20 62 65 73 74 20 64 72 69 6e 6b 20 69 6e 20 65 78 69 73 74 65 6e 63 65 20 69 73 20 74 68 65 20 50 61 6e 20 47 61 ...> ]
Trace: check by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:106:15)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
Trace: do by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:108:17)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
Trace: check by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:106:15)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
Trace: do by nextTick
at check2 (/home/djkings/mdblog/node_modules/step/lib/step.js:108:17)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
I guess now you know what I mean. Ideally we should handle the empty group case just like handling the empty parallel case, checking it synchronously rather than by process.nextTick
. As an exercise I'll try to fix this problem but it might involve major revision in step.js
.
the code example in step's readme
Step(
function readDir() {
fs.readdir(__dirname, *this*); <--------
},
function readFiles(err, results) {
if (err) throw err;
// Create a new group
* var group = this.group() *; <--------
results.forEach(function (filename) {
if (/\.js$/.test(filename)) {
fs.readFile(__dirname + "/" + filename, 'utf8', *group()*); <--------
}
});
},
function showAll(err , files) {
if (err) throw err;
console.dir(files);
}
);
can just use this more clear code instead, replace all (this, this.group, this.async) away, step jsut play a trick, it sell you something you just have own already.
var fs = require('fs');
fs.readdir(__dirname, readFiles);
function readFiles(err, results){
if (err) throw err;
// Create a new group
async.map(results, function(filename, cb){ <-------- use async.map to finally get all async result
if (/\.js$/.test(filename)) {
fs.readFile(__dirname + "/" + filename, 'utf8', cb);
}
}, showAll);
}
function showAll(err, files){
if (err) throw err;
console.dir(files);
}
I have to replace the line at number 38 with :
if (arguments[0]) {
// throw arguments[0];
var sys = require("sys");
console.log("Uncaught error: " + sys.inspect(arguments[0],true,10,true));
}
otherwise, when I run all my nodeunit tests, all testcases passed ,but this statement still thrown exception.
step.async()
Many CoffeeScript users will appreciate this change.
step(
contextObject,
function (e, next) {
// ...
next(e, 4)
},
function (e, result, next) {
doAsync(next.parallel())
doAsync(next.parallel())
},
function (e, r1, r2) {
// ...
}
)
If you omit the contextObject, "next" will still be the context of the calls but is passed as the last argument as well. Convention: The first argument of every step is the error, the number of further arguments is dynamic, the last argument is "next"
Result: There are no significant API changes, so nobody needs to change any code, but you have more sophisticated options to make step more useful.
I could send a patch to integrate this functionality.
Andi
Removing this line would be nice. :)
https://github.com/creationix/step/blob/master/lib/step.js#L102
This would cause issues when no group()
has been called, but it would allow groups to be setup later (in callbacks), which would be pretty cool.
I have this code on node v6.3.1 with step 1.0.0
var Step = require('step');
Step(
function one() {
doThis(0, "0", this);
},
function two(err, number, string) {
console.log(err, number, string);
var group = this.group();
for (var i=0; i<2; i++) {
doThis(i, String(i), group());
}
},
function three(err, numbers, strings) {
console.log(err, numbers, strings);
if (err) process.exit(1);
process.exit(0);
}
);
function doThis(number, string, callback) {
callback(null, number, string);
}
I get the following output
null 0 '0'
undefined [ 0, 1 ] undefined
So one calls do this which successfully callbacks with 0 and '0' but then I put it into a loop and use this.group() and it can no longer return the second parameter.
the expected output would be
null 0 '0'
null [ 0, 1 ] [ '0', '1' ]
When I execute the following program I expect it to crash or trow errors. However the first function is ignored and 'step 2' is printed on the console.
var step = require ('step');
if (!module.parent)
step (
function () {
asdf // <=== THIS IS WRONG
console.log ('step 1');
this ();
},
function () {
console.log ('step 2');
this ();
}
)
Is this suppose to run on Node 0.4.0? I can't seem to get second function to work. It doesn't even seem to run it because first line after standard if(err) throw err; is sys.puts('second') and it doesn't output anything.
test.js just spits out lot's of stuff and I'm not sure how to determine if the test is passing or not.
I recently tried using Step with path.exists, which doesn't pass an error
argument as the first argument to the callback. Here's the script I tested:
var Step = require('step'),
path = require('path');
var stuff = ['file1', 'file2', 'file3', 'file4'];
Step(
function () {
var group = this.group();
stuff.forEach(function (item) {
path.exists(item, group());
});
},
function (err, list) {
console.log(err);
console.log(list);
}
);
This should output null
(or some other falsy value) for error, and an array containing false
: [false, false, false, false]
. Instead, we get:
$ node stepissue.js
undefined
[ undefined, undefined, undefined, undefined ]
I don't think this is a version compatibility issue, but for good measure, I'm running Step 0.0.4, node 0.4.1 on Ubuntu 10.10.
There has to be a this.destroy() that releases all the functions in a step, otherwise there is a memory leak in the case of any error or in the case that a function wants to exit without completing the steps.
Simple leak case:
step(function(){ return; }, function(){ });
2nd function is leaked. The 1st should be allowed to call this.destroy() before returning.
To put simply:
var group = this.group();
[].forEach(group());
And:
var group = this.group();
[].forEach(function() {
doSomething(group());
});
will never terminate.
I know this is by design, but maybe there should be some documentation of how to handle such situations?
Or possibly have dummy callbacks?
Hi,
I started to create an app with the most popular node web framework Express.js and I discovered that Step might be broken if you want to use it inside of Express.
For me it seems that res.send()
is causing Step to "step forward" which is a bad, bad thing. Look at the following example app:
var express = require("express");
var step = require("step");
var app = express.createServer();
app.get("/", function(req, res)
{
step(
function one()
{
console.log("one");
if (true)
return res.send("one");
this();
},
function two()
{
console.log("two");
res.send("two");
}
);
});
app.listen(8080, function() {
console.log("App is running on port 8080");
});
If you run this example and go in your browser to http://localhost:8080
this is what you get:
As you can see in the code, the output "two" actually never should be happen because the first function inside of step should always return.
IMHO this is a really crucial bug because it makes Step (which is awesome by the way) almost unusable for most node projects.
I have this:
db.queryOne(query,insertCurso(curso,insertNotas(idAlumno,curso.id,notas,finRegistro)));
with insertCurso and insertNotas being closures, how can I implement this module in this situation?
This project really needs a change log.
I'd like a way to be able to set the step back however many times I need.
For example, setting it back 1 will execute the last action, then the action after that..
Basically, a way to alter the internal counter would be fantastic. :) (but functions like previous could also be helpful)
Hi!
Running the first example yields:
has no method 'toUpperCase'
Simple fix:
return text.toString('utf8').toUpperCase();
Regards,
Włodek Bzyl
Hello
I believe there is a race condition in handling of group()-generated callbacks in Step. The problem is described in:
http://code.google.com/p/marmalade/issues/detail?id=35
which has been worked around by the following change in Marmalade:
http://code.google.com/r/gleberp-marmalade/source/detail?r=e61d87b61000ba7eaecebdd6482d52057e4e6227
Hope this helps!
Gleb Peregud
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.