tape-testing / tape-run Goto Github PK

Using "browserify": "~3.33.1", "tape-run": "~0.1.1", when piped through browserify the tests run successfully:

gulp.task('test', function() {
  browserify('./test.js')
    .bundle()
    .pipe(run())
    .pipe(process.stdout)
})

However, if I browserify the tests, then pipe the browserified js from gulp.src:

gulp.task('test', function() {
  gulp.src('./public/js/test.js')
    .pipe(run())
    .pipe(process.stdout)
})

I get this stack trace:

TypeError: first argument must be a string or Buffer
  at ServerResponse.OutgoingMessage.write (http.js:853:11)
  at Stream.ondata (stream.js:51:26)
  at Stream.EventEmitter.emit (events.js:95:17)
  at drain (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/through/index.js:35:16)
  at Stream.stream.queue.stream.push (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/through/index.js:41:5)
  at Stream.write (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/through/index.js:16:43)
  at Stream.stream.write (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/through/index.js:25:11)
  at write (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/enstore/index.js:46:8)
  at [object Object]._onTimeout (/home/mr-house/projects/backbone-node-project/node_modules/tape-run/node_modules/browser-run/node_modules/enstore/index.js:51:7)
  at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

Running the browserified file with the cli works: tape-run < ./public/js/test.js

Running some longer tests here interacting with DOM stuff and would like to add a timeout. But looks like it does not work

browserify test/**/*.test.js | tape-run --wait=10

Tests usually take 2 seconds but with the above, they don't timeout.

so people can submit their own .html file to be used

Consider this test:

var test = require('tape')

test('...', function (t) {
  throw new Error('hmm')
  t.end()
})

Running this through tape-run like so will cause it to succeed.

$ browserify test.js | tape-run
TAP version 13
# ...
Error: hmm
    at Test.<anonymous> (http://localhost:51270/bundle.js:11:9)
    at Test.bound [as _cb] (http://localhost:51270/bundle.js:5237:32)
    at Test.run (http://localhost:51270/bundle.js:5253:10)
    at Test.bound [as run] (http://localhost:51270/bundle.js:5237:32)
    at next (http://localhost:51270/bundle.js:5055:15)
    at Item.run (http://localhost:51270/bundle.js:2566:14)
    at drainQueue (http://localhost:51270/bundle.js:2536:42)

$ echo $?
0

because if you do

$ browserify test.js | tape-run | tap-spec

you won't get the correct exit code. Better:

$ browserify test.js | tape-run --render="tap-spec"

With the browserify command, all gets bundled and in case of an error, it does not point to the correct line number of the source code. Is there a trick to make this happen?

If I have this javascript file

// hello.js
var hello = function () {
  return 'hello';
};

and this test file

// test/test.js
var test = require('tape');

test('simple test', function (t) {
  t.ok(true);
  t.end();
});

test('first test', function (t) {
  t.equal(hello(), 'hello');
  t.end();
});

How can I include hello.js when I use tape-run? something like

 browserify  test/*.js hello.js | tape-run

I'm trying to have tape-run piping its result to tap-mocha-reporter to keep consistency with the rest of the test suite of my project. But it can't make it work whatever I try:

browserify -t [ stringify --extensions [.html .twig] ] test/tests/integration/browser.js | tape-run | tap-mocha-reporter spec

Works but the output lacks the actual/expected report - while it works if I pipe to faucet for example.

browserify -t [ stringify --extensions [.html .twig] ] test/tests/integration/browser.js | tape-run --render="tap-mocha-reporter spec"

Fails, which is expected I guess.

browserify -t [ stringify --extensions [.html .twig] ] test/tests/integration/browser.js | tape-run --render="tap-spec" | tap-mocha-reporter spec

Works, but tap-mocha-reporter formatting is lost.

Is there a way to handle this?

Any chance of implementing a parameter for an HTML file that can be used as a page template?

Hey, I realise I can't provide much info but when I try tape-run on WSL it fails without any output. The identical setup works on ubuntu, and in travis ci.

Example of a repo that works everywhere but WSL: https://github.com/KoryNunn/scroll-into-view/

Example output from WSL:

/scroll-into-view$ npm test

> [email protected] test /scroll-into-view
> browserify ./test/test.js | tape-run

npm ERR! Test failed.  See above for more details.

If I had to guess it'd be because wsl runs fully headless, and there might be some assumptions being made by electron (or it's configuration) around display output.

Currently, a change in DOM at test 1 is still remains at test 2. In other words, the state of the browser is shared between test cases, which is something people try to avoid in unit tests. Is there a best practice to reset browser state between cases when using tape-run?

An obvious approach is to implement a setUp and tearDown routines into the test suite that try to undo all the changes. This however feels cumbersome, especially because some state changes are asynchronous, adding extra closure to each test case. In my case the async change is resizing of the browser window, that requires async wait to take place.

I was hoping to see an option where tape-run would run each test case in a separate browser window.

https://github.com/juliangruber/tape-run/blob/master/package.json#L20 means --cache-min Infinity freezes. Any chance you could publish https://github.com/substack/tap-finished/pull/6 as @juliangruber/tap-finished and update it here? Would allow for better offline hacking timeee ✨ Thanks!

I have a small test program:

const test = require('tape');

test('foo', t=>{
    t.ok(true);
    t.end();
});

if I run browserify ./test/index.js | tape-run

everything is fine.

However If I try to use a different browser, I keep having the same enigmatic error

internal/streams/legacy.js:59
      throw er; // Unhandled stream error in pipe.
      ^
Unexpected data: 

with any of the following command

 browserify ./test/index.js | tape-run --browser chrome

 browserify ./test/index.js | tape-run --browser=chrome

 browserify ./test/index.js | tape-run --browser="chrome"

 browserify ./test/index.js | tape-run --browser "chrome"

I am pretty sure the browsers are installed :) I am on MacOS 10.14.6

do you have any idea ?

Thanks !

I am developing a zoomable user interface library. Its headless tape-run test suite needs an ability to set window size to test how the lib behaves with windows of different sizes. However, tape-run does not provide options to specify the size. So how to overcome this?

The default windows size for phantomjs seems to be 400, 300 and for electron it is 800, 578. There is a JS method window.resizeTo but browsers do not and should not allow to call the method on the main window.

For electron, there is require('electron').remote.getCurrentWindow().setSize(width, height) that should change the size of the current window (see remote and BrowserWindow). The ability to call that from the test suite would be wonderful. Unfortunately require('electron') throws TypeError: fs.existsSync is not a function. The error is understandable, there is no fs module in the browser, although I have mocked it with {} as instructed by webpack-tape-run.

So how to solve this? How to resize the headless browser window created by tape-run?

I'm using tape-run programmatically with electron like so:

var browserify = require('browserify');
var run = require('tape-run');

var runner = run();
runner.pipe(process.stdout);

var b = browserify('./test/index.js');
b.bundle().pipe(runner);

The tests run properly and the output is fine, but afterwards an error is thrown:

net.js:1236
    throw new Error('Not running');
          ^
Error: Not running
    at Server.close (net.js:1236:11)
    at Server.server.destroy (<myproject>/node_modules/server-destroy/index.js:15:12)
    at Electron.<anonymous> (<myproject>/node_modules/browser-run/index.js:97:18)
    at Electron.EventEmitter.emit (events.js:98:17)
    at Electron._exit (<myproject>/node_modules/electron-stream/index.js:96:8)
    at ChildProcess.EventEmitter.emit (events.js:98:17)
    at Process.ChildProcess._handle.onexit (child_process.js:797:12)

Wrapping node_modules/browser-run/index.js:97 in a try/catch fixes the issue. But it seems hacky and I'd rather not use a fork.

It's possible I'm doing something wrong but I'm not sure what.

After running:
browserify test.js | tape-run -b chrome | faucet
I have some failed test, I think could be interesting to have an option to keep the process active and the browser open to use the browser debugging tools to investigate the error, something like
browserify test.js | tape-run -b chrome --keepopen | faucet

This is my test task in gulp

gulp.task('test', () => {
    browserify('spec/test.spec.js')
   .bundle()
   .pipe(run())
   .pipe(process.stdout);
});

How can I pipe that through a pretty reporter like faucet?
https://github.com/substack/faucet

Just for heads-up and thanks for the awesome project, I'll already port it to webpack as plugin.

webpack-tape-run

#64

Long shot here, but has anyone else had issues running tests with tape-run on Travis recently?

My newest project has been testing fine locally with tape-run, but failing on Travis because of tape-run.

You can see my experiments in the recent commits on this branch.

When I add a console.log('something') in the code I am testing with tape-run, that output is not shown. Any ideas why?

Hiya, I read the README and the part about how to get Electron working in Travis, but didn't grok that AFAICT this is imperative for getting tape-run to work with Travis at all. At the moment it reads more like an aside for technically-minded people who may have their own esoteric reasons for preferring Electron to Phantom, and it still wasn't obvious to me that this advice was necessary (in the absence of more involved concerns) to get Travis to build after repeated failures there, since Travis doesn't log any informative error messaging in the case of failing to do this, and its own document suggests that Phantom is a panacea.

I would PR, but I'm not confident about the exact nature of the conflict in assumptions to write up why this is the case 😅

Hi @juliangruber I'm trying to use tape-run as a light-weight testing running environment for my react-testing-recipes repo here.

One thing I've noticed is that the setup only works on Mac but not Linux, I saw you have a section to demo how to use it with travis, but unfortunately I can not use Travis in our project, it's a customised linux testing server.

I did a few research and end up with this setup in a docker environment docker-tape-run

The idea it's pretty simple by installing xvfb manually and start a sever on the system(but could also be troublesome as there's no error message at all when running on Linux), I thought this could be something helpful to be add to the readme here?

Please let me know what you think and I'll send a pull request if you are OK with that, thanks for the project by the way!

Maybe I am mistaken here (still digging around), but the --node and --basedir flags are no longer working because tape-run sets the loc option in electron-stream, which disables these.

https://github.com/nichoth/wn-blob/blob/main/package.json

It returns 'no matches for chrome' even though chrome is installed

A test:

    "test": "browserify -d -p esmify test/index.js | tape-run --browser=\"chrome\" --render=\"tap-spec\""

results:

Error [ERR_UNHANDLED_ERROR]: Unhandled error. ('no matches for chrome/*')
    at new NodeError (node:internal/errors:371:5)
    at Stream.emit (node:events:383:17)
    at Stream.onerror (node:internal/streams/readable:773:14)
    at Stream.emit (node:events:406:35)
    at Stream.reemit (/Users/nick/code/wn-blob/node_modules/throughout/node_modules/duplexer/index.js:75:16)
    at Stream.emit (node:events:406:35)
    at /Users/nick/code/wn-blob/node_modules/browser-run/index.js:103:29
    at runRunner (/Users/nick/code/wn-blob/node_modules/browser-launcher/index.js:64:14)
    at /Users/nick/code/wn-blob/node_modules/browser-launcher/index.js:74:22
    at /Users/nick/code/wn-blob/node_modules/browser-launcher/index.js:47:22 {
  code: 'ERR_UNHANDLED_ERROR',
  context: 'no matches for chrome/*'

I've been experimenting using rollup to bundle my code instead of browserify, but I can't get it working with tape-run! Has anyone else tried this?

Thanks for such a great package! Just upgraded to v3, and whenever I run my tests it produces a file in my root directory that looks like this:

.source.1489078148412.html

Which contains the compiled code with the HTML wrap. Here is the command that I'm running:

browserify -t [ babelify --presets [ es2015 ] ] test/*.js | tape-run

Tests run/exit correctly, but is there any way to not create this file? For now I can just remove them manually (new one is created with the current Date timestamp on each run).

Any chance of implementing a parameter for an HTML file that can be used as a page template?

Are tape-run deps out of date?

> browserify --bare ./test | tape-run --node

.../projects/heapwolf/tonic/index.js:121
      for await (const value of v()) {
          ^^^^^

SyntaxError: Unexpected reserved word

Hey, I think I've come upon some peculiar functionality...

var test = require('tape')

test('search query', function (t) {
  window.location.search = 'foo=bar'
  t.ok(true)
  t.end()
})

And running with:

browserify test.js | tape-run

Somehow it's run 6 times in the console.

TAP version 13
# search query
ok 1 should be truthy

1..1
# tests 1
# pass  1

# ok

TAP version 13
# search query
ok 1 should be truthy

1..1
# tests 1
# pass  1

# ok

(I only pasted 2 for brevity)

Odd, right? If you comment out the line setting search it works as expected.

EDIT: Seems to occur when setting window.location.pathname too

I use tape-run for running tests in a browser using my own test harness called tapdance which emits TAP, and it works flawlessly.

https://github.com/fortunejs/fortune/blob/master/package.json#L30
https://github.com/fortunejs/fortune/blob/master/test/browser.js

It looks like earlier versions of browserify (<=4.2.0) included a potentially vulnerable version of the syntax-error module (see advisory).

Steps to reproduce:

$ git clone [email protected]:juliangruber/tape-run.git .
$ npm i
$ npm shrinkwrap --dev
wrote npm-shrinkwrap.json

$ # sudo npm i nsp -g
$ nsp shrinkwrap
Name          Installed   Patched  Vulnerable Dependency
syntax-error    0.0.1    >= 1.1.1  tape-run > browser-run > browserify
syntax-error    0.0.1    >= 1.1.1  tape-run > browserify

You'll also need to bump the version of browser-run up to your shiny new [email protected] which includes the browserify fix there (juliangruber/browser-run#5).

And for giggles, here's a list of outdated modules, per npm outdated:

$ npm outdated --depth 0 | sort
Package       Current  Wanted  Latest  Location
browser-run     0.3.0   0.3.0   0.4.0  browser-run
browserify     2.14.2  2.14.2   5.9.3  browserify
optimist        0.5.2   0.5.2   0.6.1  optimist
tap-finished    0.0.0   0.0.0   0.0.1  tap-finished
tape            1.0.4   1.0.4  2.14.0  tape

When trying to run my tests on travis ci, I keep getting this error about firefox, even though I request phantom as the browser. My tests run just fine locally.

browserify test/*.js | ./node_modules/.bin/tape-run --browser phantom

firefox --no-remote -CreateProfile browser-launcher-6a189183 exited with code 1: Error: no display specified

See the full log here: https://travis-ci.org/lakenen/wrap-range-text/builds/48451040

Any idea what's going on here? Is tape-run not supported on travis ci? FWIW I get a similar error when using testling (which also runs fine locally). I imagine it could be a problem with browser-launcher, but I don't know enough about that project to know for sure.

tape-run fails silently on my Ubuntu 18.04 machine. However it works perfectly on other environments that I have tried.

I have tried fiddling around with the recipe described under "For gnu/linux installations without a graphical environment", but to no avail.

Any tips or magic incantations that might help?

└─( 2 ) ❱❱❱ npm test                                                                                                                                                                                            1 ⏎  +3112 21:31 ❰─┘

> [email protected] test /home/michael-heuberger/code/videomail-client
> browserify test/**/*.test.js -d -t babelify | tape-run --wait 4000

SyntaxError: Unexpected token :
npm ERR! Test failed.  See above for more details.

Using your latest version here. What can I change to get more information on the error? Doesn't tape-run have additional parameters for that? Or can you add a debug mode or so?

Unit tests of my npm library https://github.com/binarykitchen/videomail-client fail on travis ci each time (https://travis-ci.org/binarykitchen/videomail-client) and there's not much information why.

Also have asked a dev from travis to reproduce this on a docker image. Here the Gist log
https://gist.github.com/BanzaiMan/0450b14967f6677a68f253781586a058

Doesn't say much either. Is tape-run somehow hiding the real error? Command to run test browserify test/**/*.test.js --debug | tape-run --render=tap-spec

Hope anyone has a clue?

Running with:

• [email protected]
• [email protected] -> [email protected]

Output:

  ---
    operator: equal
    expected: false
    actual:   true
    at: Test.assert [as _assert] (jspm_packages/npm/[email protected]/lib/test.js:215:9)
  ...

It should be pointing to the test:

    t.test('should define the method if it is not already defined', (test) => {
        let defaultCalled = false;
        let subject: any = {};
        defineDefault(subject, 'notExist', function() {
            defaultCalled = true;
        });
        subject.notExist();
        test.isEqual(defaultCalled, false); // <-- here
        test.end();
    });

Don't know if it is related to systemjs/builder#547 or an issue with tape

Related to juliangruber/electron-stream#37, in order to use tape-run in a Docker container

Using the sample code from the readme, to just try this lib out, I get an error. Running

var test = require('tape');

test('some test', function (t) {
    t.ok(true);
    t.end();
})

with browserify test/*.js | tape-run produces

Error: spa events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: spawn ENOENT
    at errnoException (child_process.js:988:11)
    at Process.ChildProcess._handle.onexit (child_process.js:779:34)

[email protected] is currently triggering some dependabot alerts:

test cat dist/build.js | tape-run failed with webpack. The stream freeze once inside @juliangruber/tap-finished.

ref issue #35

If browserify has an error and the js doesn't get piped in, everything just hangs...

Example:

$ browserify missing.js|tape-run
Error: Cannot find module './missing.js' from '...'
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:55:21
    at load (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:69:43)
    at onex (/usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:92:31)
    at /usr/local/lib/node_modules/browserify/node_modules/resolve/lib/async.js:22:47
    at FSReqWrap.oncomplete (fs.js:82:15)

It would be nice to have a configurable timeout on that.

~/c/videomail-client ❯❯❯ browserify test/**/*.test.js --debug | tape-run                             develop ✭ ✖ ✱
(node:24784) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
SyntaxError: Invalid or unexpected token

But does not happen when using --browser chrome. How can I investigate the cause best or it is a known Electron issue?

In my opinion the environment between tape-run and tape-run -p should be as close as possible. Yet, running with -p introduces stuff like source-map-support. This can be confusing as to why source maps might work with -p but not headless electron.

Consider a test suite test/static.test.js below. It contains single test case where a static image is fetched and response headers and body inspected.

var test = require('tape')

test('test static', function (t) {
  window.fetch('image.png').then(function (response) {
    if (response.ok) {
      console.log(Array.from(response.headers.entries()))
      response.text().then(function (text) {
        console.log(text)
      })
      return t.end()
    }
    return t.end(new Error('Fetch not successful'))
  }).catch(function (err) {
    t.end(err)
  })
})

Assume test/assets/image.png is a valid PNG image. Let us run the suite. Output indicates that the response contains a HTML document and not a trace of image.png:

$ browserify test/static.test.js | tape-run --static test/assets | tap-spec

  test static

    [ [ 'connection', 'keep-alive' ],
      [ 'content-length', '83' ],
      [ 'content-type', 'text/html' ],
      [ 'date', 'Sat, 02 Dec 2017 15:57:43 GMT' ] ]
    <!DOCTYPE html><meta charset="utf8"><body><script src="/bundle.js"></script></body>


  total:     0
  passing:   0
  duration:  2.9s

However, if we replace --static test/assets flag with --port 8000 the response suddenly contains the image.png. This makes no sense!

$ browserify test/static.test.js | tape-run --port 8000 | tap-spec

  test static

    cache-control,max-age=3600,content-length,155264,content-type,image/png; charset=utf-8,date,Sat, 02 Dec 2017 15:48:51 GMT,etag,"45643276-155264-"2017-12-02T15:23:13.000Z"",last-modified,Sat, 02 Dec 2017 15:23:13 GMT,server,ecstatic-2.2.1
    �PNG
    
IHD�?1
��wTS��Ͻ7��" %�zfileH    �;HQ�
    ...

This might have something to do with #61 that also deals with unexpected behavior with --port'.

In addition, if --node is added, a TypeError: Failed to fetch is thrown. I am becoming hopeless.

$ browserify test/static.test.js | tape-run --node --static test/assets | tap-spec

  test static


    ✖ TypeError: Failed to fetch
    -----------------------------
      operator: error
      expected: |-
        undefined
...

Using latest tape-run, simple test, node v5.11.0, have you seen anything like that?

It would be nice to be able to extract test coverage information.

The way I've started to implement this (in https://github.com/rtsao/tape-run/tree/coverage and https://github.com/rtsao/browser-run/tree/messages) is as follows:

1. Add a generic duplex stream between browser-run and the browser using https://github.com/substack/shoe

2. In tape-run, prepend input test source with a script that listens for a message from the duplex stream, then sends the coverage information back.

3. If test completes with no errors, send a message to the browser via the duplex stream to signal tests have completed. When coverage is returned, exit. Or if the test returned errors, just exit.

Do you have any thoughts on this? I'm happy to create a PR when I'm finished. Any interest in including this sort of functionality? It would be nice to not use a fork.

So our testing bundle ends up having a few script tags that we write to the page. To test other bits that read from those script elements. Those are in our test code as strings. such as '<script type="text/json" name="capabilities"></script>'.

This causes browsers to throw errors when run through tape-run because the bundle is shoved into a script tag and browsers (electron in particular) get mad about that.

For right now I've worked around this by just rewriting the script tags after bundling. But that's pretty janky:

browerify-my-tests | sed -e "s/<\/script/&lt;\/script/" | tape-run

Now, this is old code that still used jquery, and this could be avoided by creating script elements with proper JS, but it'd also be great if our bundle didn't blow up if it contained </script>.

So… I guess my question is, what do you think the best way to deal with this is? My first thought was to pass the bundle as a <script src="bundle.js"> instead of entirely in a script tag, but wanted ask if you agreed with that approach.