Update from moderator:

The biggest issue is that our documentation shows global use of nyc without ever suggesting that nyc be installed globally.

Original Message:

I am not quite sure how the instructions in the README ever worked. npm i nyc installs the module just locally and nothing gets added to the path. Calling

nyc tap ./test/*.js

gives a "command not found" (to no surprise). One would think that

./node_modules/nyc/bin/nyc.js tap ./test/*.js

should work. But instead that gives

    $ ./node_modules/nyc/bin/nyc.js tap ./test/*.js
    events.js:141
          throw er; // Unhandled 'error' event
          ^

    Error: spawn tap ENOENT
        at exports._errnoException (util.js:874:11)
        at Process.ChildProcess._handle.onexit (internal/child_process.js:178:32)
        at onErrorNT (internal/child_process.js:344:16)
        at doNTCallback2 (node.js:439:9)
        at process._tickCallback (node.js:353:17)
        at Function.Module.runMain (module.js:469:11)
        at startup (node.js:134:18)
        at node.js:961:3

We already have rimraf as a dependency, and we're not using del's glob functionality -- we can probably use rimraf where we currently use delete.

To get a report I currently run

nyc node test/index.js && nyc report --reporter text --reporter html

It would be great to just run

nyc node test/inedx.js --reporter text --reporter html

Also note I'm relying on the "--reporter" can appear multiple times feature which may be accidental :)

Hi,

So I tried out nyc on my project that I test with tape. I noticed that it doesn't report about files that are never touched by unit tests. Is this expected behavior? Everything looks very green, when in reality some files aren't yet tested at all.

Hello.

I'm trying to use nyc with Babel.
But I got a stackoverflow error.
What am I doing wrong?

> nyc --require babel-register node_modules/.bin/mocha test.js

path.js:174
  resolvedTail = normalizeArray(resolvedTail.split(/[\\\/]+/),
                                             ^

RangeError: Maximum call stack size exceeded
    at String.split (native)
    at Object.win32.resolve (path.js:174:46)
    at Object.win32.relative (path.js:259:16)
    at getRelativePath (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:72:28)
    at shouldIgnore (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:123:12)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:137:9)
    at requireHook (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\nyc\index.js:139:7)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:138:7)
    at requireHook (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\nyc\index.js:139:7)
    at require.extensions.(anonymous function) (C:\Users\t-nagashima.AD\Documents\GitHub\sand\node_modules\babel-register\lib\node.js:138:7)

Windows 7 64bit
Node v4.2.2
npm v3.5.0

package.json

{
  "name": "nyc-example",
  "version": "0.0.0",
  "private": true,
  "description": "Please run `npm install` and `npm test`.",
  "scripts": {
    "test": "nyc --require babel-register node_modules/.bin/mocha test.js"
  },
  "devDependencies": {
    "babel-core": "^6.3.2",
    "babel-preset-es2015": "^6.1.18",
    "babel-register": "^6.3.2",
    "mocha": "^2.3.4",
    "nyc": "^4.0.1"
  }
}

.babelrc

{
  "presets": ["es2015"]
}

test.js

class Foo {
}

console.log(Foo);

As first reported via https://twitter.com/boennemann/status/619103847043203072

I'm trying to instrument the integration tests for https://github.com/semantic-release/semantic-release.
They are running a shell script to start the npm-registry-couchapp before tap/nixt tests are running against it. Everything works fine, but once I run them with nyc I get this error (multiple times):

[eval]:1
/private/tmp/node-spawn-wrap-17602-9d862ee799ec/node
                                   ^
SyntaxError: Unexpected token ILLEGAL
    at Object.exports.runInThisContext (vm.js:53:16)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:426:26)
    at node.js:567:27
    at doNTCallback0 (node.js:408:9)
    at process._tickCallback (node.js:337:13)

Steps to reproduce:

git clone [email protected]:semantic-release/semantic-release.git
cd semantic-release
git fetch origin  next:next
git checkout next
npm install
npm run build && npm run test:build
npm run test:unit # nyc built in, passes and reports coverage
npm run test:integration # passes
nyc npm run test:integration # fails

You need couchdb installed. Sorry for the bloated test case, will try to reduce it if this doesn't get you anywhere.

The relevant script is here: https://github.com/semantic-release/semantic-release/blob/next/test/registry/start.sh
Which gets execd here: https://github.com/semantic-release/semantic-release/blob/next/test/registry/index.js#L9

Make it nyc_output => .nyc_output.

It's only a temp folder for reports so there's nothing for end-users to see there.

Hi

I'm using nyc with mocha and coveralls
I've setup my .travis.yml and package.json as specified on the readme

The test is passing on travis and test coverage are send to coveralls.
But when I do npm test on my computer, it fails

> [email protected] test C:\perso\node-annotation-parser
> nyc mocha

events.js:85
      throw er; // Unhandled 'error' event
            ^
Error: spawn mocha ENOENT
    at exports._errnoException (util.js:746:11)
    at Process.ChildProcess._handle.onexit (child_process.js:1053:32)
    at child_process.js:1144:20
    at process._tickCallback (node.js:355:11)
    at Function.Module.runMain (module.js:503:11)
    at startup (node.js:129:16)
    at node.js:814:3
npm ERR! Test failed.  See above for more details.

The repo: https://github.com/mastilver/node-annotation-parser

Those annoying comments are the main reason I never bothered trying out coveralls.

https://github.com/bcoe/nyc#integrating-with-coveralls

the tap suite runs several tests on require-hooks itself, these seem to be failing on newer versions of nyc:

test/require-hooks.js ................................. 4/6 6s
  compile-to-js require hook > failing test with unmatched stack-trace
  not ok error happened in the *.faux file
    found: ''
    pattern: 'file: test/fixtures/using-require-hook.faux'
    at:
      file: test/require-hooks.js
      line: 9
      column: 1000
      function: verifyOutput
    source: |
      test('compile-to-js require hook', function (t) {
    stack: |
      verifyOutput (test/require-hooks.js:9:1000)
      ChildProcess.exithandler (child_process.js:204:5)
      maybeClose (internal/child_process.js:764:16)
      Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

  compile-to-js require hook > failing test with unmatched stack-trace
  not ok error happened in the *.faux file
    found: >
      /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



          1..0



      not ok 1 -
      /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        ---



        results:



          ok: false



          count: 0



          pass: 0



          plan:



            start: 1



            end: 0



            skipAll: true



        arguments:



          - /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        exitCode: 1



        timeout: 240000



        command: /Users/benjamincoe/.nvm/versions/io.js/v3.2.0/bin/iojs



        file: /Users/benjamincoe/bcoe/node-tap/test/fixtures/using-require-hook.js



        ...







      1..1
    pattern: 'file: test/fixtures/using-require-hook.faux'
    at:
      file: test/require-hooks.js
      line: 9
      column: 1000
      function: verifyOutput
    source: |
      test('compile-to-js require hook', function (t) {
    stack: |
      verifyOutput (test/require-hooks.js:9:1000)
      ChildProcess.exithandler (child_process.js:204:5)
      maybeClose (internal/child_process.js:764:16)
      Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)

I bet this is being caused by the overall of require-hook that we've been iterating on.

It would be great to get this fixed so that @isaacs can pull the new improved version into tap.

Any thoughts: @jamestalmage, @novemberborn?

According to @sindresorhus, that is the more common convention.

Obviously we would need to support the current usage as well for the foreseeable future.

is-windows returns a function instead of the advertised boolean: https://github.com/jonschlinkert/is-windows/issues/2

That means

if (isWindows) return done()

always skips the test

It would be nice if there was some environment variable that could be set that would disable the nyc injection in a child process (and any of its descendants).

The use case is for writing multi-process tests for things like process supervisors where instrumenting both the supervisor and the example app it is running is needless overhead when what is being tested is how the supervisor responds in integration tests.

I'm thinking something like NYC_DISABLE or something, or maybe a max depth on the number of process boundaries to cross.

Is there any way to control where it spits out the generated report without involving .istanbul.yml?

There is no .json file created in the project folder. Any tips on this @bcoe ? tnx

Running nyc tape test/*.js, I see the output of both the tests and the coverage report. When I run the coverage reporter, I'm not actually interested in the test output. It would be nice if there was an option like --silent for the test command itself, to suppress its output and only have the coverage report printed to my terminal.

I have a test file that require a relative file (the file to test) https://github.com/MoOx/statinamic/blob/19fc8085ceb14a55a149feb330526f4275889cfa/src/PageContainer/__tests__/component.js#L8
But when I run test using nyc babel-tape-runner src/**/__tests__/*.js (or nyc --require babel-core/register tape src/**/__tests__/*.js) I got weird result that include the entire filename after the relative parent folder like this

...

1..23
# tests 23
# pass  23

# ok

-------------------------------------------------------------------------------------------------------------------|----------|----------|----------|----------|----------------|
File                                                                                                               |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
-------------------------------------------------------------------------------------------------------------------|----------|----------|----------|----------|----------------|
 PageContainer/Users/MoOx/Sync/Development/statinamic/src/PageContainer/                                           |    73.33 |    71.88 |      100 |    73.33 |                |
  component.js                                                                                                     |    73.33 |    71.88 |      100 |    73.33 |... 79,80,84,87 |
 PageContainer/__tests__/Users/MoOx/Sync/Development/statinamic/src/PageContainer/__tests__/                       |      100 |      100 |       25 |      100 |                |
  component.js                                                                                                     |      100 |      100 |       25 |      100 |                |

When I check the report, I can clearly see the issue

$ nyc report --reporter=text-lcov
TN:
SF:./src/PageContainer/__tests__/Users/MoOx/Sync/Development/statinamic/src/PageContainer/__tests__/component.js
FN:15,noop
FN:16,Page
FN:17,PageError
FN:19,(anonymous_5)
FNF:4
FNH:1
...

Note that es6 import syntax is transpiled by babel to the same relative path

$ babel src/PageContainer/__tests__/component.js 

...

var _react2 = _interopRequireDefault(_react);

var _reactAddonsTestUtils = require("react-addons-test-utils");

var _component = require("../component"); /// <- same relative path

What am I doing wrong to get this weird path?

Files in .nyc_output are at https://gist.github.com/aredridel/67a635a727047e541c32

Output:

:; nyc report
------------|----------|----------|----------|----------|----------------|
File        |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------|----------|----------|----------|----------|----------------|
 __root__/  |    94.92 |    81.13 |      100 |    94.92 |                |
  common.js |    94.37 |    69.23 |      100 |    94.37 | 94,146,150,154 |
  index.js  |    95.74 |    92.59 |      100 |    95.74 |          48,68 |
------------|----------|----------|----------|----------|----------------|
All files   |    94.92 |    81.13 |      100 |    94.92 |                |
------------|----------|----------|----------|----------|----------------|

Trying to use latest (master) nyc with a custom tap reporter:

$ nyc tap --reporter=spec

Things work fine until the end, when nyc tries to find a spec reporter.

The typical solution is to stop processing after a -- delimiter is found

$ nyc tap -- --reporter=spec

I ran into a strange bug that will kill nyc right away without any error message.

Just comment this line out and running that test will kill nyc (that line should actually be deleted but the tests won't run afterwards anymore).

I am trying to use nyc with ava and running into a small issue for good measure here's a link to the conversation at ava repo.

So essentially i have a few directories in node_modules directory that i would like to cover since i do unit tests for them.

Unfortunately a setup like so:

"config": {
    "nyc": {
      "exclude": [
        "node_modules/"
      ]
    }
  }

results in no coverage at all.

setting up more specific like node_modules/utils which are some of the files i need to cover results in a coverage containing too many files/packages such as babel, etc which i am not testing.

any suggestions on how to allow certain files from within node_modules directory?

thanks in advanced

I was able to get istanbul check-coverage working using it's glob functionality

istanbul check-coverage --branches=100 --lines=100 --functions=100 '.nyc_output/*.json'

It would be nice if there was a nyc check-coverage command that did this for me and was documented with the nyc tool.

It would be nice to have some unit tests for the nyc bin, I'm fine with these simply using spawn and asserting against stdout/stderr.

There is at least one test that still requires existing state from a previous test.

I propose we wipe out .nyc_output and test/fixtures/.nyc_output between tests, as well as the cache directories.

Any tests that fail due to these changes should be updated to explicitly generate their prerequisite state.

Hi! Are there any alternatives to /* istanbul ignore next */ and similar? Doesn't seem like the Istanbul variant is taken into consideration at least.

Multiple tests in test/nyc-test instantiated NYC and call wrap(), wrapping the current process. This leads to poor test isolation. Each test should probably be run in its own process.

It'd be cool to specify (via env probably, so as to not mess with args?) a minimum level of statement/branch coverage, below which it's considered a failure.

Coveralls provides this, so the travis/coveralls wire-up I've got for node-tap meets my needs for PRs and such. But it'd be good to surface this to a potential contributor ahead of time. Like, "Yes, your tests pass, but you dropped the coverage below acceptable amounts, so fix that first."

I'm trying out nyc on my project https://github.com/Janpot/fantastiq with nyc npm test which yields:

-----------------|----------|----------|----------|----------|----------------|
File             |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
-----------------|----------|----------|----------|----------|----------------|
 bin/            |      100 |      100 |      100 |      100 |                |
  fantastiq.js   |      100 |      100 |      100 |      100 |                |
 lib/            |    29.05 |    10.98 |     8.21 |    29.11 |                |
  Queue.js       |    39.73 |       20 |     12.5 |       40 |... 412,416,420 |
  QueueClient.js |    51.11 |        0 |       10 |    51.11 |... 50,52,58,59 |
  Worker.js      |    15.19 |        0 |        0 |    15.19 |... 131,134,139 |
  index.js       |    56.52 |      100 |    16.67 |    56.52 |... 29,30,31,37 |
  jobUtils.js    |    30.43 |     7.14 |       25 |    30.43 |... 36,38,39,40 |
  message.js     |       12 |        0 |        0 |       12 |... 32,33,34,39 |
  metrics.js     |    41.67 |       25 |    33.33 |    41.67 |... 32,33,34,37 |
  router.js      |    10.91 |        0 |        0 |    10.91 |... 247,253,255 |
-----------------|----------|----------|----------|----------|----------------|
All files        |    35.44 |    15.85 |    16.33 |    35.51 |                |
-----------------|----------|----------|----------|----------|----------------|

This looked very odd to me as I expected my coverage to be closer to 100%. Checking out some of the lines I was sure they were all called in my tests. After some trial and error I found that after skipping some tests I get the (hopefully) correct coverage:

----------|----------|----------|----------|----------|----------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------|----------|----------|----------|----------|----------------|
----------|----------|----------|----------|----------|----------------|
All files |      100 |      100 |      100 |      100 |                |
----------|----------|----------|----------|----------|----------------|

The tests I skipped were https://github.com/Janpot/fantastiq/blob/master/test/cli.spec.js#L13 (describe.skip(...)

This part of the code runs some commands with child_process.exec.

If you want to try this out you'll need to have a redis running somewhere, if it's not at 127.0.0.1 you can specify one with the REDIS_HOST variable.

@jamestalmage I know this is something that was fixed at one point, but I can't seem to get the lcov report working if --cache is set for yarsay.

Benjamins-MacBook-Pro:yarsay benjamincoe$ nyc report --reporter=lcov
/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:241
            text = structuredText[startLine].text;
                                            ^

TypeError: Cannot read property 'text' of undefined
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:241:45
    at Array.forEach (native)
    at annotateFunctions (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:224:26)
    at HtmlReport.Report.mix.writeDetailPage (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:427:9)
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:489:26
    at SyncFileWriter.extend.writeFile (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/util/file-writer.js:57:9)
    at FileWriter.extend.writeFile (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/util/file-writer.js:147:23)
    at /Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:488:24
    at Array.forEach (native)
    at HtmlReport.Report.mix.writeFiles (/Users/benjamincoe/bcoe/nyc/node_modules/istanbul/lib/report/html.js:482:23)

I rolled back several commits, and it doesn't seem as though this was ever working on master.

If not using the require flag, this line:
https://github.com/bcoe/nyc/blob/master/bin/nyc.js#L9
returns [""]
which when combined with this commit:
f33ac6d
breaks with this error:
AssertionError: missing path

I'll put together a PR to quickly fix...

All the branches merged were in the green, but it seems as though the combined build is failing on travis:

https://travis-ci.org/bcoe/nyc

Currently you exclude it at any level.

This excludes https://github.com/sindresorhus/ava/blob/master/lib/test.js even though it's lib/test.js.

Ticket to track progress. @bcoe should probably flesh this out more 😺

$(npm bin)/tap test/nyc-test.js fails, where $(npm bin)/tap --coverage test/nyc-test.js succeeds. This seems wrong, tap comes bundled with nyc for its own coverage reports but that shouldn't impact the tests.

The README suggests that this shouldn't happen.

I notice I see it for the only one of my deps that are npm linked into my project.

Its not a big deal, I can visually skip over it the parts I don't care of, but is this intended behaviour?

Hi

I am writing tests for my library with AVA. I want to test my code coverage with nyc, but something seems wrong.

This is the library I am talking about: https://github.com/samverschueren/dynongo

I only wrote tests for the libs/utils/name.js file but nyc reports no coverage at all.

If I run nyc on a project tested with mocha, the report is correct. Am I missing something here?

Thanks in advance!

Right now nyc fail while handling reverse source-mapping when the original source contains async functions like this one:

export function func() {
    return new Promise((resolve, reject) => {
        console.log('test');
        resolve(true);
    });
}

/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/array-set.js:91
    throw new Error('No element indexed by ' + aIdx);
    ^

Error: No element indexed by 1
    at ArraySet_at [as at] (/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/array-set.js:91:11)
    at SourceMapConsumer_originalPositionFor [as originalPositionFor] (/tmp/test-cover/node_modules/nyc/node_modules/source-map/lib/source-map-consumer.js:618:36)
    at mapLocation (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:33:25)
    at /tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:93:18
    at Array.forEach (native)
    at SourceMapCache._rewriteStatements (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:92:40)
    at /tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:23:11
    at Array.forEach (native)
    at SourceMapCache.applySourceMaps (/tmp/test-cover/node_modules/nyc/lib/source-map-cache.js:15:23)
    at NYC.writeCoverageFile (/tmp/test-cover/node_modules/nyc/index.js:263:25)
    at EventEmitter.onExit.alwaysLast (/tmp/test-cover/node_modules/nyc/index.js:240:11)

The istanbul spec calls for absolute paths, but we pass in relative ones. This is not just a nit, it causes real problems.

It's pretty awkward having to write a regex that supports both forward and backwards slashes.

https://github.com/jamestalmage/ava/blob/172f6d8f298351dc78ca9b06ffce26ba6a80d468/package.json#L81-L84
https://github.com/bcoe/nyc/blob/4e9545a0c039032585149bf503c797f975e1e982/index.js#L28

Ideally, exclude would just support globbing.

Is there a way to have nyc use babel-istanbul or isparta under the hood to support covering ES5+ code?

there was a regression with the most recent patches that landed on master:

EDIT: looks like the remapping branch hasn't landed yet -- nothing is immediately jumping out at me as to why this is now cropping up.

.//node_modules/babel-cli/node_modules/convert-source-map/index.js:  // //# sourceMappingURL=foo.js.map                       /*# sourceMappingURL=foo.js.map */

Leading to:

[Error: An error occurred while trying to read the map file at /Users/benjamincoe/bcoe/yarsay/node_modules/babel-core/node_modules/convert-source-map/foo.js.map

Reproducing

This issue can be reproduced by running tests on this library:

https://github.com/bcoe/yarsay

CC: @novemberborn, @jamestalmage

In [email protected], I am getting the following error whenever --all is used:

user$nyc --all ava 'test/**/*.js'
fs.js:549
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: ENOENT: no such file or directory, open '/Users/user/project/.nyc_output/7910.json'
    at Error (native)
    at Object.fs.openSync (fs.js:549:18)
    at Object.fs.writeFileSync (fs.js:1156:15)
    at NYC.writeCoverageFile (/Users/user/project/node_modules/nyc/index.js:259:6)
    at NYC.addAllFiles (/Users/user/project/node_modules/nyc/index.js:171:8)
    at Object.<anonymous> (/Users/user/project/node_modules/nyc/bin/nyc.js:122:23)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)

Not sure where to start looking on this issue.

NYC now does 3 things:

1. Installs an (optionally cacheable) transform via append-transform that captures coverage and source-map data.
2. Remaps istanbul reports using source-map data.
3. Uses spawn-wrap to make all this happen across forked processes.

I think 1 and 2 need to be split out.

see https://github.com/jamestalmage/promisify-firebase

changing nothing but index.js and test.js to a minimal test case (i.e., not changing any of the build config / package.json / etc), I can get working coverage. This means that the code of my module itself is somehow disabling the coverage.

I've poked around trying to figure out exactly what line is causing it, so far no luck.

any help would be appreciated.

--all causes NYC to fail immediately for code with ES6 modules. I am using Node @5.1.0, nyc @5.0.0 with mocha and babel-register.

The error looks like:

/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:5703
            throw e;
                  ^
Error: Line 1: Unexpected token
    at constructError (/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:2405:21)
    at createError (/Users/joe/code/nyc-all-issue/node_modules/nyc/node_modules/istanbul/node_modules/esprima/esprima.js:2424:17)
...

.babelrc:

{
  presets: ["es2015"]
}

package.json:

{
  "scripts": {
    "test": "nyc --require babel-register mocha test.js",
    "test-all": "nyc --require babel-register --all mocha test.js",
    "run": "node test.js"
  },
  "dependencies": {
    "babel-core": "^6.3.17",
    "babel-preset-es2015": "^6.3.13",
    "babel-register": "^6.3.13",
    "mocha": "^2.3.4",
    "nyc": "^5.0.0"
  }
}

$ npm test succeeds and $ npm run test-all fails where test.js uses import, or requires a file that uses import.

https://github.com/jwhitfieldseed/nyc-issue-91 reproduces. It runs NYC with/out --all on code with/out import.

It seems other syntax that needs transpiling is OK with or without --all. I tested with rest args, which are not in Node 5.

with import

$ npm run test       // success
$ npm run test-all   // fails unexpectedly
$ npm run run        // SyntaxError, as expected - no rest args or import in Node 5

without import

$ npm run test       // success
$ npm run test-all   // success
$ npm run run        // SyntaxError, as expected - no rest args in Node 5

There are a few tests I think I did a poor adapting to fit our new testing strategy. @novemberborn had an alternate test strategy PR that we didn't go with, but I think it did adapt a few of the tests better.

We need to make sure we identify those and merge them in.

See:

• #85
• #101 (comment)

@novemberborn has done a great job of generalizing the logic around extending require hooks, I think this logic would be a great candidate for pulling into its own module -- with a thorough test suite.

CC: @jamestalmage, @ariporad

#108 (comment)

I think this could be an awesome general purpose module. My thoughts:

• It needs a simple config file that lists the projects that depend on the one you want to "smoke test".
• Mimic .travis.yml for configuring scripts (just the script names from the travis Build Lifecyle, not all their config options).
• The config file would need to point to the repository and branch to clone.
• The module would clone each individual downstream dependent in a temp directory and run the scripts described above, npm linking the module under test.
• If there are any failures, it will open the temp folder in your Finder/File Explorer, and list the failing dependents (maybe deleting the folders of passing dependents).

Folder structure:

some/temp/dir/
   downstream-dependent-1/
     ...
   downstream-dependent-2/
     ...

// @bcoe @novemberborn

I think we have a need for something like this for AVA as well.

// @sindresorhus @vdemedes

It would be interesting to profile how much time is spent applying the istanbul transform and consider caching results across processes. This would help with test suites that spawn/fork a lot. AVA's tests currently spawn about 70 times, each one loading up at least 3 files that need to be instrumented.

Based on work we've been doing on wrap-ansi, there are a couple documentation updates we should make:

• point out that nyc report only works if you have already run tests inside the same project directory.
• point out that COVERALLS_TOKEN is optional if the repo is open-source (verify this to be the case).

CC: @sindresorhus, @Qix-