devDependencies is not set on the object unless it's in package.json unlike dependencies and peerDependencies which is always present.

running read-installed with a stub package.json like {"name": "foobar"} shows this behavior.

It would be great if this was instead set to a empty object like the others for consistency and also ease of iterating over the dev-dependencies.

readdir-scoped-modules is still used by us, and has had a release in the last six months.

The release, however, hasn't been pushed to github. The github repo is, in fact, archived: https://github.com/npm/readdir-scoped-modules

Is the release expected? Where is the project developed these days?

(I was hoping to remove its very legacy debuglog implementation.)

Here's a quick reproduction:

https://gist.github.com/6cd7ee00d614f74876eb

If you switch to regular depend in the packages.json it works correctly.

Prior to 1.0.1, readdir-scoped-modules would throw EMFILE errors when reading large node_modules folders. Because read-installed assumes a readdir error simply implies an empty directory, this caused some installed modules to not be reported and lead to npm/npm#6506 (see this comment in particular). read-installed should either not suppress errors here or only suppress explicitly expected error types.

options.dev doesn't actually do anything at all.

contributes to the cause of npm/npm#5952

Installs and works just fine on 0.10.x and 0.11.x, but I can't get it to install on node 0.8.x

It seems to be an issue with installing util-extend@^1.0.1. This could be an npm issue - I can't see anything unusual in util-extend, which installs just fine in isolation.

Ian

227 error Error: No compatible version found: util-extend@'^1.0.1'
227 error Valid install targets:
227 error ["1.0.0","1.0.1"]
227 error     at installTargetsError (/Users/ian/.nvm/v0.8.26/lib/node_modules/npm/lib/cache.js:719:10)
227 error     at /Users/ian/.nvm/v0.8.26/lib/node_modules/npm/lib/cache.js:641:10
227 error     at saved (/Users/ian/.nvm/v0.8.26/lib/node_modules/npm/node_modules/npm-registry-client/lib/get.js:138:7)
227 error     at Object.oncomplete (fs.js:297:15)
228 error If you need help, you may report this log at:
228 error     <http://github.com/isaacs/npm/issues>
228 error or email it to:
228 error     <[email protected]>
229 error System Darwin 13.1.0
230 error command "/Users/ian/.nvm/v0.8.26/bin/node" "/Users/ian/.nvm/v0.8.26/bin/npm" "install" "read-installed"
231 error cwd /Users/ian/Personal/nlf
232 error node -v v0.8.26
233 error npm -v 1.2.30
234 verbose exit [ 1, true ]

Not sure why, but to me it seems like the depth option isn't working at all:

$ npm init
$ npm i -D read-installed
$ npm i react

// index.js

const readInstalled = require("read-installed");

let depth = 0;

const log = (pkg) => {
	console.log("-".repeat(depth) + " " + pkg.name + "@" + pkg.version);
	depth += 1;
	Object.keys(pkg.dependencies).forEach((name) => {
		log(pkg.dependencies[name]);
	});
	depth -= 1;
};

readInstalled(".", { depth: 0 }, (err, data) => log(data));

Yields the following result:

$ node index.js
 [email protected]
- [email protected]
-- [email protected]
--- [email protected]
-- [email protected]
-- [email protected]
--- [email protected]
---- [email protected]
--- [email protected]
--- [email protected]
-- [email protected]
--- [email protected]
---- [email protected]
// ... etc

Instead of what I would expect:

$ node index.js
 [email protected]
- [email protected]

Without and transient dependencies ... ?

• node 8.9.0, npm 5.5.1

See also #26 (comment), introduced by c46e5a8. Looking hopefully to @davglass for a fix.

We briefly discussed this at summer camp, but now that I'm actually getting around to it it doesn't seem as trivial as it did then.

The line that marks things as extraneous is line 190. But, in that context, we have access to obj (the package.json for the current package) and parent (the same, for the parent package). The problem is, given this structure:

npm-test-peer-deps
  dependencies: ["npm-test-peer-deps-file"]

npm-test-peer-deps-file
  peerDependencies: ["opener"]

we get the package layout

npm-test-peer-deps
  opener
  npm-test-peer-deps-file

So, when read-installed is inspecting "opener", it only has access to obj (its own package.json) and parent (that of "npm-test-peer-deps"). It doesn't know about the package.json for "npm-test-peer-deps-file".

The obvious solution is, if we mark ourself as extraneous, to go crawl parents' children and see if any of them have a peerDependency that would allow us to unmark ourself as extraneous.

I was about to do that, but wanted to make sure there wasn't something more clever you had in mind. For example, line 290 is part of a later tree-crawling operation that goes around marking stuff as non-extraneous, so maybe we could tap into that?

Here's the installation part of peerDependencies, by the way

Hello,

When running npm prune in my project, some packages are kept in node_modules that are no longer needed.
I traced the issue back to this module which gives extraneous:false for those installed dependencies even though they are not in the package.json.
Environment:
npm 1.3.24, windows 7 64bits

Reproduction:
from a newly created directory

$ npm init
$ npm install monitor
$ npm install monitor-dashboard
$ npm ls

The two modules should be marked as extraneous, but it's not the case.

Thanks

npm.commands.ls drops nested linked libraries when run programmatically. e.g.

module-a/package.json

{
    "name": "module-a",
    "dependencies": {
        "module-b" : "*",
    }
}

module-b/package.json

{
    "name": "module-b",
    "dependencies": {
        "module-c" : "*",
    }
}

module-c/package.json

{
    "name": "module-c",
}

npm.load({}, function() {
    npm.prefix = './work/module-a/';
    npm.commands.ls(undefined, true, function(err, full, lite) {
        console.log("Module A\n", lite);
        npm.prefix = './work/module-b'/;
        npm.commands.ls(undefined, true, function(err, full, lite) {
            console.log("Module B\n", lite);           
        });
    });
});

The output from the second 'ls' command does not list Module C as a dependency of Module B

The bug is in read_installed, and to do with the use of rpSeen. While traversing module-A's dependencies npm correctly detects that Module-B depends on Module-C, but after returning, this information is not present in the object stashed in rpSeen.

Excerpt of strack trace, from npm/npm#6984

900 http request GET https://registry.npmjs.org/rimraf
901 verbose type called_on_non_object
902 verbose stack TypeError: Object.keys called on non-object
902 verbose stack at Function.keys (native)
902 verbose stack at unmarkExtraneous (/Users/smikes/.nvm/v0.10.33/lib/node_modules/npm/node_modules/read-installed/read-installed.js:362:10)
902 verbose stack at /Users/smikes/.nvm/v0.10.33/lib/node_modules/npm/node_modules/read-installed/read-installed.js:137:5
902 verbose stack at next (/Users/smikes/.nvm/v0.10.33/lib/node_modules/npm/node_modules/read-installed/read-installed.js:185:14)
902 verbose stack at /Users/smikes/.nvm/v0.10.33/lib/node_modules/npm/node_modules/read-installed/read-installed.js:169:14
902 verbose stack at Object.oncomplete (evalmachine.:107:15)
903 verbose cwd /Users/smikes/src/github/npm

I will try to build a repro set.

As this module was updated to include the semver 3.x in the patch version 2.0.7 the currently stable version of npm, 1.4.21, will come to include semver 3.x when someone runs npm update -g rather than updating to semver 3.x as part of npm 2.0 as would otherwise be the case.

As I noted in my comment on the commit changing ^0.x.y to be equal to =0.x.y this is a rather big change that eg. breaks the peerDependencies of some widely used Grunt plugins like grunt-contrib-uglify and when peerDependencies are broken npm install will refuse to install anything and thus break npm in all projects currently using such dependencies.

I wouldn't expect such a big/breaking change without at least new major/minor version of npm – but since this change was introduced as a patch version in this package it will be picked up by the current npm 1.4 dependencies.

My suggestion would be to revert this change in a new 2.0.8 release and reintroduce it in either a 2.1.0 release or in a 3.0.0 release.

mkdir read-installed-bug && cd read-installed-bug
npm init -f && npm install read-installed --save

index.js

var depth = parseInt(process.argv[2], 10) || 0

read(process.cwd(), {depth: depth, dev: false}, function(err, installed) {
  if (err) throw err
  console.log(inspect(installed, {colors: true, depth: 30}))
})

Now run this with node index.js 0 to print depth 0 and you should get something like the output at the bottom of this issue.

Note that read-installed is reporting that it is a dependency of itself:

{
  name: 'read-installed-bug',
  …
  dependencies: {
     'read-installed': {
        name: 'read-installed',
        …
        dependencies: {
          'read-installed': [Circular]
        }
      }
   }
}

If you adjust the depth (node index.js 1, node index.js 2) this same pattern occurs in all dependencies at the maximum depth.

Is this somehow a feature?

{ name: 'read-installed-bug',
  version: '1.0.0',
  main: 'index.js',
  scripts: { test: 'echo "Error: no test specified" && exit 1' },
  keywords: [],
  author:
   { name: 'Tim Oxley',
     email: '[email protected]' },
  license: 'ISC',
  dependencies:
   { 'read-installed':
      { name: 'read-installed',
        description: 'Read all the installed packages in a folder, and return a tree structure with all the data.',
        version: '3.1.5',
        repository:
         { type: 'git',
           url: 'git://github.com/isaacs/read-installed' },
        main: 'read-installed.js',
        scripts: { test: 'tap ./test/*.js' },
        dependencies: { 'read-installed': [Circular] }, //  <== ????
        optionalDependencies: { 'graceful-fs': '2 || 3' },
        author:
         { name: 'Isaac Z. Schlueter',
           email: '[email protected]',
           url: 'http://blog.izs.me/' },
        license: 'ISC',
        devDependencies:
         { mkdirp: '^0.5.0',
           rimraf: '^2.2.8',
           tap: '~0.4.8' },
        gitHead: '577c3f3f4f1e435f9bd944b8f99ce3f7552709ef',
        bugs: { url: 'https://github.com/isaacs/read-installed/issues' },
        homepage: 'https://github.com/isaacs/read-installed',
        _id: '[email protected]',
        _shasum: '4ae36081afd3e2204dc2e279807aaa52c30c8c0c',
        _from: 'read-installed@>=3.1.5 <4.0.0',
        _npmVersion: '2.1.15',
        _nodeVersion: '0.10.34',
        _npmUser:
         { name: 'othiym23',
           email: '[email protected]' },
        maintainers: [ [Object], [Object] ],
        dist:
         { shasum: '4ae36081afd3e2204dc2e279807aaa52c30c8c0c',
           tarball: 'http://registry.npmjs.org/read-installed/-/read-installed-3.1.5.tgz' },
        directories: {},
        _resolved: 'https://registry.npmjs.org/read-installed/-/read-installed-3.1.5.tgz',
        readme: 'ERROR: No README data found!',
        realName: 'read-installed',
        _dependencies:
         { debuglog: '^1.0.1',
           'read-package-json': '1',
           'readdir-scoped-modules': '^1.0.0',
           semver: '2 || 3 || 4',
           slide: '~1.1.3',
           'util-extend': '^1.0.1',
           'graceful-fs': '2 || 3' },
        extraneous: false,
        path: '/Users/timoxley/Projects/tests/read-installed-bug/node_modules/read-installed',
        realPath: '/Users/timoxley/Projects/tests/read-installed-bug/node_modules/read-installed',
        link: undefined,
        parent: [Circular],
        depth: 1,
        peerDependencies: {} } },
  devDependencies: {},
  readme: 'ERROR: No README data found!',
  _id: '[email protected]',
  realName: 'read-installed-bug',
  _dependencies: { 'read-installed': '^3.1.5' },
  extraneous: false,
  path: '/Users/timoxley/Projects/tests/read-installed-bug',
  realPath: '/Users/timoxley/Projects/tests/read-installed-bug',
  link: undefined,
  depth: 0,
  peerDependencies: {},
  root: true }

Hello there.

I'd found memory leak in the var riSeen.

After 10-50 invoking of readInstalled var riSeen becomes very huge array.
Do we can to clean up it before each invoking of readInstalled?

See isaacs/npm#2553 for background.

As listed in TODOs in read-installed.js. Also see See isaacs/npm#2553 for background.

Please add a license file to clarify what license this is distributed under.

I'd like to use node/npm for a project at work, but npm depends on this package and I'm not allowed to use software that doesn't have a clear license.

Thanks!

When there is no package.json file present (for example, in the parent directory of the top-level node_modules directory), this code reads dependencies from the node_modules directory and attempts to use the catch-all version constraint, *.

Unfortunately, * no longer matches prerelease versions, after this commit in v4.3.6 of the semver package, which is what npm uses.

This effectively means that prerelease packages installed in top-level node_modules directories will be considered invalid by commands like npm ls.

I think the simplest way to fix this would be to make this condition fail when reqver === "*" (or, better, some sentinel value that cannot be confused for a version string), so that top-level prerelease packages are not marked invalid.

I'll submit a PR with that fix shortly.

What / Why

While scanning my project with auditjs, I discovered read-installed has a transitive dependency on [email protected] which has vulnerability CVE-2021-23362.

$ npm ls hosted-git-info
  [email protected] /Users/bhamail/sonatype/community/auditjs/auditjs
  └─┬ [email protected]
    └─┬ [email protected]
      └─┬ [email protected]
        └── [email protected]

Is it possible to have read-installed utilize NODE_PATH to mimic require's functionality in finding dependencies?

http://nodejs.org/api/modules.html#modules_loading_from_the_global_folders