Giter VIP home page Giter VIP logo

broccoli-funnel's Introduction

Broccoli Funnel

Build Status Build status

Given an input node, the Broccoli Funnel plugin returns a new node with only a subset of the files from the input node. The files can be moved to different paths. You can use regular expressions to select which files to include or exclude.

Documentation

funnel(inputNode, options)

inputNode {Single node}

A Broccoli node (formerly: "tree"). A node in Broccoli can be either a string that references a directory in your project or a node object returned from running another Broccoli plugin.

If your project has the following file structure:

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select a subsection of the tree via Funnel:

const funnel = require('broccoli-funnel');
const cssFiles = funnel('src/css');

/*
  cssFiles contains the following files:

  ├── reset.css
  └── todos.css
*/

// export the node for Broccoli to begin processing
module.exports = cssFiles;

Options

srcDir {String}

A string representing the portion of the input node to start the funneling from. This will be the base path for any include/exclude regexps.

Default: '.', the root path of the input node.

If your project has the following file structure:

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select a subsection of the node via Funnel:

const funnel = require('broccoli-funnel');
const merge = require('broccoli-merge-trees');

// root of our source files
const projectFiles = 'src';

/* get a new node of only files in the 'src/css' directory
  cssFiles contains the following files:

  ├── reset.css
  └── todos.css
*/
const cssFiles = funnel(projectFiles, {
  srcDir: 'css'
});

/* get a new node of only files in the 'src/icons' directory
  imageFiles contains the following files:

  ├── check-mark.png
  └── logo.jpg
*/
const imageFiles = funnel(projectFiles, {
  srcDir: 'icons'
});


module.exports = merge([cssFiles, imageFiles]);

destDir {String}

A string representing the destination path that filtered files will be copied to.

Default: '.', the root path of input node.

If your project has the following file structure:

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select a subsection of the directory structure via Funnel and copy it to a new location:

const funnel = require('broccoli-funnel');

const cssFiles = funnel('src/css', {
  destDir: 'build'
});

/*
  cssFiles contains the following files:

  build/
  ├── reset.css
  └── todos.css
*/

module.exports = cssFiles;

allowEmpty {Boolean}

When using srcDir/destDir options only (aka no filtering via include/exclude options), if the srcDir were missing an error would be thrown. Setting allowEmpty to true, will prevent that error by creating an empty directory at the destination path.


include {Array of GlobStrings|RegExps|Functions}

One or more matcher expression (regular expression, glob string, or function). Files within the node whose names match this expression will be copied (with the location inside their parent directories preserved) to the destDir.

Default: [].

If your project has the following file structure

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select files that match a glob expression and copy those subdirectories to a new location, preserving their location within parent directories:

const funnel = require('broccoli-funnel');

// finds all files that match /todo/ and moves them
// the destDir
const todoRelatedFiles = funnel('src', {
  include: ['**/todo*']
});

/*
  todoRelatedFiles contains the following files:
  .
  ├── css
  │   └── todos.css
  └── javascript
      └── todo.js
*/

module.exports = todoRelatedFiles;

exclude {Array of Glob Strings|Function}

One or more matcher expression (regular expression, glob string, or function). Files within the node whose names match this expression will not be copied to the destDir if they otherwise would have been.

Note, in the case when a file matches both an include and exclude pattern, the exclude pattern wins

Default: [].

If your project has the following file structure:

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select files that match a glob expression and exclude them from copying:

const funnel = require('broccoli-funnel');

// finds all files in 'src' EXCEPT `todo.js` in any directory
// or sub-directory and adds them to a node.
const nobodyLikesTodosAnyway = funnel('src', {
  exclude: ['**/todo.js']
});

/*
  nobodyLikesTodosAnyway contains the following files:
  .
  ├── css
  │   └── reset.css
  ├── icons
  │   ├── check-mark.png
  │   └── logo.jpg
  └── javascript
      └── app.js
*/

module.exports = nobodyLikesTodosAnyway;

files {Array of Strings}

One or more relative file paths. Files within the node whose relative paths match will be copied (with the location inside their parent directories preserved) to the destDir.

Default: [].

If your project has the following file structure

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    ├── icons/
    │   ├── check-mark.png
    │   └── logo.jpg
    └── javascript/
        ├── app.js
        └── todo.js

You can select a specific list of files copy those subdirectories to a new location, preserving their location within parent directories:

const funnel = require('broccoli-funnel');

// finds these specific files and moves them to the destDir
const someFiles = funnel('src', {
  files: ['css/reset.css', 'icons/check-mark.png']
});

/*
  someFiles contains the following files:
  .
  ├── css
  │   └── reset.css
  └── icons
      └── check-mark.png
*/

module.exports = someFiles;

getDestinationPath {Function}

This method will get called for each file, receiving the currently processing relativePath as its first argument. The value returned from getDestinationPath will be used as the destination for the new node. This is a very simple way to rename files or move them from one path to another (replacing the need for broccoli-file-mover for example).

The return value of this method is cached for each input file. This means that getDestinationPath will only be called once per relativePath.

In the following example, getDestinationPath is used to move main.js to ember-metal.js:

const node = funnel('packages/ember-metal/lib', {
  destDir: 'ember-metal',

  getDestinationPath: function(relativePath) {
    if (relativePath === 'lib/main.js') {
      return 'ember-metal.js';
    }

    return relativePath;
  }
});

Extending Funnel

If you desire to extend funnel follow the below snippet

const { Funnel } = require('broccoli-funnel');
class CustomFunnel extends Funnel {
  // cutstom implementation
}

ZOMG!!! TESTS?!?!!?

I know, right?

Running the tests:

npm install
npm test

License

This project is distributed under the MIT license.

broccoli-funnel's People

Contributors

caleb avatar chadhietala avatar chriskrycho avatar davewasmer avatar dependabot[bot] avatar ef4 avatar gaurav0 avatar hjdivad avatar igorminar avatar joliss avatar jrowlingson avatar krisselden avatar locks avatar lolmaus avatar nathanhammond avatar raytiley avatar reidab avatar rwjblue avatar sparshithnr avatar stefanpenner avatar timmfin avatar trek avatar twokul avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

broccoli-funnel's Issues

funnel is copying the entire folder instead of a single file

Hello,

I'm using the latest version of broccoli funnel 2.0.1, along with 1.2.0, which is being used by babel-transpiler:

├─┬ [email protected]
│ └── [email protected]

I am having problems selecting single files. This is my configuration:

const info = new Funnel(appRoot, {
    files: ["plugin.info", 'styles/tiddlywiki.files'],
    annotation: "Metadata files",
});

As you can see, I only want to copy two files, however funnel is copying all the files that are on the folder, producing big problems to me. One of the problems is that I can not merge trees unless I specify the option overwrite to true.
Is this a bug with the latest version or may be a problem with babel-transpiler ?

Here is how I have the merge trees, and js configured:

const info = new Funnel(appRoot, {
    files: ["plugin.info", 'styles/tiddlywiki.files'],
    annotation: "Metadata files",
});

js = uglify(js, {
    annotation: 'Uglify js files',
    uglify: {
        warnings: true,
        sourceMap: !process.env.PRODUCTION,
        output: { comments: 'some' }
    }
});

module.exports = new Merge([js,  info], { overwrite: true, annotation: "Final output" });

Thanks in advance

error when you specify root directory containing tmp directory

/myproject
   /tmp                // broccoli temporary folder
   Brocfile.js

Brocfile.js

var Funnel = require('broccoli-funnel');

module.exports = new Funnel(__dirname, {
  include: ['**/*.d.ts'],
  exclude: ['tmp'],
  destDir: 'out'
});
$cd myproject
$broccoli serve
Serving on http://localhost:4200

Built with error:
Error: ELOOP: too many symbolic links encountered, stat '/home/sergey/Projects/client/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0/tmp/funnel-input_base_path-qEa9xQgy.tmp/0'
    at Error (native)
    at Object.fs.statSync (fs.js:892:18)
    at getStat (/home/sergey/Projects/client/node_modules/walk-sync/index.js:108:15)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:70:17)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)
    at _walkSync (/home/sergey/Projects/client/node_modules/walk-sync/index.js:76:32)

broccoli serve with mergeTrees runs out of memory

When using a mergeTrees (v1.1.1) of funnels (v1.0.0 or v1.0.1), broccoli serve runs out of memory and crashes.

Simplest Brocfile.js:

var funnel = require('broccoli-funnel');
var mergeTrees = require('broccoli-merge-trees');

module.exports = mergeTrees([
  funnel('.', { include: [ 'tests/**/*.js' ] })
]);

When I run broccoli build dist, everything works fine. When I run broccoli serve, I get

Serving on http://localhost:4200

Warning: failed to stat tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/node_modules/broccoli-babel-transpiler/node_modules/babel-core/node_modules/regenerator/node_modules/defs/node_modules/yargs/node_modules/cliui/node_modules/center-align/node_modules/align-text/node_modules/kind-of/node_modules

<--- Last few GCs --->

   20320 ms: Scavenge 1402.9 (1457.7) -> 1402.9 (1457.7) MB, 9.3 / 0 ms (+ 34.5 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
   21740 ms: Mark-sweep 1402.9 (1457.7) -> 1401.6 (1457.7) MB, 1420.3 / 0 ms (+ 295.9 ms in 1199 steps since start of marking, biggest step 34.5 ms) [last resort gc].
   23137 ms: Mark-sweep 1401.6 (1457.7) -> 1401.5 (1457.7) MB, 1397.5 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x43f935e3ac1 <JS Object>
    2: keysForTree(aka keysForTree) [/Users/jamesarosen/Desktop/graphs/node_modules/broccoli/node_modules/broccoli-kitchen-sink-helpers/index.js:49] [pc=0x1d274e92b223] (this=0x43f93504189 <undefined>,fullPath=0x6d8b44e2d19 <String[983]: tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv1.tmp/0/tmp/funnel-input_base_path-DF6uadv...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

The warning seems to have some sort of infinite loop in the path, which suggests a reason for the out-of-memory error.

I've tried this on node v0.12.0 and v5.7.1.

[3.0.0] Leading / in destDir is treated as ROOT directory

As of v3.0.0, a leading slash in the destDir parameter is now treated as the root directory. In the 2.x series, destDir was sanitized through path.join, but the new approach in 3.x doesn't do this.

In 2.x, it was:

    let namespacedTree = new Funnel(tree, {
      srcDir: '/',
      destDir: `/${this.moduleName()}`,
    });

In 3.x, it needs to be:

    let namespacedTree = new Funnel(tree, {
      srcDir: '/',
      destDir: `${this.moduleName()}`,
    });

Here's a real-world example of this manifesting:

Build Error (Funnel)
EROFS: read-only file system, symlink '/Users/jhawker/Projects/GitHub/ember-sinon-qunit/addon-test-support' -> '/ember-sinon-qunit'

elwayman02/ember-sinon-qunit#248
https://travis-ci.org/elwayman02/ember-sinon-qunit/jobs/655610389

regression in 1.2

It was a large code change, but it appears to have some issues. Still need to narrow it down, but my guess is an issue with my cache invalidation code.

Plan is to release 1.2.1 that is 1.1 code

Recursively walk directory

.
├── Brocfile.js
└── src/
    ├── css/
    │   ├── reset.css
    │   └── todos.css
    │   ├── morecss/
          │   ├── reset.css
          │   └── todos.css

new Funnel('src/css') does not walk morecss.

Broccoli-funnel - no such file or directory, symlink

Very new to Ember & aws world. I'm trying to deploy a simple ember app to ElasticBeanstalk and I'm having the following problem on Win 10 machine. Any help would be appreciated.

C:\Users\eralgan\Desktop\ElasticBeanstalk\todo>ember deploy production
version: 2.4.2

  • build failed
    Error: The Broccoli Plugin: [Funnel] failed with:
    Error: ENOENT: no such file or directory, symlink 'C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-ajax\addon' -> 'C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\tmp\funnel-input_base_path-OmJjrelm.tmp\0'
    at Error (native)
    at Object.fs.symlinkSync (fs.js:897:18)
    at symlinkWindows (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\symlink-or-copy\index.js:90:16)
    at symlinkOrCopySync (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\symlink-or-copy\index.js:56:5)
    at C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\broccoli-plugin\read_compat.js:58:9
    at lib$rsvp$$internal$$tryCatch (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\rsvp\dist\rsvp.js:1036:16)
    at lib$rsvp$$internal$$invokeCallback (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\rsvp\dist\rsvp.js:1048:17)
    at lib$rsvp$$internal$$publish (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\rsvp\dist\rsvp.js:1019:11)
    at lib$rsvp$asap$$flush (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\rsvp\dist\rsvp.js:1198:9)
    at nextTickCallbackWith0Args (node.js:420:9)

The broccoli plugin was instantiated at:
at Funnel.Plugin (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\broccoli-plugin\index.js:7:31)
at new Funnel (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\node_modules\broccoli-funnel\index.js:44:10)
at Class.Addon.addonJsFiles (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:692:30)
at Class.Addon.processedAddonJsFiles (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:722:33)
at Class.Addon.compileAddon (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:632:22)
at Class.Addon.treeForAddon (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:403:24)
at Class._treeFor (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:322:31)
at Class.treeFor (C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\models\addon.js:290:19)
at C:\Users\eralgan\Desktop\ElasticBeanstalk\todo\node_modules\ember-cli\lib\broccoli\ember-app.js:446:20
at Array.map (native)

Pipeline aborted

Consider a rename of the plugin/project to more accurately reflect what it does.

Reading through the documentation, this plugin doesn't seem to function as a funnel at all. It may make sense to rename to more accurately reflect it's purpose. Some more accurate names might be sieve, strainer, or colander. Personally, I prefer colander, if only for the visual of broccoli being rinsed and drained. (how is there no animated gif for this?) Though, I'm not sure how playful the broccoli community is with their naming.

broccoli-colander

Obviously, renaming a project has some potentially negative side-effects as the user base adjusts to the change. However, supporting both packages and APIs for a short deprecation period shouldn't be too much overhead.

Please change examples to `funnel`, not `new Funnel`

In short, documenting Funnel as a class is leading some devs to treat Funnel differently than other Broccoli and even Node modules. Look how Funnel sticks out like a sore thumb:

var mergeTrees = require('broccoli-merge-trees'),
    Funnel = require('broccoli-funnel'),
    concat = require('broccoli-concat'),
    path = require('path'),
    find = require('lodash/collection/find'),
    fs = require('fs');

If I made a brief PR to change example usages in README, would you accept it?

srcDir can no longer contain a leading slash

In broccoli-funnel 2.02, the following worked:

      let node = new Funnel('.', {
        srcDir: '/',
        destDir: 'foo',
      });

However, in 3.0.2, it fails with:

Relative path is expected, path / is an absolute path. inputPath gets prefixed to the reltivePath provided.

If this change is intended, it should be documented in changelog as a breaking change.

Pass the changes to out when linking roots

When linking source root (this.in[0]) to destination root (this.out) directly (include, exclude, files, destinationPath are all empty), should be able to pass the _changes and entries of this.in[0] to this.out.

globbing support?

Hi,
is it possible to provide wildcards / globbing when using the files option?
For example:
var someFiles = funnel('src', {
files: ['*.html']
});
Thanks!

Dramatically slow rebuild times against a single file touch

I am currently profiling / adding temporal instrumentation debug console logs to several broccoli plugins in my project in order to find the cause of horribly slow rebuild times (2minutes).

I am particularly looking at 20s to 35s seconds in between handling input trees (adding up to a 1m of the build process). The suspects are:

  • Funnel.prototype.read
  • Funnel.prototype.handleReadTree
  • Funnel.prototype.cleanup.

What I am trying to understand is why does Broccoli use up to 30s in average to process input trees when I touch just a single file in my app? 🆘

[Regression 0.2.4] Mixing RegExps and strings in include array is broken

Hey,

since 0.2.4 it's not possible anymore to mix strings and regexps in the include option:

var Funnel = require('broccoli-funnel');

module.exports = new Funnel('foobar/', {
  include:  [
    'index.html',
    /\.js$/,
    /\.map$/,
    /\.css$/
  ]
});

This code works in 0.2.3 but fails in 0.2.4 with this error:

TypeError: undefined is not a function
    at MatcherCollection.match (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/walk-sync-matcher/may-match.js:13:26)
    at MatcherCollection.mayContain (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/walk-sync-matcher/may-match.js:38:12)
    at walkSync (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/walk-sync-matcher/index.js:19:15)
    at Funnel.processFilters (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/index.js:125:15)
    at Funnel.build (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/index.js:114:10)
    at /home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/broccoli-plugin/read_compat.js:61:34
    at lib$rsvp$$internal$$tryCatch (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/broccoli-plugin/node_modules/promise-map-series/node_modules/rsvp/dist/rsvp.js:493:16)
    at lib$rsvp$$internal$$invokeCallback (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/broccoli-plugin/node_modules/promise-map-series/node_modules/rsvp/dist/rsvp.js:505:17)
    at lib$rsvp$$internal$$publish (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/broccoli-plugin/node_modules/promise-map-series/node_modules/rsvp/dist/rsvp.js:476:11)
    at lib$rsvp$asap$$flush (/home/domme/dev/sources/foobar/node_modules/broccoli-funnel/node_modules/broccoli-plugin/node_modules/promise-map-series/node_modules/rsvp/dist/rsvp.js:1198:9)

Build failed

BR,
Domme

[3.0.1] Relative path is expected, path / is an absolute path.

Doesn't happen on 3.0.0.

ERROR Summary:

  - broccoliBuilderErrorStack: Error: Relative path is expected, path / is an absolute path.
    at Object.handleFs (/home/boris/project/node_modules/broccoli-funnel/node_modules/broccoli-output-wrapper/dist/index.js:27:15)
    at Funnel._copy (/home/boris/project/node_modules/broccoli-funnel/index.js:475:19)
    at Funnel.build (/home/boris/project/node_modules/broccoli-funnel/index.js:235:16)
    at /home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/wrappers/transform-node.js:68:45
    at new Promise (<anonymous>:null:null)
    at TransformNodeWrapper.build (/home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/wrappers/transform-node.js:53:16)
    at /home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/builder.js:99:36

  - code: [undefined]
  - codeFrame: Relative path is expected, path / is an absolute path.
  - errorMessage: Relative path is expected, path / is an absolute path.
        at Funnel
-~- created here: -~-
    at new Plugin (/home/boris/project/node_modules/broccoli-funnel/node_modules/broccoli-plugin/dist/index.js:45:33)
    at new Funnel (/home/boris/project/node_modules/broccoli-funnel/index.js:48:5)
    at new funnel (/home/boris/project/node_modules/broccoli-funnel/index.js:498:10)
    at module.exports (/home/boris/project/ember-cli-build.js:291:5)
    at Builder.readBuildFile (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:52:14)
    at Builder.setupBroccoliBuilder (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:66:22)
    at new Builder (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:32:10)
    at BuildTask.run (/home/boris/project/node_modules/ember-cli/lib/tasks/build.js:15:19)
    at /home/boris/project/node_modules/ember-cli/lib/models/command.js:239:24
    at tryCatcher (/home/boris/project/node_modules/rsvp/dist/rsvp.js:326:21)
    at invokeCallback (/home/boris/project/node_modules/rsvp/dist/rsvp.js:498:33)
    at /home/boris/project/node_modules/rsvp/dist/rsvp.js:562:16
    at flush (/home/boris/project/node_modules/rsvp/dist/rsvp.js:2441:7)
    at processTicksAndRejections (internal/process/task_queues.js:79:11)

-~- (end) -~-
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: [undefined]
    - line: [undefined]
    - treeDir: [undefined]
  - message: Relative path is expected, path / is an absolute path.
        at Funnel
-~- created here: -~-
    at new Plugin (/home/boris/project/node_modules/broccoli-funnel/node_modules/broccoli-plugin/dist/index.js:45:33)
    at new Funnel (/home/boris/project/node_modules/broccoli-funnel/index.js:48:5)
    at new funnel (/home/boris/project/node_modules/broccoli-funnel/index.js:498:10)
    at module.exports (/home/boris/project/ember-cli-build.js:291:5)
    at Builder.readBuildFile (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:52:14)
    at Builder.setupBroccoliBuilder (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:66:22)
    at new Builder (/home/boris/project/node_modules/ember-cli/lib/models/builder.js:32:10)
    at BuildTask.run (/home/boris/project/node_modules/ember-cli/lib/tasks/build.js:15:19)
    at /home/boris/project/node_modules/ember-cli/lib/models/command.js:239:24
    at tryCatcher (/home/boris/project/node_modules/rsvp/dist/rsvp.js:326:21)
    at invokeCallback (/home/boris/project/node_modules/rsvp/dist/rsvp.js:498:33)
    at /home/boris/project/node_modules/rsvp/dist/rsvp.js:562:16
    at flush (/home/boris/project/node_modules/rsvp/dist/rsvp.js:2441:7)
    at processTicksAndRejections (internal/process/task_queues.js:79:11)

-~- (end) -~-
  - name: BuildError
  - nodeAnnotation: [undefined]
  - nodeName: Funnel
  - originalErrorMessage: Relative path is expected, path / is an absolute path.
  - stack: Error: Relative path is expected, path / is an absolute path.
    at Object.handleFs (/home/boris/project/node_modules/broccoli-funnel/node_modules/broccoli-output-wrapper/dist/index.js:27:15)
    at Funnel._copy (/home/boris/project/node_modules/broccoli-funnel/index.js:475:19)
    at Funnel.build (/home/boris/project/node_modules/broccoli-funnel/index.js:235:16)
    at /home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/wrappers/transform-node.js:68:45
    at new Promise (<anonymous>:null:null)
    at TransformNodeWrapper.build (/home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/wrappers/transform-node.js:53:16)
    at /home/boris/project/node_modules/ember-cli/node_modules/broccoli/dist/builder.js:99:36

Windows issues

It appears ember-cli app on an NTFS mounted using [email protected] appears to be running into issues. It is possible the symlink detection (are symlinking available vs not isn't quite working as expected).

ember-cli: 2.6.2
node: 6.3.0
os: win32 x64
ember-cli-d2 1.1.7
 The Broccoli Plugin: [Funnel] failed with:
Error: ENOENT: no such file or directory, open 'E:\Work\BitBucket\New folder\tmp\funnel-output_path-W6hyqoN3.tmp\d3-plugins-dist\adnan-wahab\pathgl\named-amd\main.js'
    at Error (native)
    at Object.fs.openSync (fs.js:640:18)
    at Object.fs.writeFileSync (fs.js:1333:33)
    at symlinkWindows (E:\Work\BitBucket\New folder\node_modules\symlink-or-copy\index.js:106:18)
    at Function.symlinkOrCopySync [as sync] (E:\Work\BitBucket\New folder\node_modules\symlink-or-copy\index.js:53:5)
    at Funnel._copy (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:393:19)
    at Funnel.processFile (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:376:8)
    at Funnel.applyPatch [as _applyPatch] (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:293:12)
    at Funnel.<anonymous> (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:248:10)
    at Array.forEach (native)

The broccoli plugin was instantiated at:
    at Funnel.Plugin (E:\Work\BitBucket\New folder\node_modules\broccoli-plugin\index.js:7:31)
    at new Funnel (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:44:10)
    at Funnel (E:\Work\BitBucket\New folder\node_modules\broccoli-funnel\index.js:42:43)
    at CoreObject.module.exports.treeForVendor (E:\Work\BitBucket\New folder\node_modules\ember-cli-d3\index.js:36:43)
    at CoreObject._treeFor (E:\Work\BitBucket\New folder\node_modules\ember-cli\lib\models\addon.js:358:33)
    at CoreObject.treeFor (E:\Work\BitBucket\New folder\node_modules\ember-cli\lib\models\addon.js:326:21)
    at E:\Work\BitBucket\New folder\node_modules\ember-cli\lib\broccoli\ember-app.js:465:20
    at Array.map (native)
    at EmberApp.addonTreesFor (E:\Work\BitBucket\New folder\node_modules\ember-cli\lib\broccoli\ember-app.js:463:30)
    at EmberApp._processedVendorTree (E:\Work\BitBucket\New folder\node_modules\ember-cli\lib\broccoli\ember-app.js:919:29) 

Run time error: Merge error: file already exists, pass option { overwrite: true }

I set up funnel in ember-cli-build.js to copy a translations file into dist.
It worked, but after I left to another git branch and came back, the builds fail with the following error;

Merge error: file translations/en-us.json exists in C:\Users\john.doe\AppData\Local\Temp\broccoli-19700SykRagiBUk5I\out-272-broccoli_merge_trees_application_dist and C:\Users\john.doe\AppData\Local\Temp\broccoli-19700SykRagiBUk5I\out-274-funnel
Pass option { overwrite: true } to mergeTrees in order to have the latter file win.

Here is the relevant code:

	const extraAssets = new Funnel('translations', {
		destDir: '/translations'
	});

	return app.toTree(extraAssets);

Regression in release 0.2.6

Broccoli funnel filters out files in subdirs that would have matched the include filters in earlier versions.

    var tree = new Funnel(iconsPath, {
      include: [ '*.scss' ],
    });
  readdirSync(iconsPath); // returns ['icons.scss', ...]

Seems to be ruining images?

var assets = [
  "source/background.jpg",
  "source/bad_feed.gif",
  "source/gun_metal.png",
  "source/purty_wood.png"
]
assets = BroccoliFunnel(source, {
  files: assets,
  srcDir: ".",
  destDir: ".",
  getDestinationPath: function(relativePath) {
    if (relativePath.indexOf("source/") > -1) {
      return relativePath.replace("source/", "");
    } else {
      return relativePath;
    }
  }
});
module.exports = new BroccoliMergeTrees([assets]);

Funnel with missing inputNode fails if include and/or exclude are set

Creating a Funnel with an inputNode pointing to a directory that doesn't exist on the file system, and with the include or exclude options are set, causes processFilters to run whether or not the inputPath derived from that inputNode exists on the system:

broccoli-funnel/index.js

Lines 258 to 260 in bfea8d4

} else {
this.processFilters(inputPath);
}

This in turn always invokes walkSync.entries with the specified inputPath:

broccoli-funnel/index.js

Lines 320 to 324 in bfea8d4

if (this._matchedWalk) {
entries = walkSync.entries(inputPath, { globs: this.include, ignore: this.exclude });
} else {
entries = walkSync.entries(inputPath);
}

The result is an error like broccoliBuilderErrorStack: Error: ENOENT: no such file or directory, scandir '<bad path>'.

full stack trace
=================================================================================

ENV Summary:

  TIME: Tue Jan 22 2019 13:04:24 GMT-0700 (MST)
  TITLE: ember
  ARGV:
  - /Users/chris/.nodenv/versions/8.11.4/bin/node
  - /Users/chris/.nodenv/versions/8.11.4/bin/ember
  - s
  EXEC_PATH: /Users/chris/.nodenv/versions/8.11.4/bin/node
  TMPDIR: /var/folders/l2/3r6d4lps0w7gnxf95w0kgfxw0000gn/T
  SHELL: /usr/local/bin/fish
  PATH:
  - /Users/chris/.nodenv/versions/8.11.4/bin
  - /usr/local/Cellar/nodenv/1.2.0/libexec
  - /Users/chris/.nodenv/shims
  - /Users/chris/bin
  - /Users/chris/.cargo/bin
  - /usr/local/miniconda3/bin
  - /usr/local/bin
  - /usr/bin
  - /bin
  - /usr/sbin
  - /sbin
  - /usr/local/share/dotnet
  - ~/.dotnet/tools
  - /usr/local/MacGPG2/bin
  PLATFORM: darwin x64
  FREEMEM: 131371008
  TOTALMEM: 17179869184
  UPTIME: 597434
  LOADAVG: 6.083984375,3.72802734375,2.65869140625
  CPUS:
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  - Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz - 2800
  ENDIANNESS: LE
  VERSIONS:
  - ares: 1.10.1-DEV
  - cldr: 32.0
  - http_parser: 2.8.0
  - icu: 60.1
  - modules: 57
  - napi: 3
  - nghttp2: 1.32.0
  - node: 8.11.4
  - openssl: 1.0.2p
  - tz: 2017c
  - unicode: 10.0
  - uv: 1.19.1
  - v8: 6.2.414.54
  - zlib: 1.2.11

ERROR Summary:

  - broccoliBuilderErrorStack: Error: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
    at Object.fs.readdirSync (fs.js:904:18)
    at _walkSync (/Users/chris/dev/oss/ember/freestyle/node_modules/walk-sync/index.js:74:18)
    at Function.entries (/Users/chris/dev/oss/ember/freestyle/node_modules/walk-sync/index.js:50:10)
    at Funnel.processFilters (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:320:26)
    at Funnel.build (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:258:10)
    at resolve (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/wrappers/transform-node.js:29:35)
    at new Promise (<anonymous>)
    at TransformNodeWrapper.build (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/wrappers/transform-node.js:21:12)
    at Promise.resolve.then.then.then (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/builder.js:111:28)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)
  - codeFrame: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
  - errorMessage: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
        at Funnel
-~- created here: -~-
    at Funnel.Plugin (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-plugin/index.js:7:31)
    at new Funnel (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:58:10)
    at SASSPlugin.toTree (/Users/chris/dev/oss/ember/freestyle/node_modules/ember-cli-sass/index.js:42:16)
    at /Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:180:26
    at Array.forEach (<anonymous>)
    at processPlugins (/Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:178:11)
    at module.exports.preprocessCss (/Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:148:10)
    at Class.compileStyles (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:981:33)
    at Class.treeForAddon (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:860:27)
    at Class._treeFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:616:33)
    at Class.treeFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:576:21)
    at project.addons.reduce (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:699:26)
    at Array.reduce (<anonymous>)
    at EmberApp._addonTreesFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:697:32)
    at EmberApp._addonBundles (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1160:31)
    at EmberApp.createAddonTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1193:29)
    at EmberApp.addonTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1208:36)
    at EmberApp.getExternalTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1264:25)
    at EmberApp.toArray (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1619:12)
    at EmberApp.toTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1747:22)
    at module.exports (/Users/chris/dev/olo/web-client/ember-cli-build.js:88:14)
    at Builder.readBuildFile (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/builder.js:52:14)
-~- (end) -~-
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: [undefined]
    - line: [undefined]
    - treeDir: [undefined]
  - message: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
        at Funnel
-~- created here: -~-
    at Funnel.Plugin (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-plugin/index.js:7:31)
    at new Funnel (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:58:10)
    at SASSPlugin.toTree (/Users/chris/dev/oss/ember/freestyle/node_modules/ember-cli-sass/index.js:42:16)
    at /Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:180:26
    at Array.forEach (<anonymous>)
    at processPlugins (/Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:178:11)
    at module.exports.preprocessCss (/Users/chris/dev/olo/web-client/node_modules/ember-cli-preprocess-registry/preprocessors.js:148:10)
    at Class.compileStyles (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:981:33)
    at Class.treeForAddon (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:860:27)
    at Class._treeFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:616:33)
    at Class.treeFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/addon.js:576:21)
    at project.addons.reduce (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:699:26)
    at Array.reduce (<anonymous>)
    at EmberApp._addonTreesFor (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:697:32)
    at EmberApp._addonBundles (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1160:31)
    at EmberApp.createAddonTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1193:29)
    at EmberApp.addonTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1208:36)
    at EmberApp.getExternalTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1264:25)
    at EmberApp.toArray (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1619:12)
    at EmberApp.toTree (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/broccoli/ember-app.js:1747:22)
    at module.exports (/Users/chris/dev/olo/web-client/ember-cli-build.js:88:14)
    at Builder.readBuildFile (/Users/chris/dev/olo/web-client/node_modules/ember-cli/lib/models/builder.js:52:14)
-~- (end) -~-
  - name: BuildError
  - nodeAnnotation: [undefined]
  - nodeName: Funnel
  - originalErrorMessage: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
  - stack: Error: ENOENT: no such file or directory, scandir '/Users/chris/dev/oss/ember/freestyle/addon/styles/app/styles/'
    at Object.fs.readdirSync (fs.js:904:18)
    at _walkSync (/Users/chris/dev/oss/ember/freestyle/node_modules/walk-sync/index.js:74:18)
    at Function.entries (/Users/chris/dev/oss/ember/freestyle/node_modules/walk-sync/index.js:50:10)
    at Funnel.processFilters (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:320:26)
    at Funnel.build (/Users/chris/dev/oss/ember/freestyle/node_modules/broccoli-funnel/index.js:258:10)
    at resolve (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/wrappers/transform-node.js:29:35)
    at new Promise (<anonymous>)
    at TransformNodeWrapper.build (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/wrappers/transform-node.js:21:12)
    at Promise.resolve.then.then.then (/Users/chris/dev/olo/web-client/node_modules/broccoli/lib/builder.js:111:28)
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)

=================================================================================

I'll get a PR opened first thing tomorrow!

matching improvments

Our current matching behavior has grown fairly organically, but it is likely time to do some cleanup and refactoring.

Current matching quirks:

  • include/exclude don't work with files
  • include \w only string inputs takes a fast-path when walking the dir tree
    • excludes are applied after (but could be used to improve dir walk pruning)
    • when the fast-path is taken, we still duplicate matching effort here

Goals:

  • allow files to be a function #25
  • throw if files + includes|exclude are provided, until they work together
  • when files is included, also filter then with include/excludes
  • make include/exclude + RegExp all take the fast path (this likely requires improvements to my walk-sync fork)

note: the fast-path, mentioned above, is actually to prune dir while walking rather then after walking – more details: joliss/node-walk-sync#5

"ENOENT: no such file or directory" when deleting a file

I'm not sure this is the right module to open this issue against, but recently I started getting ENOENT: no such file or directory whenever I delete a file in my Ember app tree while ember serve is running.

For example, if I have a global-message-service component, and I run:

% rm -r app/pods/global-message-service

I then see this:

file deleted pods/global-message-service
file deleted pods/global-message-service/service.js
The Broccoli Plugin: [Funnel: Funnel: Filtered App] failed with:
Error: ENOTEMPTY: directory not empty, rmdir '/Users/denchen/git/my-app/tmp/funnel-output_path-ZFVAFCmr.tmp/pods/global-message-service/'
  at Error (native)
  at Object.fs.rmdirSync (fs.js:758:18)
  at Funnel.applyPatch [as _applyPatch] (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/index.js:346:10)
  at Funnel.<anonymous> (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/index.js:318:10)
  at Array.forEach (native)
  at Funnel.processFilters (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/index.js:317:11)
  at Funnel.build (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/index.js:233:10)
  at /Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/node_modules/broccoli-plugin/read_compat.js:61:34
  at tryCatch (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:538:12)
  at invokeCallback (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:553:13)
  at publish (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:521:7)
  at flush (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:2373:5)
  at nextTickCallbackWith0Args (node.js:420:9)
  at process._tickCallback (node.js:349:13)

The broccoli plugin was instantiated at: 
  at Funnel.Plugin (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/node_modules/broccoli-plugin/index.js:7:31)
  at new Funnel (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/broccoli-funnel/index.js:57:10)
  at EmberApp._filterAppTree (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:679:31)
  at EmberApp._processedAppTree (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:793:54)
  at EmberApp.appAndDependencies (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:1063:10)
  at EmberApp.javascript (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:1199:34)
  at EmberApp.toArray (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:1604:10)
  at EmberApp.toTree (/Users/denchen/git/my-app/node_modules/ember-cli/lib/broccoli/ember-app.js:1626:30)
  at module.exports (/Users/denchen/git/my-app/ember-cli-build.js:83:14)
  at CoreObject.module.exports.Task.extend.setupBroccoliBuilder (/Users/denchen/git/my-app/node_modules/ember-cli/lib/models/builder.js:74:19)
  at CoreObject.module.exports.Task.extend.init (/Users/denchen/git/my-app/node_modules/ember-cli/lib/models/builder.js:54:10)
  at CoreObject.superWrapper [as init] (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/core-object/lib/assign-properties.js:32:18)
  at CoreObject.Class (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/core-object/core-object.js:32:33)
  at CoreObject.module.exports.Task.extend.run (/Users/denchen/git/my-app/node_modules/ember-cli/lib/tasks/serve.js:15:19)
  at /Users/denchen/git/my-app/node_modules/ember-cli/lib/commands/serve.js:75:24
  at tryCatch (/Users/denchen/git/my-app/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:538:12)

Deleting the file first instead of the directory also triggers the error. Deleting tmp/ & dist/ before I start the server seems to have no effect.

I can't be sure, but I think the errors started happening when I did a fresh npm install of my Ember app recently. Just to verify it's not a bug I recently introduced on my side, I did a git checkout of my code from late September, did a fresh npm install, and I still see this issue. I see broccoli-funnel has updated recently, which is why I decided to open the issue here.

Any ideas?

Support dynamic files filtering?

Not sure where to put this, but I figured I'd create an issue to see if there was any interest from others.

In my (complex) internal build setup, I needed the files option to broccoli-filter to be dynamic. More specifically, I needed it to be a function that could return a different list of files every rebuild.

My guess is that kind of behavior is likely out-of-scope for broccoli-funnel, but just in case I'm wrong (or anyone else that needs that functionality stumbles along this), here is my forked branch that enables it: master...timmfin:dynamic-filters.

ps: The reason I needed this is because I'm building a tree of file dependency info during the broccoli build (via the CalculateDependenciesFilter in broccoli-dependencies) and then I want to use that info later in the files filter for a funnel. And since the dependency info isn't generated until after the the CalculateDependenciesFilter filter has run, I cannot pass broccoli-funnel the files array at Brocfile.js "eval-time".

Cannot set property 'patches' of undefined

Hey, I'm using ember-cli which uses this library. During the build, I suddenly have this stacktrace (works before). Any idea?

The Broccoli Plugin: [Funnel: Funnel: index.html] failed with:
TypeError: Cannot set property 'patches' of undefined
    at Funnel.processFilters (/app/tmp/cache/node_modules/broccoli-funnel/index.js:308:33)
    at Funnel.build (/app/tmp/cache/node_modules/broccoli-funnel/index.js:233:10)
    at /app/tmp/cache/node_modules/broccoli-plugin/read_compat.js:61:34
    at tryCatch (/app/tmp/cache/node_modules/rsvp/dist/rsvp.js:538:12)
    at invokeCallback (/app/tmp/cache/node_modules/rsvp/dist/rsvp.js:553:13)
    at publish (/app/tmp/cache/node_modules/rsvp/dist/rsvp.js:521:7)
    at flush (/app/tmp/cache/node_modules/rsvp/dist/rsvp.js:2373:5)
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)
The broccoli plugin was instantiated at: 
    at Funnel.Plugin (/app/tmp/cache/node_modules/broccoli-plugin/index.js:7:31)
    at new Funnel (/app/tmp/cache/node_modules/broccoli-funnel/index.js:57:10)
    at EmberApp.index (/app/tmp/cache/node_modules/ember-cli/lib/broccoli/ember-app.js:641:15)
    at EmberApp.toArray (/app/tmp/cache/node_modules/ember-cli/lib/broccoli/ember-app.js:1603:10)
    at EmberApp.toTree (/app/tmp/cache/node_modules/ember-cli/lib/broccoli/ember-app.js:1626:30)
    at module.exports (/tmp/build_261851e7af050fd916277787d1c1c44c/ForestAdmin-forestadmin-ea054b32b7621ae713614798a568b506d70f08a4/ember-cli-build.js:83:14)
    at CoreObject.module.exports.Task.extend.setupBroccoliBuilder (/app/tmp/cache/node_modules/ember-cli/lib/models/builder.js:74:19)
    at CoreObject.module.exports.Task.extend.init (/app/tmp/cache/node_modules/ember-cli/lib/models/builder.js:54:10)
    at CoreObject.superWrapper [as init] (/app/tmp/cache/node_modules/core-object/lib/assign-properties.js:32:18)
    at CoreObject.Class (/app/tmp/cache/node_modules/core-object/core-object.js:32:33)
 !     Push rejected, failed to compile Ember CLI app.
 !     Push failed

Add Comment

Exception is thrown in 1.0.8 with relative paths in broccoli-funnel

The new code to handle exceptions incorrectly assumes that a symlink doesn't exist and thinks that srcDir is undefined throw new Error('You specified a "srcDir": ' + this.srcDir + 'which does not exist and did not specify"allowEmpty": true.');

The problem seems to be on exists-sync that doesn't expand the path correctly.

Example:

new Funnel('../static/js/', {
					destDir: 'static'
				})

That will create a symlink like:

09:14 AM $ ls -l /Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/tmp/funnel-input_base_path-9rLzS07l.tmp/0 
lrwxr-xr-x  1 mmadero  staff  77 Nov  3 08:20 /Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/tmp/funnel-input_base_path-9rLzS07l.tmp/0 -> /Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/../static/js/

When calling existsSync(/Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/tmp/funnel-input_base_path-9rLzS07l.tmp/0 );, it returns false.

When it gets the relativeLink on https://github.com/ember-cli/exists-sync/blob/master/index.js#L15 from"/Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/../static/js/ it goes to Users/mmadero/code-zenefits/yourPeople3-bkp3/component-library/js.

There're no tests for this scenario on exists-sync, but I wanted to raise the issue in both repos in case someone runs into this or in case someone gets a chance to fix it before I do.

Diff

[3.0.1] destDir not respected

It appears that as of version 3.0.1 the destDir is no longer honored. In the reproduction below, the app.css file is correctly written to dist/assets/styles/ while the 3.0.1 example is writing the file to dist/.

Brocfile.js

const funnel = require('broccoli-funnel')
module.exports = funnel('app/styles', {
  files: ['app.css'],
  destDir: '/assets/styles',
})

3.0.0

$ tree app
app
├── index.html
└── styles
    └── app.css

1 directory, 3 files

$ rm -rf dist && broccoli build; tree dist
dist
└── assets
    └── styles
        └── app.css

2 directories, 1 file

3.0.1

$ tree app
app
├── index.html
└── styles
    └── app.css

1 directory, 3 files

$ rm -rf dist && broccoli build; tree dist
dist
├── app.css
└── assets
    └── styles

2 directories, 1 file

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.