broccolijs / broccoli-concat Goto Github PK
View Code? Open in Web Editor NEWConcatenate broccoli trees
License: MIT License
Concatenate broccoli trees
License: MIT License
I reported this issue on ember-cli repo but since I believe this is related to broccoli-concat, I am opening it here as well, please close whichever-one is less relevant.
Ember build failed.
When attempting to run serve/test/build.
I believe this has to do with broccoli-concat as the tmp folder (0/bower_components) stays empty all along.
ember-cli v2.2.0-beta.1 had the same behavior but not on all machine so it was hard to describe (maybe I did something wrong when upgrading on a particular machine).
This is on an upgraded project (upgrade from 1.13.x -> 2.2.0-beta.1 -> 2.2.0-beta.2)
Here is the full traceback:
$ ember build
version: 2.2.0-beta.2
Could not find watchman, falling back to NodeWatcher for file system events.
Visit http://www.ember-cli.com/user-guide/#watchman for more info.
BuildingSmoke-and-mirrors:: Including Vertical-Collection Debug CSS
Build failed.
ENOENT: no such file or directory, stat '/home/emma/intranet/front/tmp/source_map_concat-input_base_path-tuTVBTdN.tmp/0/bower_components/jquery-mockjax/dist/jquery.mockjax.js'
Error: ENOENT: no such file or directory, stat '/home/emma/intranet/front/tmp/source_map_concat-input_base_path-tuTVBTdN.tmp/0/bower_components/jquery-mockjax/dist/jquery.mockjax.js'
at Error (native)
at Object.fs.statSync (fs.js:849:18)
at ConcatWithMaps.keyForFile (/home/emma/intranet/front/node_modules/ember-cli/node_modules/broccoli-concat/node_modules/broccoli-caching-writer/index.js:90:20)
at Array.map (native)
at ConcatWithMaps.CachingWriter._conditionalBuild (/home/emma/intranet/front/node_modules/ember-cli/node_modules/broccoli-concat/node_modules/broccoli-caching-writer/index.js:112:65)
at /home/emma/intranet/front/node_modules/ember-cli/node_modules/broccoli-concat/node_modules/broccoli-caching-writer/node_modules/broccoli-plugin/read_compat.js:61:34
at lib$rsvp$$internal$$tryCatch (/home/emma/intranet/front/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:493:16)
at lib$rsvp$$internal$$invokeCallback (/home/emma/intranet/front/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:505:17)
at lib$rsvp$$internal$$publish (/home/emma/intranet/front/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:476:11)
at lib$rsvp$asap$$flush (/home/emma/intranet/front/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:1198:9)
package.json is:
{
"name": "intranet-ng",
"version": "0.0.0",
"description": "Small description for intranet-ng goes here",
"private": true,
"directories": {
"doc": "doc",
"test": "tests"
},
"scripts": {
"build": "ember build",
"start": "ember server",
"test": "ember test"
},
"repository": "",
"engines": {
"node": ">= 0.10.0"
},
"author": "",
"license": "MIT",
"devDependencies": {
"broccoli-asset-rev": "^2.2.0",
"ember-ajax": "0.7.1",
"ember-cli": "2.2.0-beta.2",
"ember-cli-app-version": "^1.0.0",
"ember-cli-babel": "^5.1.5",
"ember-cli-dependency-checker": "^1.2.0",
"ember-cli-htmlbars": "^1.0.1",
"ember-cli-htmlbars-inline-precompile": "^0.3.1",
"ember-cli-inject-live-reload": "^1.3.1",
"ember-cli-qunit": "^1.1.0",
"ember-cli-release": "0.2.8",
"ember-cli-sass": "^5.2.1",
"ember-cli-sri": "^1.2.0",
"ember-cli-uglify": "^1.2.0",
"ember-component-css": "0.1.6",
"ember-data": "^2.2.1",
"ember-disable-proxy-controllers": "^1.0.1",
"ember-export-application-global": "^1.0.4",
"ember-resolver": "^2.0.3",
"ember-data-factory-guy": "2.1.2",
"ember-django-adapter": "^1.1.1",
"ember-href-to": "0.0.11",
"ember-hypersearch": "0.1.0",
"ember-inline-svg": "^0.1.5",
"ember-simple-auth": "1.0.1",
"ember-truth-helpers": "1.2.0",
"ember-validations": "2.0.0-alpha.4",
"liquid-fire": "0.22.1",
"smoke-and-mirrors": "0.4.4"
}
}
bower.json is:
{
"name": "intranet-ng",
"dependencies": {
"ember": "2.2.0",
"ember-cli-shims": "0.0.6",
"ember-cli-test-loader": "0.2.1",
"ember-data": "^2.2.1",
"ember-load-initializers": "0.1.7",
"ember-qunit-notifications": "0.1.0",
"jquery": "^2.1.4",
"loader.js": "^3.5.0",
"qunit": "~1.20.0"
},
"resolutions": {
"ember": "2.2.0"
}
}
ember-cli-build.js is:
/* global require, module */
var EmberApp = require('ember-cli/lib/broccoli/ember-app');
module.exports = function(defaults) {
var app = new EmberApp(defaults, {
svg: {
paths: [
'public/assets/svgs'
]
}
});
app.import('vendor/bootstrap/bootstrap.js');
app.import('vendor/jasny-bootstrap.js');
return app.toTree();
};
repo demonstrating the problem: https://bitbucket.org/levit_scs/djember_front
I would expect { wrapInFunction: true, wrapInEval: false }
to transform
// one.js
one();
// two.js
two();
into
(function() {
one();
two();
})();
It's possible, though, that the correct answer is to create a broccoli-iife
that wraps each of its inputs in an IIFE, then apply that after this concatenation with both wrapInEval
and wrapInFunction
turned off.
We should likely only retain (in memory) files contents if they are under a certain file size.
The optimization is most important for lots of small files, but for larger files I suspect retaining the content in memory is worse then just re-reading it.
forgot an import, fixing..
Use of linkSync on line 86 causes issues when running this task inside a Vagrant box.
See ember-cli/broccoli-caching-writer#2 for more details.
I'm not 100 % sure on the claims I make here, so please correct me, if I'm wrong.
It appears that input files that already have source maps attached to them are just passed through transparently, including the sourceMappingURL
comment. This means that you will end up with a concatenated output file, that has one valid sourceMappingURL
comment—the one generated by broccoli-concat
—and possibly multiple invalid sourceMappingURL
comments somewhere in the file, that reference source maps that are not part of the output tree anymore.
Adding a file to vendor.js
via EmberApp#import()
under the hood eventually calls this plugin and I actually tripped over this in ember-cli/ember-fetch#119. ember-fetch
also uses broccoli-concat
to merge two files and since a source map is generated by default, the resulting concatenated file has a sourceMappingURL
comment, which then gets passed through to the vendor.js
file, where it eventually wreaks havoc in broccoli-uglify-sourcemap/lib/process-file.js#L31-L39
.
After upgrading to 4.2.1 our ember builds started failing (at least on macOS).
>> (node:52291) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open '/var/folders/30/3l01y1ld18l598kxr5srm7kr0000gn/T/broccoli-52291zjOJHNK9FLVI/out-541-simple_concat_concat/assets/codemirror/codemirror.js'
>> at Object.openSync (fs.js:443:3)
>> at Object.writeFileSync (fs.js:1194:35)
>> at Object.handleFs (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/node_modules/broccoli-output-wrapper/dist/index.js:45:44)
>> at Concat._doPatchBasedBuild (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/concat.js:166:17)
>> at Concat.build (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli-concat/concat.js:104:19)
>> at resolve (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/wrappers/transform-node.js:68:45)
>> at new Promise (<anonymous>)
>> at TransformNodeWrapper.build (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/wrappers/transform-node.js:53:16)
>> at Promise.resolve.then.then.then (/Users/kevinansfield/code/ghost/Ghost/core/client/node_modules/broccoli/dist/builder.js:99:36)
>> at process._tickCallback (internal/process/next_tick.js:68:7)
The relevant part of our ember-cli-build.js
where we use broccoli-concat
is as follows:
let jsTree = concat(tree, {
outputFile: 'assets/codemirror/codemirror.js',
headerFiles: ['lib/codemirror.js'],
inputFiles: ['mode/**/*'],
sourceMapConfig: {enabled: false}
});
Changing the outputFile
to remove the directory allows that step of the build to pass. Downgrading to 4.0.0 and keeping the directory in outputFile
also allows works fine.
Errors catched and rethrown are being thrown as strings (throwing only the message and not the complete error object) at https://github.com/rlivsey/broccoli-concat/blob/master/index.js#L70 .
I'm trying to concatenate some js
files using this Brocfile.js
var concat = require('broccoli-concat');
module.exports = concat('src', {
inputFiles: ['*.js'],
outputFile: '/dist/app.js'
});
This creates /tmp/concat-tmp_dest_dir-MN3cRhw9.tmp
directory that indeed has the concatenated file but it's never moved to the correct location. A bug or I'm missing something?
broccoli 0.12.3
broccoli-concat 0.0.7
cross post: ef4/fast-sourcemap-concat#45
I'm curious if there's a plan to support concating files together that each have their own source maps and create an output file + an aggregated source map?
This could be a bug to solve here or elsewhere. The scenario is like this:
file1.js:
console.log('some js code');
// last line is commented
file2.js
alert('run this code')
After the concat, concat.js
console.log('some js code');
// last line is commentedalert('run this code')
There is no newline at the end of the first file, and the two are concated to produce failing code.
Could add a default separator. Could add an option. What would you prefer?
Just starting to play with Broccoli, but as I understand concat concatenates a set of matched files into a tree with just the one output file. Wouldn't it be more useful to return a tree still including all the files that were untouched by the concat operation? I just want the same tree back, but with the selected files concatenated. Or is there something I am missing?
when I did the followong:
var concat = require('broccoli-concat');
var bundlePath = 'aura/AuraComponent';
var controllerName = 'AuraComponentController.js'
var testableController = concat(bundlePath, {
outputFile: controllerName + '.hintable',
header: "module.exports = ",
inputFiles: [controllerName],
footer: ';',
sourceMapConfig: { enabled: false },
allowNone: true
});
I got the error:
Error: ENOENT: no such file or directory, open 'aura/AuraComponent/aura/AuraComponent/AuraComponentController.js'
at Error (native)
at Object.fs.openSync (fs.js:584:18)
at Object.fs.readFileSync (fs.js:431:33)
at Simple.addFile (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat-without-source-maps.js:15:21)
at Concat.<anonymous> (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat.js:205:17)
at Array.forEach (native)
at Concat.addFiles (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat.js:203:9)
at Concat.<anonymous> (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat.js:138:10)
at Simple.end (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat-without-source-maps.js:32:17)
at Concat.build (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli-concat/concat.js:117:22)
at TransformNodeWrapper.build (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli/lib/builder.js:440:43)
at /home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/broccoli/lib/builder.js:101:23
at tryCatch (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/rsvp/dist/rsvp.js:538:12)
at invokeCallback (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/rsvp/dist/rsvp.js:553:13)
at publish (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/rsvp/dist/rsvp.js:521:7)
at flush (/home/benjamin/Desktop/projects/mavensmate/yahoo/resource-bundles/node_modules/rsvp/dist/rsvp.js:2373:5)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)
In version 3.1.1.
Reverting to version 3.0.5 fixed the issue
The current release is currently broken for me, likely due to 521d2c2 not being available in the current release
ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
=================================================================================
ENV Summary:
TIME: Wed Jul 25 2018 14:38:29 GMT+0000 (UTC)
TITLE: ember
ARGV:
- /home/travis/.nvm/versions/node/v8.11.3/bin/node
- /home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/ember-cli/bin/ember
- test
EXEC_PATH: /home/travis/.nvm/versions/node/v8.11.3/bin/node
TMPDIR: /tmp
SHELL: /bin/bash
PATH:
- /home/travis/.yarn/bin
- /home/travis/.rvm/gems/ruby-2.4.1/bin
- /home/travis/.rvm/gems/ruby-2.4.1@global/bin
- /home/travis/.rvm/rubies/ruby-2.4.1/bin
- /home/travis/.rvm/bin
- ./node_modules/.bin
- /home/travis/bin
- /home/travis/.local/bin
- /opt/pyenv/shims
- /home/travis/.phpenv/shims
- /home/travis/perl5/perlbrew/bin
- /home/travis/.nvm/versions/node/v8.11.3/bin
- /home/travis/.kiex/elixirs/elixir-1.4.5/bin
- /home/travis/.kiex/bin
- /home/travis/gopath/bin
- /usr/local/phantomjs/bin
- /usr/local/phantomjs
- /usr/local/neo4j-3.2.7/bin
- /usr/local/maven-3.5.2/bin
- /usr/local/cmake-3.9.2/bin
- /usr/local/clang-5.0.0/bin
- /home/travis/.gimme/versions/go1.7.4.linux.amd64/bin
- /usr/local/sbin
- /usr/local/bin
- /usr/sbin
- /usr/bin
- /sbin
- /bin
- /home/travis/.phpenv/bin
- /opt/pyenv/bin
- /home/travis/.yarn/bin
PLATFORM: linux x64
FREEMEM: 29225865216
TOTALMEM: 67275345920
UPTIME: 499
LOADAVG: 31.50341796875,24.88134765625,11.72412109375
CPUS:
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
- AMD EPYC 7401P 24-Core Processor - 1996
ENDIANNESS: LE
VERSIONS:
- ares: 1.10.1-DEV
- cldr: 32.0
- http_parser: 2.8.0
- icu: 60.1
- modules: 57
- napi: 3
- nghttp2: 1.32.0
- node: 8.11.3
- openssl: 1.0.2o
- tz: 2017c
- unicode: 10.0
- uv: 1.19.1
- v8: 6.2.414.54
- zlib: 1.2.11
ERROR Summary:
- broccoliBuilderErrorStack: Error: ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at Entry.get content [as content] (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:25:22)
at contentMapper (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:7:16)
at Array.map (<anonymous>)
at SimpleConcat.result (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:158:22)
at Concat._doPatchBasedBuild (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/concat.js:140:31)
at Concat.build (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/concat.js:102:19)
at /home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-plugin/read_compat.js:93:34
at tryCatch (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:525:12)
at invokeCallback (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:538:13)
at publish (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:508:7)
at flush (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:2415:5)
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
- codeFrame: ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
- errorMessage: Build Canceled: Broccoli Builder ran into an error with `SimpleConcatConcat` plugin. 💥
ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
- errorType: Build Error
- location:
- column: [undefined]
- file: [undefined]
- line: [undefined]
- treeDir: [undefined]
- message: Build Canceled: Broccoli Builder ran into an error with `SimpleConcatConcat` plugin. 💥
ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
- name: Error
- nodeAnnotation: SimpleConcatConcat
- nodeName: SimpleConcatConcat
- originalErrorMessage: ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
- stack: Error: ENOENT: no such file or directory, open '[broccoli-persistent-filter:EslintValidationFilter]/addon/engine.lint-test.js'
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at Entry.get content [as content] (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:25:22)
at contentMapper (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:7:16)
at Array.map (<anonymous>)
at SimpleConcat.result (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/lib/strategies/simple.js:158:22)
at Concat._doPatchBasedBuild (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/concat.js:140:31)
at Concat.build (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-concat/concat.js:102:19)
at /home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/broccoli-plugin/read_compat.js:93:34
at tryCatch (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:525:12)
at invokeCallback (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:538:13)
at publish (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:508:7)
at flush (/home/travis/build/sass-eyeglass/ember-cli-eyeglass/node_modules/rsvp/dist/rsvp.js:2415:5)
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
=================================================================================
What do you think of adding a passthrough
option which would copy forward any unmatched files?
I ran into an issue where files with a trailing single line comment would break subsequent files. One example of this in practice is JSS followed by ZeroClipboard:
var exports = new Jss(document);
exports.forDocument = function(doc) {
return new Jss(doc);
};
return exports;
})();
typeof module !== 'undefined' && module.exports && (module.exports = jss); // CommonJS support/*!
* zeroclipboard
* The Zero Clipboard library provides an easy way to copy text to the clipboard using an invisible Adobe Flash movie, and a JavaScript interface.
* Copyright 2012 Jon Rohan, James M. Greene, .
* Released under the MIT license
* http://jonrohan.github.com/ZeroClipboard/
* v1.1.7
*/(function() {
"use strict";
As you can see, the opening block comment from ZeroClipboard is eaten by the trailing comment in JSS.
fast-sourcemap-concat
is currently pinned to v1.1.x due to a failing test. We should investigate what is causing this failure and fix and update if possible.
see #117
I'm new to broccoli (thanks for writing this much needed module btw) !
Here is my question. I have a templates tree that is my compiled handlebars templates. I then have a vendor tree that is all my stuff concatted (jquery/ember/etc)
How can I use this concat module to put them both together, such that I output a single JS file ?
Here is that I have so far (note -its returning several files today -not a single JS file)
module.exports = function (broccoli) {
var concat = require('broccoli-concat')
var filterTemplates = require('broccoli-template')
var pickFiles = require('broccoli-static-compiler')
function preprocess (tree) {
tree = filterTemplates(tree, {
extensions: ['handlebars'],
compileFunction: 'Ember.Handlebars.compile'
})
return tree
}
var sourceTree = broccoli.makeTree('js');
var templates = pickFiles(sourceTree, {
srcDir: '/templates/',
destDir: '/compiled'
})
var appTemplates = preprocess(templates);
var sourceTree = broccoli.makeTree('js');
var appJs = concat(sourceTree, {
inputFiles: [
'vendor/jquery/jquery.js',
'vendor/handlebars/handlebars.min.js',
'vendor/ember/ember.js',
'app.js',
'other.js'
],
outputFile: '/deps.min.js'
});
return [appJs, appTemplates];
}
Seems missing from the list of options in the README, and the specifics of what it supports (globbing, order based on array position, etc) are potentially important...
Reference: https://twitter.com/timo_erl/status/513491815891484672
I am using ember-cli and thus whenever I do a fresh install of packages I get the new version. I don't have the option of locking back to 3.2.2 without locking every one of my packages - shouldn't the minor versions be non-breaking?
consider the following file structure:
- app/
|- app.js
|- app.html
- Brocfile.js
app/app.js
console.log('testing');
app/app.html
<div>test</div>
Brocfile.js
var concat = require('broccoli-concat');
module.exports = concat('app', {
inputFiles : [ '**/*.js' ],
footerFiles : ['app.html'],
outputFile : 'scripts/app.js',
annotation : 'Concat: App'
});
While running a broccoli serve
, the initial build returns the expected output:
scripts/app.js
console.log('testing');
<div>test</div>
However, if I make a change to app/app.html
, the file does not change. The console suggests a rebuild has happened, but this isn't generating a change in the concatinated file.
Sample terminal output
broccoli serve
Serving on http://localhost:4200
Slowest Trees | Total
----------------------------------------------+---------------------
SourceMapConcat: Concat: App | 29ms
Built - 32 ms @ Sat Apr 23 2016 13:37:43 GMT-0500 (CDT)
Slowest Trees | Total
----------------------------------------------+---------------------
SourceMapConcat: Concat: App | 2ms
Built - 4 ms @ Sat Apr 23 2016 13:38:00 GMT-0500 (CDT)
See here:
I have been working to update (and improve) https://github.com/ef4/broccoli-sourcemap-concat although I would also love to have the time to update this one. I unfortunately do not. Rather then having two contending libraries, (one falling behind recent improvements) let me suggestion merging efforts.
I will gladly add features missing to broccoli-sourcemap-concat, in hopes to unify the two worlds, and give users the best experience.
As for project name, I can go either way. broccoli-concat is likely the better name for the final project.
Anyways, let me know what you think.
broccoli-sourcemap-concat
defaults mapCommentType
to 'line'
, which means //
comments, which are invalid CSS. This library doesn't override that default at all. This library does pass sourceMapConfig
down to the Strategy
, but there's no way to configure mapCommentType
on a per-file or per-file-type basis.
Possible solutions:
*.js
with mapCommentType: 'line'
and one for *.css
with mapCommentType: 'block'
. This doesn't work too well with Ember-CLI since that assumes a single sourcemaps
configuration.mapCommentType: (/\.css\b/.test(this.mapFile) ? 'block' : 'line')
See also #17 and ef4/broccoli-sourcemap-concat#37
It'd be nice to be able to pass an entire glob for my input tree , but then exclude certain files like modernizr which can't often be used at the bottom of html body.
Right now if allowNone
is true, the first inputFiles
element that doesn't have any glob matches will cease processing. Ideally, allowNone
should be accepted by the multiGlob helper method (see broccolijs/broccoli-kitchen-sink-helpers#19 ) , but this could be worked around by rejecting inputFiles
for which glob.sync
has no matches.
It appears the during SourceMap concat, we accidentally support absolute paths, but in Simple we do not. We should actually never support absolute paths, and that was merely a mistake.
People run into this, in ember via app.import(someAbsolutePath)
. And it leads to sadness and confusion, as often they run sourceMaps in dev, but not prod. Resulting in dev including the file and prod not. This sucks.
Two options:
a) We drop support for absolute paths and the rollout plan would be something like:
b) We add support uniformly for absolute paths.
Build failed.
The Broccoli Plugin: [SimpleConcatConcat: Concat: Vendor Styles/assets/vendor.css] failed with:
Error: Concat: Result is empty and allowNone is falsy.
package.json:
{
"name": "crm",
"version": "0.1.0",
"description": "CRM application",
"private": true,
"directories": {
"doc": "doc",
"test": "tests"
},
"scripts": {
"build": "ember build",
"start": "ember server",
"test": "ember test"
},
"repository": "",
"engines": {
"node": ">= 4.5.0"
},
"author": "",
"license": "MIT",
"devDependencies": {
"active-model-adapter": "~2.1.1",
"broccoli-asset-rev": "^2.5.0",
"broccoli-clean-css": "^1.1.0",
"ember-ajax": "~2.5.4",
"ember-cli": "~2.10.1",
"ember-cli-app-version": "^1.0.1",
"ember-cli-babel": "~5.1.10",
"ember-cli-chai": "~0.3.2",
"ember-cli-coffeescript": "~1.15.0",
"ember-cli-dependency-checker": "^1.2.0",
"ember-cli-emblem": "~0.8.1",
"ember-cli-flash": "1.4.1",
"ember-cli-foundation-6-sass": "0.0.16",
"ember-cli-htmlbars": "^1.1.1",
"ember-cli-htmlbars-inline-precompile": "^0.3.6",
"ember-cli-inject-live-reload": "^1.6.1",
"ember-cli-jshint": "^1.0.5",
"ember-cli-mirage": "0.2.5",
"ember-cli-mocha": "~0.13.2",
"ember-cli-mocha-coffeescript": "git+https://github.com/mhluska/ember-cli-mocha-coffeescript.git",
"ember-cli-moment-shim": "~2.2.1",
"ember-cli-numeral": "0.2.0",
"ember-cli-rails-addon": "~0.8.0",
"ember-cli-release": "^0.2.9",
"ember-cli-sass": "5.6.0",
"ember-cli-sri": "^2.1.1",
"ember-cli-test-loader": "^1.1.1",
"ember-cli-uglify": "^1.2.0",
"ember-cli-version-checker": "^1.1.6",
"ember-data": "~2.10.0",
"ember-export-application-global": "^1.1.1",
"ember-font-awesome": "3.0.3",
"ember-load-initializers": "^0.5.1",
"ember-moment": "7.3.0",
"ember-power-select": "1.4.3",
"ember-power-select-with-fallback": "https://github.com/pohodnya/ember-power-select-with-fallback.git",
"ember-resolver": "^2.1.1",
"ember-simple-auth": "~1.2.0",
"ember-welcome-page": "^1.0.4",
"loader.js": "^4.1.0",
"rsvp": "^3.3.3"
},
"ember-addon": {
"paths": [
"../-shared"
]
}
}
bower.json:
{
"name": "crm",
"dependencies": {
"ember": "~2.10.2",
"ember-cli-shims": "~0.1.3",
"foundation-sites": "~6.2.4",
"moment": "^2.17.1",
"font-awesome": "~4.7.0"
}
}
ember-cli-build.js:
var EmberApp = require('ember-cli/lib/broccoli/ember-app');
module.exports = function(defaults) {
var app = new EmberApp(defaults, {
babel: {
includePolyfill: true
},
fingerprint: {
prepend: '/crm/',
extensions: ['js', 'css', 'png', 'gif', 'jpeg', 'jpg', 'svg', 'eot', 'ttf', 'woff']
},
'ember-font-awesome': {
fontsOutput: "/fonts/",
useScss: true
},
'ember-cli-foundation-6-sass': {
'foundationJs': 'all'
}
});
app.import('bower_components/moment/locale/ru.js');
return app.toTree();
};
full error:
Error: Concat: Result is empty and allowNone is falsy.
at Concat._doPatchBasedBuild (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/node_modules/broccoli-concat/concat.js:143:11)
at Concat.build (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/node_modules/broccoli-concat/concat.js:104:17)
at /home/aleksey/projects/ta/frontend/crm/node_modules/broccoli-plugin/read_compat.js:93:34
at tryCatch (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:538:12)
at invokeCallback (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:553:13)
at publish (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:521:7)
at flush (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:2373:5)
at nextTickCallbackWith0Args (node.js:436:9)
at process._tickCallback (node.js:365:13)
The broccoli plugin was instantiated at:
at Concat.Plugin (/home/aleksey/projects/ta/frontend/crm/node_modules/broccoli-plugin/index.js:7:31)
at new Concat (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/node_modules/broccoli-concat/concat.js:38:10)
at module.exports (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/node_modules/broccoli-concat/index.js:26:10)
at EmberApp.concatFiles (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/broccoli/ember-app.js:369:10)
at EmberApp._concatFiles (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/broccoli/ember-app.js:382:15)
at EmberApp.styles (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/broccoli/ember-app.js:1334:28)
at EmberApp.toArray (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/broccoli/ember-app.js:1647:10)
at EmberApp.toTree (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/broccoli/ember-app.js:1668:30)
at module.exports (/home/aleksey/projects/ta/frontend/crm/ember-cli-build.js:41:14)
at CoreObject.module.exports.Task.extend.setupBroccoliBuilder (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/models/builder.js:72:19)
at CoreObject.module.exports.Task.extend.init (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/models/builder.js:52:10)
at CoreObject.superWrapper [as init] (/home/aleksey/projects/ta/frontend/crm/node_modules/core-object/lib/assign-properties.js:32:18)
at CoreObject.Class (/home/aleksey/projects/ta/frontend/crm/node_modules/core-object/core-object.js:32:33)
at CoreObject.module.exports.Task.extend.run (/home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/tasks/build.js:15:19)
at /home/aleksey/projects/ta/frontend/crm/node_modules/ember-cli/lib/commands/build.js:32:24
at tryCatch (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:538:12)
at invokeCallback (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:553:13)
at /home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:628:16
at flush (/home/aleksey/projects/ta/frontend/crm/node_modules/rsvp/dist/rsvp.js:2373:5)
at nextTickCallbackWith0Args (node.js:436:9)
at process._tickCallback (node.js:365:13)
TypeError: undefined is not a function
at cleanupTree (/Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/lib/builder.js:147:17)
at /Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/node_modules/promise-map-series/index.js:11:14
at lib$rsvp$$internal$$tryCatch (/Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/node_modules/rsvp/dist/rsvp.js:493:16)
at lib$rsvp$$internal$$invokeCallback (/Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/node_modules/rsvp/dist/rsvp.js:505:17)
at lib$rsvp$$internal$$publish (/Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/node_modules/rsvp/dist/rsvp.js:476:11)
at lib$rsvp$asap$$flush (/Users/krainboltgreene/Code/socialkardia/dashboard/node_modules/broccoli/node_modules/rsvp/dist/rsvp.js:1198:9)
at process._tickCallback (node.js:355:11)
at Function.Module.runMain (module.js:503:11)
at startup (node.js:129:16)
at node.js:814:3
Build failed
While doing build work, I often open up dist/assets/vendor.js
or dist/assets/vendor.css
to inspect the contents (to confirm I didn't make any mistakes etc). It would be very convenient if we added delimiters surrounding each file that we concat.
Concretely, I am proposing:
fileDelimeters
option. A few special values could be used for substitution (e.g. {{FILE_PATH}}
would be the path of the file being added).
fileDelimeters: ['\n/* BEGIN {{FILE_PATH}} */', '\n/* END {{FILE_PATH}} */']
fileDelimeters
option in non-production builds.Thoughts?
Currently, this is the simplest possible concat
call in a Brocfile.js
:
var concat = require('broccoli-concat');
var appJs = concat('js', {
inputFiles: ['**/*'],
outputFile: '/all-js.js',
});
I think it would make sense for inputFiles
to have the value ['**/*']
by default. That is, by default, concat
concatenates all files in the given tree, in an arbitrary order. That would allow the plugin to be used like this:
var appJs = concat('js', {
outputFile: '/all-js.js',
});
Not requiring the inputFiles
parameter would make it easier for people to get started using this plugin.
While upgrading another dependency yarn decided to upgrade broccoli-concat to 3.5.0. I have many dependencies with broccoli-concat@^3.2.2 so this seems to be valid. On build I get the following error that I believe has something to do with ember-fetch.
~/dev/project1 master ● ember build
cleaning up...
Build failed.
Build Error (SimpleConcatConcat)
ENOENT: no such file or directory, open '[BroccoliMergeTrees]/abortcontroller-polyfill-only.js'
Stack Trace and Error Report: /tmp/error.dump.b9b5ea32ef6b5a0a03e1b7c1c26ba4ec.log
Just testing out the updates: ember-cli/ember-cli#8091
in package.json:
+ "ember-cli": "github:NullVoxPopuli/ember-cli#1f54b505f09fd7718cf6d4486238a8f91f9e0a58",
Got this error:
$ yarn broccoli-concat-analyser ./concat-stats-for
$ /home/me/Development/NullVoxPopuli/emberclear/packages/frontend/node_modules/.bin/broccoli-concat-analyser ./concat-stats-for
⠙ processing...{ Error: ENOENT: no such file or directory, open '/home/me/Development/NullVoxPopuli/emberclear/packages/frontend/concat-stats-for/1-ember-cli-notifications.css/ember-cli-notifications/styles/addon.css'
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at fileNames.forEach (/home/me/Development/NullVoxPopuli/emberclear/packages/frontend/node_modules/broccoli-concat-analyser/lib/summarize.js:31:63)
at Array.forEach (<anonymous>)
at Function.summarize (/home/me/Development/NullVoxPopuli/emberclear/packages/frontend/node_modules/broccoli-concat-analyser/lib/summarize.js:31:13)
at process.<anonymous> (/home/me/Development/NullVoxPopuli/emberclear/packages/frontend/node_modules/workerpool/lib/worker.js:76:27)
at emitTwo (events.js:126:13)
at process.emit (events.js:214:7)
at emit (internal/child_process.js:772:12)
at _combinedTickCallback (internal/process/next_tick.js:141:11)
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/home/me/Development/NullVoxPopuli/emberclear/packages/frontend/concat-stats-for/1-ember-cli-notifications.css/ember-cli-notifications/styles/addon.css' }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
me@Tyche ~/Development/NullVoxPopuli/emberclear/packages/frontend
⬡ v8.11.3 (code-quality-checking) $ la concat-stats-for/
total 72K
drwxr-xr-x 5 me me 4.0K Oct 5 14:38 .
drwxr-xr-x 14 me me 4.0K Oct 5 14:39 ..
drwxr-xr-x 10 me me 4.0K Oct 5 14:38 1-emberclear.js
-rw-r--r-- 1 me me 16K Oct 5 14:38 1-emberclear.js.json
-rw-r--r-- 1 me me 282 Oct 5 14:38 1-ember-cli-notifications.css.json
-rw-r--r-- 1 me me 6.6K Oct 5 14:38 2-emberclear-fastboot.js.json
drwxr-xr-x 10 me me 4.0K Oct 5 14:38 2-vendor.js
-rw-r--r-- 1 me me 20K Oct 5 14:38 2-vendor.js.json
drwxr-xr-x 10 me me 4.0K Oct 5 14:38 3-vendor.css
-rw-r--r-- 1 me me 267 Oct 5 14:38 3-vendor.css.json
If I have this in my Brocfile.js
:
var concat = require('broccoli-concat');
var appJs = concat(sourceTree, {
inputFiles: ['**'],
outputFile: 'all-js.js',
});
Then I get the following error:
Error: Path must be absolute: "all-js.js"
I am forced to add a /
to outputFile
, like this:
var appJs = concat(sourceTree, {
inputFiles: ['**'],
outputFile: '/all-js.js',
});
I don’t see why this should be necessary. If the caller doesn’t put a /
at the front of the path, it is reasonable to assume that they mean the root directory of the output tree. What else would they mean?
This could be implemented by checking whether outputFile
starts with /
, and prepending /
if it doesn’t, before doing anything else with that parameter.
https://github.com/rlivsey/broccoli-concat/blob/master/index.js#L87
The above line can cause operations to fail on windows where hard linking two directories is not permitted. http://msdn.microsoft.com/en-us/library/windows/desktop/aa365006%28v=vs.85%29.aspx
You could do a junction fs.symlinkSync(srcPath, dstPath, 'junction')
I think #30 is related. Also is there any reason this isn't abstracted to use https://www.npmjs.com/package/symlink-or-copy.
I'm traveling today, but will put a PR together later today if no one else has.
Hi there,
I'm not sure if should post it here, or stackoverflow, but I've tried all that I could think of, and I can't get globbing to work for me. My current file structure is like this:
styles/
application.css
bootstrap.min.css
And I want it to become this after build, with bootstrap.min.css first and then application.css afterwards in main.css:
app/css/
main.css
The processing is done like this:
// create tree for css
var cssTree = pickFiles("styles", {
srcDir: "/",
files: ["**/*.css"],
destDir: "app/css"
});
cssTree = concatCSS(cssTree, {
inputFiles: [
"**/*.css",
"!app/css/application.css",
"app/css/application.css"
],
outputFile: "/app/css/main.css",
});
If I in stead try globbing like this, it will give me the error Error: Path or pattern "!styles/application.css" did not match any files
:
inputFiles: [
"**/*.css",
"!styles/application.css",
"styles/application.css"
],
Is globbing not working as expected, or am I doing it wrong?
The documentation reads:
The structure of output.js will be as follows:
// - header
// - ordered content of the files in headerFiles
// - un-ordered content of files matched by inputFiles, but not in headerFiles or footerFiles
// - ordered content of the files in footerFiles
// - footer
Does this mean that if I have:
inputFiles: ['fileA', 'fileB']
The order of fileA and fileB in the final outputted file is not guaranteed? If so, how can I guarantee the order is as specified in the inputFiles array?
CVE-2019-10744
lodash/lodash#4336
high severity
Vulnerable versions: < 4.6.2
Patched version: lodash.merge@^4.6.2
Affected versions of lodash are vulnerable to Prototype Pollution.
The function defaultsDeep could be tricked into adding or modifying properties of Object.prototype using a constructor payload.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.