Giter VIP home page Giter VIP logo

exorcist's Introduction

exorcist build status

become a patron

Externalizes the source map found inside a stream to an external .map file or stream.

Works with both JavaScript and CSS input streams.

var browserify = require('browserify')
  , path       = require('path')
  , fs         = require('fs')
  , exorcist   = require('exorcist')
  , mapfile    = path.join(__dirname, 'bundle.js.map')

// from a file, to a file, and send source map to its own file
browserify({debug: true})
  .require(require.resolve('./main'), { entry: true })
  .bundle()
  .pipe(exorcist(mapfile))
  .pipe(fs.createWriteStream(path.join(__dirname, 'bundle.js'), 'utf8'))

//  from a stream, to a stream, and send source map to a stream
browserify([readableSourceStream], browserifyOptions)
  .bundle()
  .pipe(exorcist(targetSourceMapStream, '/url/path/to/replace/source/comment/with/bundle.js'))
  .pipe(writableTargetStream)

command line example

browserify main.js --debug | exorcist bundle.js.map > bundle.js

Table of Contents generated with DocToc

Usage

exorcist map_file [options]

  Externalizes the source map of the file streamed in.

  The source map is written as JSON to map_file, and the original file is streamed out with its
  sourceMappingURL set to the path of map_file (or to the value of the --url option).

OPTIONS:

              --base -b   Base path for calculating relative source paths.
                          (default: use absolute paths)

              --root -r   Root URL for loading relative source paths.
                          Set as sourceRoot in the source map.
                          (default: '')

               --url -u   Full URL to source map.
                          Set as sourceMappingURL in the output stream.
                          (default: map_file)

  --error-on-missing -e   Abort with error if no map is found in the stream.
                          (default: warn but still pipe through source)

EXAMPLE:

  Bundle main.js with browserify into bundle.js and externalize the map to bundle.js.map.

    browserify main.js --debug | exorcist bundle.js.map > bundle.js

Installation

npm install exorcist

API

exorcist(input, url, root, base, errorOnMissing) → {TransformStream}

Externalizes the source map of the file streamed in.

The source map is written as JSON to file, and the original file is streamed out with its sourceMappingURL set to the path of file (or to the value of url).

Events (in addition to stream events)

  • missing-map emitted if no map was found in the stream and errorOnMissing is falsey (the src is still piped through in this case, but no map file is written)
Parameters:
Name Type Argument Description
input String / Object

full path to the map file to which to write the extracted source map or a writable stream

url String <optional>

full URL to the map file, set as sourceMappingURL in the streaming output (default: file)

root String <optional>

root URL for loading relative source paths, set as sourceRoot in the source map (default: '')

base String <optional>

base path for calculating relative source paths (default: use absolute paths)

errorOnMissing Boolean <optional>

when truthy, causes 'error' to be emitted instead of 'missing-map' if no map was found in the stream (default: falsey)

Source:
Returns:

transform stream into which to pipe the code containing the source map

Type
TransformStream

generated with docme

Integration with other tools

License

MIT

exorcist's People

Contributors

aurelioderosa avatar bendrucker avatar dcohenb avatar jamesreggio avatar jasonholloway avatar jasonkarns avatar jgravois avatar meeber avatar nickheiner avatar oliverjash avatar thlorenz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

exorcist's Issues

Warning in mold-source-map repo

node:internal/process/warning:50: (node:17866) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/xxx/www/yyy/node_modules/mold-source-map/package.json' of 'mold-source-map.js'. Please either fix that or report it to the module author

See: thlorenz/mold-source-map#14
This is fixed in version 0.4.1

Test coverage reduced with replacement to #34

@bendrucker I like the new auto-directory creation functionality in the latest commit, but it's worth noting that #34 was focused on preventing errors from being swallowed during the file-writing process, regardless of reason (e.g., permission issue or disk space full, not just non-existent directory). This was done as a precursor to adding the --error-on-missing option in #36.

A consequence of the latest commit is that the test that verified proper error handling during the file-writing process was lost. I'm not sure if it's worth adding it back in (and/or adding a test for failed directory creation), both due to the low frequency of such errors, as well as increased difficulty in testing them, but I figured it was worth noting! :D

Absolute path and local server

Hello.

I cannot manage the case where a local server is serving javscript files and source maps.

Currently, it seems that exorcist (or maybe actually browserify) adds the "local absolute path" to the "sources" properties in the source map file. In my current development environment, but it could be a quite common situation, it could be useful to have an option to set the path according to where the server is actually serving those files, otherwise they remain unreachable.

How do you manage this kind of situation? Am I missing something?

Dramatically increases size of files without source maps

Here I create two bundles, once without --debug and once with, then exorcise them and compare the sizes:

$ `npm bin`/browserify ../../src/js/app.js > bundle.js
$ `npm bin`/exorcist app.map < bundle.js > bundle.ex.js
The code that you piped into exorcist contains no source map!
Therefore it was piped through as is and no external map file generated.
$ ls -lah
total 11M
drwxr-xr-x 2 user Numa 4.0K Mar  9 13:56 .
drwxr-xr-x 3 user Numa 4.0K Mar  9 11:18 ..
-rw-r--r-- 1 user Numa 8.8M Mar  9 13:56 bundle.ex.js
-rw-r--r-- 1 user Numa 2.2M Mar  9 13:56 bundle.js


$ `npm bin`/browserify --debug ../../src/js/app.js > bundle-debug.js
$ `npm bin`/exorcist app.map < bundle-debug.js > bundle-debug.ex.js
$ ls -lah
total 22M
drwxr-xr-x 2 user Numa 4.0K Mar  9 13:57 .
drwxr-xr-x 3 user Numa 4.0K Mar  9 11:18 ..
-rw-r--r-- 1 user Numa 2.6M Mar  9 13:57 app.map
-rw-r--r-- 1 user Numa 2.2M Mar  9 13:57 bundle-debug.ex.js
-rw-r--r-- 1 user Numa 5.6M Mar  9 13:57 bundle-debug.js
-rw-r--r-- 1 user Numa 8.8M Mar  9 13:56 bundle.ex.js
-rw-r--r-- 1 user Numa 2.2M Mar  9 13:56 bundle.js

The bundle without source maps has 2.2M, with source maps 5.6M.
After exorcising, the one without source maps grew to 8.8M, whereas the one with source maps shrank to 2.2M (the size of the original bundle without maps, as expected).

So the bundle without source maps grew by a factor 4 although exorcist claimed that it's just "piping it through".

I couldn't reproduce this on a minimal example so far.

what is .js.map for?

Hi,

Thank you very much for the quick response for my last issue.

I generated me .js.map file. But since it is a big file and the structure is pretty simple, I got last just as when I look at the result of browserify file. I thought .js.map will help me get better understanding of what is going on in .js file generated by browserify, in .js.map, there is only sources, sourcesContent? Could you show me the structure of this file and maybe how to use it please? Thanks. I appreciate it very much.

Add -o arguments

Hi @thlorenz,

How do you think about adding -o arguments for JS content and MAP content, need two arguments.

I m looking for that because i have some troubles to pipe cli stdout / stderr on windows.

Please let me know, so that i will fork or PR depending!

Inline sourcemaps not fully replaced

The source map is created and the soureMappingURL is updated correctly but after that a bunch of base64 is still in the bundle file.

browserify --debug app.js > bundle.js

tail -n1 bundle.js
//#sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbIi...

browserify --debug app.js > bundle2.js | exorcist bundle2.js.map > bundle2.js

tail -n2 bundle2.js
//# sourceMappingURL=bundle2.js.map
uIjozLCJzb3VyY2VzIjpbIi4uLy4uLy5ucG0tcGFja2FnZXMvb...

Don't know if this is an issue within exorcist, mold-source-map or convert-source-map.

Does Exorcist support Browserify 5?

Has anyone gotten Exorcist to work with Browserify 5? I'm getting TypeError: Invalid non-string/buffer chunk errors after upgrading to:

My bundle method looks like this:

bundler = browserify({
  entries: ["#{conf.script_path}/index.coffee"],
  extensions: ['.coffee', '.hbs']
  debug: true
  cache: {}
  packageCache: {}
  fullPaths: true
})

bundle = (bundler, conf) ->
  return bundler
    .bundle()
    .pipe(sourcestream('index.js'))
    .pipe(exorcist("#{conf.script_path_dist}/index.js.map"))
    .pipe(gulp.dest(conf.script_path_dist))
    .on('end', ->
      livereload({auto: false})
    )

I'm using 3 shims:

  • coffeeify
  • browserify-shim
  • hbsfy

Taking a writeable stream rather than file path

it would be really useful if exorcist could take a writeable stream instead of a file path.
I am not sure how you feel about arg that could be either a string or a stream, so another option would be to add another method. (for example mkdirp also exports mkdirp.sync)
I'd be happy to submit a PR to do this if you think it would be a useful addition.

TypeError: Arguments to path.resolve must be strings

This works:

gulp.task('exorcist-test', function() {
    browserify({ debug: true })
        .add("intl")
        .bundle()
        .on('error', handleErrors)
        .pipe(source('appPolyfills.js'))
        .on('error', handleErrors)
        .pipe(exorcist("testmap"))
        .on('error', handleErrors)
        .pipe(fs.createWriteStream(path.join(__dirname, 'bundle.js'), 'utf8'));
});

But this does not:

gulp.task('exorcist-test', function() {
    browserify({ debug: true })
        .add("intl")
        .bundle()
        .on('error', handleErrors)
        .pipe(source('appPolyfills.js'))
        .on('error', handleErrors)
        .pipe(exorcist("testmap"))
        .on('error', handleErrors)
        .pipe(gulp.dest('./build'));
});
path.js:439
      throw new TypeError('Arguments to path.resolve must be strings');
      ^
TypeError: Arguments to path.resolve must be strings
    at Object.posix.resolve (path.js:439:13)
    at DestroyableTransform.saveFile [as _transform] (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/lib/dest/index.js:36:26)
    at DestroyableTransform.Transform._read (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at DestroyableTransform.Transform._write (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
    at writeOrBuffer (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)
    at DestroyableTransform.Writable.write (/home/sebastien/Desktop/Stample-react/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:194:11)
    at Stream.ondata (stream.js:51:26)
    at Stream.emit (events.js:107:17)
    at drain (/home/sebastien/Desktop/Stample-react/node_modules/exorcist/node_modules/mold-source-map/node_modules/through/index.js:35:16)

I have tried with 2 different versions of NodeJS 0.10.24 and 0.12.5, both throw the error.

Notice that I have found other issues related in other projects that use the "through" stream library.
See BlessCSS/gulp-bless#2

./generateAndvalidate_one.js.map did not generated

Hi,
why is ./generateAndvalidate_one.js.map not generated?

var fs = require('fs');
var path = require('path');
var exorcist = require('exorcist');
var browserify = require('browserify');
var mapfile = path.join(__dirname, './generateAndvalidate_one.js.map');
var b = browserify();
b.add('./GenerateAndValidate.js');
b.bundle().pipe(exorcist(mapfile))
.pipe(fs.createWriteStream(path.join(__dirname, './generateAndvalidate_one.js'), 'utf8'));

Output file name overwritten by generated.js

When using CLI, generated source maps file property points to generated.js even though command given should output to some other file.

For example:

node_modules/.bin/browserify ./foo.js --debug | node_modules/.bin/exorcist bundle.js.map > foo.o1.js

npm doesn't render the README properly

npm displays the html tags <code></code> in the first sentence of the readme instead of showing a code block like it should.

screen shot 2016-03-06 at 6 42 01 am 1

I'm guessing the leading period trips it up somehow. I have some ideas for how to work around it but I don't have access to see my changes as I make them so I figured I'd make an issue instead of a pull request.

Add newline to output

The fact that no newline is added to the output after the sourceMappingURL comment, results in a Parse Error when that module is used in PhantomJS. So obviously it's not really an issue of exorcist, but in general it is good practice to have a newline at the end of a file. Browserify itself also keeps/adds it (debug mode or not).

The thing is, we could simply add it:

return { json: json, src: newSrc + '\n' + comment + '\n' }

But it might be a problem to some, theoretically (not sure how). Anyway, I'm happy to submit a PR (with updated tests). Should we just change current behavior or make it an option?

Error: bundle() no longer accepts option arguments.

Hi,

I ran this in a script called index.js and got errors below:

var browserify = require('browserify')
, path = require('path')
, fs = require('fs')
, exorcist = require('exorcist')
, mapfile = path.join(__dirname, './generateAndvalidate_one.js.map');

browserify()
.require(require.resolve('./GenerateAndValidate.js'), { entry: true })
.bundle({ debug: true })
.pipe(exorcist(mapfile))
.pipe(fs.createWriteStream(path.join(__dirname, './generateAndvalidate_one.js'), 'utf8'));

/usr/local/bin/node index.js
/Users/yingyang/Documents/JavaScript/node_modules/browserify/index.js:762
throw new Error(
^

Error: bundle() no longer accepts option arguments.
Move all option arguments to the browserify() constructor.
at Browserify.bundle (/Users/yingyang/Documents/JavaScript/node_modules/browserify/index.js:762:15)
at Object. (/Users/yingyang/Documents/JavaScript/performance/browserify/index.js:9:3)
at Module._compile (module.js:541:32)
at Object.Module._extensions..js (module.js:550:10)
at Module.load (module.js:458:32)
at tryModuleLoad (module.js:417:12)
at Function.Module._load (module.js:409:3)
at Module.runMain (module.js:575:10)
at run (node.js:348:7)
at startup (node.js:140:9)

could you tell me if your example works or not, or where I am wrong using it please? Many thanks.

Does not fail correctly

When there is an error in browserify, exorcist swallows it and doesn't exit with an error code. For example, if you have a JavaScript error in your source code, browserify will exit with an exit code of 1. This is correct behavior when there is an error and works with make and build systems to know it failed.

When piping to exorcist, the JS error is swallowed and an exit code of 0 for success is returned. So build systems assume the build was successful, but no file is created.

The log to the console is:

The code that you piped into exorcist contains no source map!
Therefore it was piped through as is and no external map file generated.

Exorcist should exit with an exit code of 1 if the code is empty. This will allow to fail silently if there is no map, but not fail silently if there is no javascript (a sure sign that there was an error upstream).

This issue is regarding command-line usage.

browserify main.js --debug | exorcist bundle.js.map > bundle.js

proposal: add an option to completely remove the mapfile reference at the end of the js files.

hey @thlorenz, what do you think about such a feature, if you think it should be supported by exorcist I can finish it and open a PR?

https://github.com/sinan/exorcist/commit/58063af17e70490eac59a67d045d00a1fb080d2a

This is useful when you want to use your sourcemaps in production with an external service such as rollbar, errorception, etc. but you want your production code to not include mappings to avoid 404s in the browser console.

What's the reasoning for automatically changing sourceRoot?

What's the reasoning for doing this:

var json = inlined
  .setProperty('sourceRoot', root || '')
  .toJSON(2);

Why would sourceRoot necessarily change because an inline source map is being extracted? sourceRoot is for the user agent to use in combination with sources values to request the source files, right?

I was using exorcist to help rapidly debug issues with, wait for it...sourceRoot output in traceur and exorcist was obscuring the output from traceur (so I had to stop using it for that).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.