Giter VIP home page Giter VIP logo

cache-loader's Introduction

DEPRECATED

Consider upgrading webpack to version 5 and setup cache https://webpack.js.org/configuration/other-options/#cache


npm node deps tests coverage chat size

cache-loader

The cache-loader allow to Caches the result of following loaders on disk (default) or in the database.

Getting Started

To begin, you'll need to install cache-loader:

npm install --save-dev cache-loader

Add this loader in front of other (expensive) loaders to cache the result on disk.

webpack.config.js

module.exports = {
  module: {
    rules: [
      {
        test: /\.ext$/,
        use: ['cache-loader', ...loaders],
        include: path.resolve('src'),
      },
    ],
  },
};

⚠️ Note that there is an overhead for saving the reading and saving the cache file, so only use this loader to cache expensive loaders.

Options

Name Type n Default Description
cacheContext {String} undefined Allows you to override the default cache context in order to generate the cache relatively to a path. By default it will use absolute paths
cacheKey {Function(options, request) -> {String}} undefined Allows you to override default cache key generator
cacheDirectory {String} findCacheDir({ name: 'cache-loader' }) or os.tmpdir() Provide a cache directory where cache items should be stored (used for default read/write implementation)
cacheIdentifier {String} cache-loader:{version} {process.env.NODE_ENV} Provide an invalidation identifier which is used to generate the hashes. You can use it for extra dependencies of loaders (used for default read/write implementation)
compare {Function(stats, dep) -> {Boolean}} undefined Allows you to override default comparison function between the cached dependency and the one is being read. Return true to use the cached resource
precision {Number} 0 Round mtime by this number of milliseconds both for stats and dep before passing those params to the comparing function
read {Function(cacheKey, callback) -> {void}} undefined Allows you to override default read cache data from file
readOnly {Boolean} false Allows you to override default value and make the cache read only (useful for some environments where you don't want the cache to be updated, only read from it)
write {Function(cacheKey, data, callback) -> {void}} undefined Allows you to override default write cache data to file (e.g. Redis, memcached)

Examples

Basic

webpack.config.js

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: ['cache-loader', 'babel-loader'],
        include: path.resolve('src'),
      },
    ],
  },
};

Database Integration

webpack.config.js

// Or different database client - memcached, mongodb, ...
const redis = require('redis');
const crypto = require('crypto');

// ...
// connect to client
// ...

const BUILD_CACHE_TIMEOUT = 24 * 3600; // 1 day

function digest(str) {
  return crypto
    .createHash('md5')
    .update(str)
    .digest('hex');
}

// Generate own cache key
function cacheKey(options, request) {
  return `build:cache:${digest(request)}`;
}

// Read data from database and parse them
function read(key, callback) {
  client.get(key, (err, result) => {
    if (err) {
      return callback(err);
    }

    if (!result) {
      return callback(new Error(`Key ${key} not found`));
    }

    try {
      let data = JSON.parse(result);
      callback(null, data);
    } catch (e) {
      callback(e);
    }
  });
}

// Write data to database under cacheKey
function write(key, data, callback) {
  client.set(key, JSON.stringify(data), 'EX', BUILD_CACHE_TIMEOUT, callback);
}

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          {
            loader: 'cache-loader',
            options: {
              cacheKey,
              read,
              write,
            },
          },
          'babel-loader',
        ],
        include: path.resolve('src'),
      },
    ],
  },
};

Contributing

Please take a moment to read our contributing guidelines if you haven't yet done so.

CONTRIBUTING

License

MIT

cache-loader's People

Contributors

alexander-akait avatar evilebottnawi avatar gusvargas avatar joshwiens avatar jsg2021 avatar michael-ciniawsky avatar mistic avatar mordred avatar munter avatar sokra avatar whxaxes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cache-loader's Issues

Should source maps be cached?

I'm trying to help debug an issue with using Webpack and Karma together. I'm thinking that it could be resolved if each cached module contained its inline source map (when devtool is "inline-source-map") to prevent Webpack from having to reevaluate. Would this make sense for cache-loader?

Cache TTL to assests

I would like to set my Cache TTL for my assests to 1 year by recommendation from Google PageInsights.
But I dont see in the docs where to do this. Im using webpack on my react client.

feat: add cache invalidation (`options.?`)

As pointed out in #3, it's important to be able to expire the cache for this loader when the build environment changes. We're currently doing this by naming the cache directory based on a hash of relevant file content (NPM shrinkwrap, webpack config, etc) - however, over time this means developers accumulate a bunch of old cache directories, which need to be cleaned up manually or in a separate build step.

Would it be possible to add a cache-key option that, when changed, would cause the loader to just ignore and overwrite existing cache files?

Less output not getting updated

cache-loader version: 1.0.3

Relevant rule:

{
   test: /\.less$/,
   use: extractStyles.extract([
       'cache-loader',
       'css-loader',
      'less-loader'
   ])
}

Just added cache-loader and am finding that the output of my project less files doesn't get updated. Ran into it on multiple machines. No errors reported. It must not be invalidating the cache appropriately in this case. Works fine with JS and TS.

usage with eslint-loader, warnings/errors are swallowed after successful compilation

  • Operating System: OS X
  • Node Version: 11.12.0
  • NPM Version: 6.7.0
  • webpack Version: 4.30.0
  • cache-loader Version: 3.0.0

Expected Behavior

with eslint-loader set up to not fail on errors/warnings, and if cache loader is used with babel-loader, warnings/errors should show up consistently.

  • on initial startup, should see errors/warnings
  • after stopping, then starting again, should see the same errors

Actual Behavior

with eslint-loader set up to not fail on errors/warnings, and if cache loader is used with babel-loader, warnings/errors should show up consistently.

  • on initial startup, see errors/warnings
  • after stopping, then starting again, the same errors/warnings are not shown

Code

 {
        enforce: 'pre',
        test: /\.(ts|tsx)$/,
        include: [path.join(cwd, 'src/js'), path.join(cwd, '../shared/src/js')],
        loader: 'eslint-loader',
        options: {
          configFile: path.resolve(cwd, './.eslintrc.js'),
          emitWarning: true,
          failOnWarning: false,
        },
      },
      {
        include: [path.join(cwd, 'src/js'), path.join(cwd, '../shared/src/js')],
        test: /\.(ts|tsx)$/,
        use: [
          {
            loader: 'cache-loader'
          },
          {
            loader: 'babel-loader',
            options: {
              cacheDirectory: true,
              cacheCompression: false,
              compact: false,
              babelrc: false,
              plugins: ['@babel/plugin-syntax-dynamic-import'],
              presets: ['babel-preset-tock'],
            },
          },
        ],
      },

How Do We Reproduce?

introduce a warning from linting, such as no-unused-variables, startup webpack with webpack dev server, and see the warning.
stop webpack dev server, then start again. you will not see the error.

Notes

Without cache-loader, the expected behavior above does happen. i'm not sure if the issue is with cache-loader, or with eslint-loader

webpack-contrib/eslint-loader#267

Cache warning and errors from loader

There some loaders, (for example tslint-loader), which only emit warnings or errors. It would be useful to cache and re-emit that stuff with cache-loader.

cache files are randomly corrupted

Hi,
I'm using cache-loader v1.0.3, and I love it for it saves a lot of webpack build time. (~1min)
The problem is that coworkers randomly experience build error of Empty files. It happens from our build machines a lot randomly, so we stopped using it recently. Can you find the root cause of it? Stack trace is as below:

Empty files:
ERROR in node_modules/.cache-loader-cache/1ab816c016e1bd0ed02e3712897f9d0d.json:1
1: 
^ Unexpected end of input
node_modules/.cache-loader-cache/46d561d9dbc4d7b6bb43238ffd0fea50.json:1
1: 
^ Unexpected end of input
node_modules/.cache-loader-cache/879d8d4764e11a424b407b0c5eb9b921.json:1
1: 
^ Unexpected end of input
node_modules/.cache-loader-cache/ec76dc763d9fbc22c5e4cc47972e2038.json:1
1: 
^ Unexpected end of input
node_modules/.cache-loader-cache/ed921eb0045f85051790b9a0c17f6868.json:1
1: 
^ Unexpected end of input
node_modules/.cache-loader-cache/fbb669036963f03d234002a129f75639.json:1
1: 
^ Unexpected end of input

Module build failed: TypeError: Cannot read property 'async' of undefined at react-app-dashboard/node_modules/babel-loader/lib/index.js:50:27

Uing nwb for creating library, so here I want to use webpack and babel for some additional packaging functionality.

//package.json
{
"name": "react-app-dashboard",
"version": "1.0.0",
"description": "Describe react-app-dashboard here",
"private": true,
"scripts": {
"build": "nwb build-react-app",
"clean": "nwb clean-app",
"start": "nwb serve-react-app",
"test": "nwb test-react",
"test:coverage": "nwb test-react --coverage",
"test:watch": "nwb test-react --server"
},
"dependencies": {
"@babel/core": "^7.2.2",
"react": "^16.7.0",
"react-dom": "^16.7.0"
},
"devDependencies": {
"babel-core": "^6.26.3",
"babel-loader": "^8.0.4",
"babel-preset-env": "^1.7.0",
"babel-preset-react": "^6.24.1",
"cache-loader": "^1.2.2",
"webpack": "^4.28.3"
},
"author": "",
"license": "MIT",
"repository": ""
}

// nwb.config.js

module.exports = {
type: 'react-app',
babel: {
cherryPick: 'some-module',
plugins: ["babel-loader", "babel-preset-env", "babel-preset-react"]
},
webpack: {
publicPath: '',
extra: {
entry: './src/index.js',
output: {
filename: 'main.js',
path: path.resolve(__dirname, 'dist')
}
}
},
}

Hard to set custom cacheIdentifier that uses cache-loader version number

The default value for cacheIdentifier is cache-loader:{version} {process.env.NODE_ENV}. My project has multiple webpack config files where the step that uses cache-loader gets the same input from previous loaders but performs different transformations (eg. Istanbul instrumentation for test builds only).

I like that the default string includes the cache-loader version number, since it makes sense to invalidate the cache when this loader is updated. When setting a custom value, it is difficult to replicate this behavior since retrieving the cache-loader version number is tricky. It seems like the best options are hard-coding the version number into the string and hoping to remember to update it when upgrading to a new version of cache-loader, or some hacky manipulation of the path returned by require.resolve("cache-loader").

Couple of possible fixes:

  1. Append the user-provided cacheIdentifier to the default string instead of replacing the default string
  2. Export the cache-loader version number

Happy to submit a PR with whatever fix is deemed best!

Use hashed directory structure

Currently all outputs are retrieved from a single directory. For large projects and certain hardware this might have a negative impact on performances.

Something that might help with that would be to store outputs into subfolders that contain the first two letters of the hash.

Running multiple processes

I would like to know whether there are any unintended consequences from using cache-loader from multiple processes writing into the same cache directory?

To give a bit of a background, I would like to create a manager for my project that would handle requests for building checked out project sources in parallel. At the moment, I am constructing a strong hash from the environment for all the loader dependencies that's added as a component to the cache output path.

I am curious if there would be any issues when I try to run two parallel builds when the environments are not different (cache output directories are the same), but my project sources changed? If so, perhaps this is something that might be added to the README.

feat: add cache invalidation

This is a really great idea. However, there is an important issue that should be mentioned before anyone uses this in a production environment. At the very least there should be a big bold warning in the readme. Currently, this loader is assuming that the output of all affected loaders is going to remain the same forever for any given input. But updating packages from npm, or modifying options for the loaders can easily change this. To make matters even worse, some loaders read from configuration files for their options (e.g .babelrc). Basically, anything in the build environment that can potentially change the output of a loader needs to be included in the cache key. This is hard to solve generically for all loaders, because you don't know what a loader could be using that could affect its output.

A partial (but not easily usable) solution is to intelligently include what we can in the cache key (e.g webpack version and loader version/options for each cached loader). And then anything else will have to be manually added by the user.

Let's take babel-loader for example. It actually already has its own disk caching, and according to the readme their cache key by "default is a string composed by the babel-core's version, the babel-loader's version, the contents of .babelrc file if it exists and the value of the environment variable BABEL_ENV with a fallback to the NODE_ENV environment variable." So the user would have to tell cache-loader about the babel-core version, .babelrc contents and BABEL_ENV/NODE_ENV.

So this is not so great. The actual good solution to this problem would be to make disk cached loaders a standard thing within webpack. Then loader developers would be responsible for telling webpack about their cache dependencies and the user wouldn't have to do anything. That is the dream!

cache-loader causes imagemin-webpack-plugin to not generate the images

module:{
	rules:[
		{
			test:/\.jsx?$/,
			include:[srcPath],
			use:[
				{
					loader:'cache-loader',
				},{
					loader:'babel-loader',
				}
			]
		},
		{
			test: /\.(css|scss)$/,
			include: [srcPath],
			use: ExtractTextPlugin.extract({
				fallback: 'style-loader',
				use: ['cache-loader','css-loader', 'sass-loader']
			})
		},
		{
			test:/\.(png|jpe?g|gif|svg)(\?.*)?$/,
			include:[srcPath],
			use:[
				{
					loader:'cache-loader'
				},{
					loader:'url-loader',
					options:{
						publicPath:'/',
						limit: 10000,
						name: utils.resourcePath('img/[name].[hash:7].[ext]')
					},
				}
			]
		},
		{
			test:/\.(mp4|webm|ogg|mp3|wav|flac|aac)(\?.*)?$/,
			include:[srcPath],
			use:[
				{	
					loader:'cache-loader'
				},{
					loader:'url-loader',
					options:{
						publicPath:'/',
						limit: 10000,
						name: utils.resourcePath('media/[name].[hash:7].[ext]')
					},
				}
			]
		},
		{
			test:/\.(woff2?|eot|ttf|otf)(\?.*)?$/,
			include:[srcPath],
			use:[
				{
					loader:'cache-loader'
				},{
					loader:'url-loader',
					options:{
						publicPath:'/',
						limit: 10000,
						name: utils.resourcePath('fonts/[name].[hash:7].[ext]')
					},
				}
			]
		}
	]
},
Hash: c593f2eeb3a74cc82923
Version: webpack 3.5.5
Time: 40972ms
                                Asset       Size  Chunks                    Chunk Names
  ./js/vendor.26bfd90b6f42c7d80bfa.js    1.18 MB       0  [emitted]  [big]  vendor
   ./js/index.6ae9e86f6a638e7fe8be.js     876 kB       1  [emitted]  [big]  index
./js/manifest.92cf23c7c278d9152237.js    1.45 kB       2  [emitted]         manifest
               css/index-c593f2ee.css     342 kB       1  [emitted]  [big]  index
                           index.html  510 bytes          [emitted]

No picture

No files emitted with responsive-loader

  • Operating System: WSL 1 (Bash on Windows)
  • Node Version: v8.14.0
  • NPM Version: 6.14.1
  • webpack Version: 3.10.0
  • cache-loader Version: 4.1.0

Expected Behavior

Image files emitted by the responsive-loader should correctly cached or at least be written to disk.

Actual Behavior

All image files emitted by the responsive-loader are missing, they are not written as files to disk.

Code

// webpack.config.js
let webpackConfig = {
  module: {
    rules: [
      {
        test: /\.(jpe?g|png)$/i,
        use: [{
          loader: 'cache-loader',
        }, {
          loader: 'responsive-loader',
          options: {
            adapter: require('responsive-loader/sharp'),
            quality: 100, // (default)
            name: '[path][name]-[width].[ext]',
          },
        }],
      },
  [...]
}

How Do We Reproduce?

webpack setup with responsive-loader and cache-loader.
First leave the cache-loader disabled and let the responsive-loader resize some images (e.g. during CSS build). Notice that the resized images are correctly written to disk as files.
Now enable the cache-loader, run the build again. Notice that these files are not written to disk anymore, they are completely missing. Also inspect the cache folder and notice there are cache files for these resized images, but too small for an image file and they contain similar base64 encoded payload as above.

Additional notes

The cache files are very small for image files, the base64-decoded payload looks like this:

module.exports = {srcSet:__webpack_public_path__ + "images/layout/header-1557.jpg"+" 1557w",images:[{path:__webpack_public_path__ + "images/layout/header-1557.jpg",width:1557,height:743}],src:__webpack_public_path__ + "images/layout/header-1557.jpg",toString:function(){return __webpack_public_path__ + "images/layout/header-1557.jpg"},placeholder: undefined,width:1557,height:743};

No binary data of the actual image files can be found.

Control `cacheDirectory` with env variable

Feature Proposal

Add an option to control the cacheDirectory with an env variable (e.g. CACHE_LOADER_CACHE_DIR).

Feature Use Case

Useful for keeping the default for local dev environment and controlling the value on CI/CD.

The cacheDirectoryoption should still take precedence if specified.

memory leak when using different splitchunks optimizations options

  • Operating System: MacOsX
  • Node Version: 12.4.0
  • NPM Version: 6.14.2
  • webpack Version: 4.42.0
  • cache-loader Version: 4.1.0

Expected Behavior

it should not cause memory leak

Actual Behavior

it causes memory leak

Code

cache-loader babel-loader usage

{
        test: /\.(js|jsx|ts|tsx)?$/,
        exclude: [/node_modules/],
        // use: 'happypack/loader?id=js',
        use: ['cache-loader', 'babel-loader?cacheDirectory'],
        include: [srcPath, path.join(cwd, '../')],
      },

one optimization option that causes memory leak

optimization: {
     removeAvailableModules: false,
    removeEmptyChunks: false,
    splitChunks: false,
}

another optimization option that causes memory leak (create 1 chunk per monorepo package)

const outModulesPrefix = 'entria';
const ourModulePattern = new RegExp(`^@${outModulesPrefix}\/([a-z]+[-[a-z]+]*)$`);

splitChunks: {
      cacheGroups: {
        default: false,
        vendors: false,
        vendor: {
          name: 'vendor',
          chunks: 'all',
          test: /node_modules/,
          priority: 20,
        },
        common: {
          name: module => `feedback-${module.rawRequest.match(ourModulePattern)[1]}`,
          minChunks: 1,
          chunks: 'all',
          maxAsyncRequests: Infinity,
          maxInitialRequests: Infinity,
          minSize: 0,
          maxSize: 1,
          test: module => ourModulePattern.test(module.rawRequest),
          priority: 10,
          enforce: true,
        },
      },
    },

How Do We Reproduce?

try to use this babel-loader config with this optimization settings in a big codebase

invalidate with cache

  • Operating System: MacOS
  • Node Version: 10
  • NPM Version: 6
  • webpack Version: 4
  • cache-loader Version: latest

Feature Proposal

Serialize this.loaders options to invalidate cache when things change.
The remaining request looks like this:

file??ref--8-0!file??ref--8-1!file??ref--8-2!file

It is not clear if those refs change when loader options change.

Feature Use Case

As far as I can tell, cache-loader does not account for changes in options passed to loaders in the chain.

Do not rely on file `mtime`

npm 5.8+ set file mtime to October 1985 (See npm/npm#20439 ), it's a feature for them, so the cache-loader does not work anymore.

Furthermore, f24f723 tells to not trust file mtime.

Should we relly on file content hash instead ?

Absolute paths in cache.

Hello!

We've been continuing to tune our build performance, and are happy to see that cache-loader seems to perform slightly better than babel-loader's cache (and are definitely happy to see that it applies to other loaders as well).

We're seeing an issue with it though in that it stores absolute paths in the cache files, which means that the cache cannot be shared across our build workers. I was hoping that I could correct the paths in the cache files directly, but it seems that the cacheKey used for naming the files is also generated using absolute paths.

This means that it's totally unusable for us in CI, which is where I think the primary benefit will be seen.

Any idea if this would be possible to correct?

Should cache-loader use `module.exports.raw = true` to handle binary file ?

  • Operating System: MacOS
  • Node Version: 11.0.0
  • NPM Version: 6.9.0
  • webpack Version: 4.30.0
  • cache-loader Version: 2.0.1

Expected Behavior

// webpack.config.js
const path = require("path");

module.exports = {
  mode: "development",
  entry: "./src/index.js",
  output: {
    path: path.join(__dirname, "dist"),
    filename: "index.js"
  },
  module: {
    rules: [
      {
        test: /\.png$/,
        use: ["file-loader", "cache-loader"]
      }
    ]
  }
};
// index.js
import png from "./index.png";

console.log(png);

In this case. The .png is broken because cache-loader is not a raw loader. Which means the source is an UTF-8 string instead of Buffer. The binary data will be broken during conversion processed.

Some loaders, e.g. file-loader, added module.exports.raw = true to solved this issue.

Actual Behavior

.png file broken.

How Do We Reproduce?

cc: @jimexist

Change default cache directory

  • Operating System: macOS 10.14.3
  • Node Version: 10.15.2
  • NPM Version: 6.4.1
  • webpack Version: 4.29.5
  • cache-loader Version: 2.0.1

Feature Proposal

Instead of ./cache-loader, it should be ./node_modules/.cache/cache-loader

Feature Use Case

Consistent with other loaders and to avoid a specific gitignore.

using with file-loader, files are not builded if in cache, even if dist folder is empty

  • Operating System: ubuntu 16.04
  • Node Version: 6.14.3
  • NPM Version: 3.10.10
  • webpack Version: 4
  • cache-loader Version: 3.0.1

Expected Behavior

file-loader outputs all to assets folder.
cache-loader used with file-loader.
While using with file-loader, if output directory does not content assets folder — get those files from cache and put in assets folder.

Actual Behavior

Webpack doesn't output assets folder, that shoul be generated by file-loader.

Code

        {
          test: /.(jpg|jpeg|png|svg|gif|ttf|otf|eot|woff|woff2)$/,
          use: [
            {
              loader: 'cache-loader'
            },
            {
              loader: 'file-loader',
              options: {
                outputPath: 'assets',
                name: isDevelopment ? '[path][name].[ext]' : '[hash].[ext]'
              }
            }
          ]
        }

How Do We Reproduce?

Add cache-loader before file-loader.

Phrasing "in front of" confusing?

The cache-loader readme mentions that the cache-loader needs to be added in front of other loaders:

Add this loader in front of other (expensive) loaders to cache the result on disk.

I find the phrasing "in front of" confusing, since the loaders are used right from left / last to first (use docs). So Looking at the code examples they should be used last, but added in front of the use configuration?

Exception: Cannot read property 'stat' of undefined

  1. Latest version: 1.2.3
  2. Works fine if downgraded to 1.2.2 (via explicit dev-dependency)
  3. The error occurs during vue-service-cli build for a project

Please check the declaration of module dependency.

Module build failed (from ./node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 10)
 Cannot read property 'stat' of undefined
     at toDepDetails (C:\agent\_work\2\s\node_modules\cache-loader\dist\index.js:65:14)
     at arrayIterator (C:\agent\_work\2\s\node_modules\neo-async\async.js:3780:9)
     at timesSync (C:\agent\_work\2\s\node_modules\neo-async\async.js:2292:7)
     at Object.mapLimit (C:\agent\_work\2\s\node_modules\neo-async\async.js:3775:5)
     at Array.<anonymous> (C:\agent\_work\2\s\node_modules\cache-loader\dist\index.js:88:18)
	 at arrayEachFunc (C:\agent\_work\2\s\node_modules\neo-async\async.js:2512:19)
     at Object.parallel (C:\agent\_work\2\s\node_modules\neo-async\async.js:6867:9)
     at Object.loader (C:\agent\_work\2\s\node_modules\cache-loader\dist\index.js:87:9)

cache-loader vs hard-source-webpack-plugin

Hi, what are the differences between this loader and the hard-source-webpack-plugin?

Also, will the cacheloader speed up paralell webpacks? I'm running webpack-dev-server and karma-webpack at the same time and am looking for something that can help speed up the two.

Regards,
Tarjei

.cache-loader file make mistake

i use git for my project . when i checkout one branch to another. it got error in my npm start. but when i deleted this file, i restarted successfully. what can i do?

Using relative paths - sources still contains absolute paths

  • Operating System: OSX 10.14.4
  • Node Version: v11.12.0
  • NPM Version: 6.7.0
  • webpack Version: 4.28.4
  • cache-loader Version: 2.0.1

Hi,

I know the relative paths has been a fairly recent feature addition (thank you!)... but I noticed that the cache contents still contain absolute paths.

This sets the correct relative paths in the cache details, but then I see things like this inside the cache file, under the "results":[...] object:

Expected Behaviour

...,{"version":3,"sources":["src/js/...

Is it possible to parse these "sources" as relative paths too - so that they context stays the same?

This is for javascript files running through uglifyjs-webpack-plugin (not sure if that is relevant or not).

Actual Behaviour

...,{"version":3,"sources":["/Users/jason/Sites/...

Code

module.exports = (env, argv) => {

    return {
        mode: argv.mode,
        ...
        module: {
            rules: [
                {
                    {
                        test: /\.m?js?$/,
                        exclude: /(node_modules|bower_components)/,
                        use: [{
                            loader: 'cache-loader',
                            options: {
                                cacheContext: path.normalize(path.relative(__dirname, '.')),
                                cacheDirectory: 'cache/'
                        }, {
                            loader: 'babel-loader'
                        }]
                    }
                }
            ]
        }
    }
}

How Do We Reproduce?

Happens in both modes development and production.

Command been using: webpack --mode production

Use relative path on this.fs.stat returns stale metadata

  • Operating System: macOS 10.14.5
  • Node Version: 10.15.2
  • NPM Version: 6.9.0
  • webpack Version: 4.23.1
  • cache-loader Version: 4.0.0

Expected Behavior

When using cache-loader configured with cacheContext under a webpack in watch and running a development configuration, it was expected to have the final bundles always in sync with the changed source files.

Actual Behavior

In the above mentioned cache-loader version when cacheContext is used some relative paths are being used when calling this.fs.stat. That call would sometimes return wrong stale metadata and the cache-loader would return the cached results for that file instead of the ones newly changed on disk.

How Do We Reproduce?

Just use the mentioned cache-loader version in a simple application configured with webpack in watch mode running a development configuration. Start changing the source files and you'll see that sometimes, despite having webpack running the compilation, the bundles are not updated correctly with the last changes on disk.

This is a bug fixed by #83 but we might want to further investigate the cause for this directly in webpack/webpack dependencies that's why I'm opening that issue.

/CC @evilebottnawi

SASS and `@import` dependencies not busting

  1. Version 1.2.0
  2. I cannot seem to produce this on a vanilla project. I'm still trying though.
  3. (My computer) Node version 6.11.4, OSX 10.12.6 -- this actually happens across many computers in my office, Sierra and High Sierra

Here's how I can reproduce this in my private repository:

I have a dependency chain (which is convoluted due to legacy reasons):

  • application.html.erb
  • _internal_css.html.erb
  • app/assets/stylesheets/internal-base.scss (which is processed via Webpack and its bundled path available in Rail ERB partials).
  • app/assets/stylesheets/internal/base/index.scss
  • app/assets/stylesheets/internal/base/shared.scss

Here are the relevant contents (with the issue) of my stylesheets:

app/assets/stylesheets/internal-base.scss:

... a bunch of imports (~38)
@import "./internal/base/index";

app/assets/stylesheets/internal/base/index.scss:

... ~8 imports
@import "./shared";
... ~4 imports

app/assets/stylesheets/internal/base/shared.scss:

... ~7 imports

... ~2000 lines of SASS

Any changes to the shared.scss will not bust the cache! Changes to base/index.scss will also not bust the cache! However, changes to internal-base.scss will bust the cache!

In order for changes to get picked up in shared.scss, I have to provide an the .scss extension in the @import statement for internal-base.scss. That is the only change to get it tracked well by cache-loader. base/index.scss remains the same -- which is a bit odd.

I'm still trying to reproduce in a vanilla repository @ https://github.com/juanca/cache-loader-sass-bug

It might have to do with my webpack configuration in my private repository... I can provide snippets from it as requested.

Anyone had similar problems to these? Or are these known issues?

custom write and read are much slower than the default

I was trying to speed up the cache read/write, so I decided to store stuff in memory with node-cache, instead of in the filesystem (default behavior).

To my surprise, the build time is slower with in-memory cache than with filesystem, it's so slow even a build without cache-loader is faster.

node-cache has pretty much the same function signatures as the example provided in the docs here for Redis. so my implementation is basically carbon copy of that

const NodeCache = require( "node-cache" );
const myCache = new NodeCache();
const BUILD_CACHE_TIMEOUT = 24 * 3600; // 1 day
// Read data from database and parse them
function read(key, callback) {
  myCache.get(key, (err, result) => {
    if (err) {
      return callback(err);
    }

    if (!result) {
      return callback(new Error(`Key ${key} not found`));
    }

    try {
      let data = JSON.parse(result);
      callback(null, data);
    } catch (e) {
      callback(e);
    }
  });
}

// Write data to database under cacheKey
function write(key, data, callback) {
  myCache.set(key, JSON.stringify(data), BUILD_CACHE_TIMEOUT, callback);
}


...
module.exports = {
  // rest of config
  use: [
          {
            loader: "cache-loader",
            options: { read, write }
          },
          { loader: "babel-loader"},
          { loader: "ts-loader" }
        ]
}

My project is quite massive, hundreds of tsx files that need both babel and ts loaders (which is why I wanted to do caching).

Incremental build times:

  • Without cache-loader: 30s
  • With cache-loader and default read/write: 20s
  • With cache-loader and custom read/write to memory: 50s

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.