Webpacker compilation takes more than 1.5 hours about webpacker HOT 11 CLOSED

What version of Webpacker and Rails?

What is your JS directory structure?

from webpacker.

webpacker (4.3.0)
rails (6.0.3.5)

app/javascript
    react-configs/
    main_react_apps/

webpacker.yaml

source_path: app/javascript
source_entry_path: packs
public_root_path: public
public_output_path: assets
cache_path: tmp/cache/webpacker

inside packs we have this piece of code

import xxxconfig from '../react-configs/xxxconfig';
ReactOnRails.register({
    ...xxxconfig,
});

xxxconfig.js

const lessons = {
    "main_react_apps_1": require('main_react_apps/smthing/smthing/smthing/smthng').default,
    "main_react_apps_2": require('main_react_apps/smthing/smthing/smthing/smthng').default,
    ...
    ..
  }

Is that clear ?
@rossta

from webpacker.

+1 Same Problem

from webpacker.

Hard to say what’s going on with limited info.
If your build is taking 1.5hrs then some is either wrong or ill-matched with the setup or you have have thousands of source files (do you?)

Generally for slow builds, I’d add the following:

One suggestion is to upgrade Webpacker as it does look like you’re behind a few versions with v6 about to release.

You didn’t show what’s in your packs directory but if you have many files in there, that will definitely cause a slow build.

This article provides some debugging tips including how to use the speed-measure-webpack-plugin to identify a slow step—could be that disabling certain plugins in development could help with the local dev ex.

from webpacker.

@rossta
Our packs folder has 5 js files, I included in my last comment the main file that has include most of the files.
I tried upgrading my webpacker before and didn't work.
and yes we have huge amount of assets, I would say maybe almost a thousand.

My main question is about how caching work, does it check if all assets are the same or it checks only modified/deleted/new files ?
as far as I see it only checks if the all the files are the same it skips compile step, but if only one file is edited or added it re compiles everything.

from webpacker.

@karimelghazouly, first, can you try updating to the latest RC release?

Next, is your issue only with production or development as well.

You should be using hot-reloading with webpack for development work, per this example: https://github.com/shakacode/react_on_rails_tutorial_with_ssr_and_hmr_fast_refresh.

For production, very old versions of webpack and Node would clearly hurt.

My main question is about how caching work, does it check if all assets are the same or it checks only modified/deleted/new files ?

Caching? There is none. You're probably referring to the saved digest value that determines if recompilation should run during development. In general, don't use that as it's crazy slow. Use the webpack-dev-server or webpack --watch.

from webpacker.

Please open this as a discussion here: https://discuss.rubyonrails.org/c/webpacker/10

from webpacker.

We're saving issues for bug reports and features.

from webpacker.

@justin808 we are using hot reloading for development, my main issue is in production where we need a precompilation.
I think webpack support such behavior using caching as mentioned here
https://webpack.js.org/configuration/cache/

webpack-dev-server or webpack --watch is for development only

from webpacker.

@justin808 I believe this is an issue, as this behavior is not really efficient, recompilation of everything is a total waste of resources.

from webpacker.

We have caching turned on. Once you identify this issue still occurs in using the current RC version, I'll reopen the issue.

from webpacker.