Produced JS bundles are not minified.

Is your feature request related to a problem? Please describe.

I seem to have a lot of trouble using node-sass on my MacBook - perhaps related to Apple Silicon (M1 processor)?

Describe the solution you'd like

I tested by replacing node-sass in devDependencies with sass@"1.49.9" (https://github.com/sass/dart-sass) and it seemed to work OK.

When searching online, I saw others mentioning that they gave up trying to make node-sass work on ARM architecture, and just replaced their dependency with dart-sass instead.

node-sass even has this warning in their readme:

Warning: LibSass and Node Sass are deprecated. While they will continue to receive maintenance releases indefinitely, there are no plans to add additional features or compatibility with any new CSS or Sass features. Projects that still use it should move onto Dart Sass.

https://github.com/sass/node-sass#node-sass

Describe alternatives you've considered

I am not sure what to do, the error message is cryptic and I'm not able to understand what is the problem.

Additional context

Here is a part of the error I see when trying to build the assets on my machine. I believe this is when it's trying to get the node-sass dependency, so even before it reaches any webpack step at all (the command I ran was npm install):

npm ERR! /Users/thomasrambo/.node-gyp/17.8.0/include/node/v8-internal.h:563:38: error: no template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
npm ERR!             !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
npm ERR!                                 ~~~~~^~~~~~~~~~~
npm ERR!                                      remove_cv
npm ERR! /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/type_traits:710:50: note: 'remove_cv' declared here
npm ERR! template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv
npm ERR!                                                  ^
npm ERR! 1 error generated.
npm ERR! make: *** [Release/obj.target/binding/src/binding.o] Error 1
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/Users/thomasrambo/Documents/devel/invenio3/invenio-assets/invenio_assets/assets/node_modules/node-gyp/lib/build.js:262:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Darwin 21.2.0
npm ERR! gyp ERR! command "/opt/homebrew/Cellar/node/17.8.0/bin/node" "/Users/thomasrambo/Documents/devel/invenio3/invenio-assets/invenio_assets/assets/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
npm ERR! gyp ERR! cwd /Users/thomasrambo/Documents/devel/invenio3/invenio-assets/invenio_assets/assets/node_modules/node-sass
npm ERR! gyp ERR! node -v v17.8.0
npm ERR! gyp ERR! node-gyp -v v3.8.0
npm ERR! gyp ERR! not ok
npm ERR! Build failed with error code: 1

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/thomasrambo/.npm/_logs/2022-03-28T17_28_19_551Z-debug-0.log

clean-css has some changes:

• it should be installed via clean-css-cli instead of clean-css
• the --root option has been removed

See the github repo for more details.

Here's a proposal to solve the entrypoint chunks dependency issue. It involves:

1. A new webpack plugin, which will generate a manifest file which lists for
   each entry a list of its dependent assets (exactly as displayed in the
   output of the webpack build). This requires releasing a new NPM module (e.g.
   webpack-entrypoints-manifest)
2. Some "magic" (aka. Flask globals dependent) Jinja functions for rendering
   de-duplicated script/stylesheet tags. These could be part of a potential
   pywebpack.Manifest subclass instead...

Here's a code exmaple of how this would work:

// webpack.config.js
...
var webpackConfig = {
  entry: {
    // Both of these depend on "lodash"
    base: './src/base.js',
    foo: './src/foo.js',
    bar: './src/bar.js',
  },
  ...
  optimization: {
    splitChunks: { chunks: 'all' },
    runtimeChunk: { name: "manifest" }
  },
  ...
  plugins: [
    // New custom plugin: "webpack-entrypoints-manifest" ???
    function() {
      this.plugin('done', stats => {
        const entrypoints = stats.toJson().entrypoints;

        // Build the manifest from the "entrypoints[].assets"
        const result = {};
        Object.entries(entrypoints).forEach(
          ([key, value]) => { result[key] = value.assets }
        );

        // Write the manifest file
        require('fs').writeFileSync(
          path.join(__dirname, 'dist/entrypoints-manifest.json'),
          JSON.stringify(result, null, 2)
        );
      })
    }
  ],
}
module.exports = webpackConfig

Will generate this in dist/entrypoints-manifest.json:

{
  "base": [
    "manifest.dd6ebf7f9944898eabd2.js", // runtime
    "2.dba3d6c4a7b297b6ddae.js", // lodash
    "4.7ef0251fa5668a890bc9.js" // base.js
  ],
  "foo": [
    "manifest.dd6ebf7f9944898eabd2.js", // runtime
    "2.dba3d6c4a7b297b6ddae.js", // lodash
    "1.2327f6fa7827a3e692df.js" // foo.js
  ],
  "bar": [
    "manifest.dd6ebf7f9944898eabd2.js", // runtime
    "2.dba3d6c4a7b297b6ddae.js", // lodash
    "3.db8025172bf68af1fc69.js" // bar.js
  ]
}

In Jinja templates:

<!-- invenio_theme/javascript.html -->
{% block javascript %}
{{ webpack_register('base') }}
{% endblock %}
{% block javascript_render %}
{{ webpack_render('js') }}
{% endblock %}

<!-- overlay_foo/page.html (inheriting from invenio_theme/page.html) -->
{% block javascript %}
{{ super() }}
{{ webpack_register('foo') }}
{% endblock %}

<!-- overlay_bar/page.html (inheriting from overlay_foo/page.html) -->
{% block javascript %}
{{ super() }}
{{ webpack_register('bar') }}
{% endblock %}

The "magic" Jinja functions:

from flask import request
from flask_webpackext import current_manifest

# NOTE: These use "flask.request" to store some global state...
# Any kind of "global" could do the trick though...

def webpack_register(entry):
    if not hasattr(request, 'jinja_webpack_entrypoints'):
        setattr(request, 'jinja_webpack_entrypoints', {'js': set(), 'css': set()})
    current_entrypoints = request.jinja_webpack_entrypoints

    assets = current_manifest[entry]
    # TODO: Figure out if order should be preserved...
    current_entrypoints['js'] |= {a for a in assets if a.endswith('.js')}
    current_entrypoints['css'] |= {a for a in assets if a.endswith('.css')}

def webpack_render(ext)
    current_entrypoints = gettatr(request, 'jinja_webpack_entrypoints', {})
    return Markup(...)  # render script/stylesheet tags based on "ext"

This will render:

...
<!-- deduplicated webpack files -->
<script src="manifest.dd6ebf7f9944898eabd2.js"></script>
<script src="2.dba3d6c4a7b297b6ddae.js"></script>
<script src="4.7ef0251fa5668a890bc9.js"></script>
<script src="1.2327f6fa7827a3e692df.js"></script>
<script src="3.db8025172bf68af1fc69.js"></script>
...

Also, fix comment

In the test_webpack.py change semanticui to semantic-ui

Left-over from #113

The following cookiecutter change:

inveniosoftware/cookiecutter-invenio-module#98

should be propagated to this Invenio module.

Namely, in run-tests.sh, the sphinx for doctests is invoked after pytest run:

$ tail -3 ./\{\{\ cookiecutter.project_shortname\ \}\}/run-tests.sh
sphinx-build -qnNW docs docs/_build/html && python setup.py test && sphinx-build -qnNW -b doctest docs docs/_build/doctest

This sometimes led to problems on Travis CI with the second sphinx-build run due
to "disappearing" dependencies after the example application was tested.

A solution that worked for invenio-marc21 (see
inveniosoftware/invenio-marc21#49 (comment))
and that was integrated in cookiecutter (see
inveniosoftware/cookiecutter-invenio-module#98) was to
run doctest execution in pytest, removing the second sphinx-build invocation.

This both solved Travis CI build failures and simplified test suite execution.

Note that this change may necessitate to amend the code tests etc so that things
would be executed with the Flask application context (see
inveniosoftware/invenio-marc21@09e98fc).

• Invenio v3.0 depreciated both features
• Invenio v3.1 provided an alternative via Webpack
• Invenio v3.2 maintained both build systems (Webpack/Flask-Assets).
• Invenio v3.3 will maintain both build systems (Webpack/Flask-Assets).

User will have had more than two years to make their migration (from March 2019 until end of life for v3.3 expected to be around June 2021).

Tasks:

• Remove Flask-Assets and webassets dependencies.
• Unpin node-semver to see if problems can be fixed.
• Clean up:
  • Docs for text related to Flask-Assets.
  • Tests related to removed code.
• In npm.py everything but extract_deps().
• Remove glob.py
• Remove filters.py
• In cli.py remove everything execpt collect().
• In ext.py remove Flask-Assets related initialization and configuration.

Currently, we hard-code the on from invenio-themes in the resolver as an alias: https://github.com/inveniosoftware/invenio-assets/pull/115/files#diff-7041ec923e4ebff4bd440b0171487e16R30

This should be injected somehow from Flask into the Webpack config, to also allow other bundles to override it and provide their own...

Rebuilding assets or using the depends key inside bundles for automatic rebuilds is currently unreliable, produces duplicate generated versions of files, and usually requires deleting part or even the entire static directory contents in order to work properly. This creates problems and frustration during development and production.
Probable cause of this issue could be either some configuration variable from the webassets package and/or improper initialization of the package, during InvenioAssets.init_app.

Example use-cases:

• During Development:

  • Change some style.scss and save
  • Assets are automatically rebuilt because style.scss is included in some depends key of a Bundle and is being "watched" by webassets
  • Development flask server auto-reloads or is manually restarted but still uses old generated assets

• On Production:

  • Deploy new version of static files
  • Rebuild assets
  • Deployed application still uses old generated assets

There is a problem using relative paths with the import tag in scss files following the version release 0.12 of webassets. With this being the minimum version currently in this module, the latest release is not useable for us at the moment.

Package version (if known): 1.3.0

Describe the bug

Running invenio webpack clean buildall fails because the templates directory is missing from the instance path

Steps to Reproduce

1. cookiecutter gh:inveniosoftware/cookiecutter-invenio-instance -c v3.4 --no-input
2. Install the following set of packages:

    invenio[auth, metadata, files, postgresql, elasticsearch7 ] >= 3.5.0a4
    invenio-admin >= 1.3.2,<1.4.0
    invenio-assets >= 1.3.0,<1.4.0
    invenio-formatter >= 1.1.3,<1.2.0
    invenio-logging >= 1.3.0,<1.4.0
    invenio-mail >= 1.0.2,<1.1.0
    invenio-rest >= 1.2.8,<1.3.0
    invenio-theme >= 1.3.17,<1.4.0
    lxml >=4.3.0,<5.0.0
    marshmallow >=3.0.0,<4.0.0
    uwsgi >=2.0
    uwsgi-tools >=1.1.1
    uwsgitop >=0.11

(Note that these are slightly different from those in the default Pipfile, notably invenio >= 3.5.0a4 and invenio-assets >= 1.3.0. We need to specify a whole bunch of other invenio-* packages because specifying invenio-assets manually clashes with invenio[base])
3. Run invenio webpack clean buildall
4. Get the following error:

ERROR in ./js/invenio_theme/templates.js
Module not found: Error: Can't resolve '@templates' in '/venv/var/instance/assets/js/invenio_theme'
 @ ./js/invenio_theme/templates.js 26:19-28:47
 @ ./js/invenio_search_ui/util.js
 @ ./js/invenio_search_ui/app.js

I managed to manually fix this by running mkdir /venv/var/instance/assets/templates, but it was pointed out in the Discord that invenio-cli symlinks this directory (although I couldn't work out what the target should be). However, if you're using Invenio Framework, then that tool won't work.

Could the webpack command itself instead take care of ensuring the directory exists?

https://github.com/inveniosoftware/invenio-assets/blob/master/invenio_assets/bower.py#L1-L8

v1.2 or v1.3 will change the manifest plug to use bundle tracker.

We need a small upgrade note to tell people what have changed, and how to upgrade, which will likely include:

• Remove {{ webpack['manifest'] }}
• Remove {{ webpack['vendor'] }}

This is best done, when upgrading InvenioRDM to SemanticUI

see: https://eslint.org/docs/latest/user-guide/configuring/configuration-files#configuration-file-formats

Package version (if known): 1.2.5

Describe the bug

There's an error/warning in the build logs coming from the clean-webpack-plugin, potentially affecting cleaning up the dist folder after building assets.

Steps to Reproduce

1. Run invenio assets build
2. See the following error in the logs:

...
clean-webpack-plugin: dangerouslyAllowCleanPatternsOutsideProject requires dry: false to be explicitly set. Enabling dry mode
...

Expected behavior

Probably since no one complained yet, it's not that critical, but seems like a small change in the webpack.config.js to add dry: true.

Necessary to safely use Node.js v18

PRs

1. #152
2. inveniosoftware/invenio-previewer#166
3. inveniosoftware/invenio-cli#331

Current output with these 3 PRs:

WARNING in ./js/invenio_app_rdm/user_dashboard/requests.js 414:32-56
export 'RequestCancelButtonModal' (imported as 'RequestCancelButtonModal') was not found in '@js/invenio_requests/components/Buttons' (possible exports: CancelButton, RequestAcceptButton, RequestCancelButton, RequestDeclineButton, SaveButton)

WARNING in ./js/invenio_communities/requests/index.js 117:32-56
export 'RequestCancelButtonModal' (imported as 'RequestCancelButtonModal') was not found in '@js/invenio_requests/components/Buttons' (possible exports: CancelButton, RequestAcceptButton, RequestCancelButton, RequestDeclineButton, SaveButton)

WARNING in ./node_modules/semantic-ui-less/semantic.less (./node_modules/semantic-ui-less/semantic.less.webpack[javascript/auto]!=!./node_modules/css-loader/dist/cjs.js!./node_modules/less-loader/dist/cjs.js!./node_modules/semantic-ui-less/semantic.less)
Invalid dependencies have been reported by plugins or loaders for this module. All reported dependencies need to be absolute paths.
Invalid dependencies may lead to broken watching and caching.
As best effort we try to convert all invalid values to absolute paths and converting globs into context dependencies, but this is deprecated behavior.
Loaders: Pass absolute paths to this.addDependency (existing files), this.addMissingDependency (not existing files), and this.addContextDependency (directories).
Plugins: Pass absolute paths to fileDependencies (existing files), missingDependencies (not existing files), and contextDependencies (directories).
Globs: They are not supported. Pass absolute path to the directory as context dependencies.
The following invalid values have been reported:
 * "less/site/globals/site.variables"
 @ ./node_modules/semantic-ui-less/semantic.less
 @ ./js/invenio_theme/theme.js 13:0-40

webpack 5.74.0 compiled with 3 warnings in 63745 ms

The arg --unsafe has been removed from the webpack install cmd. This also allows to use directly webpack buildall instead of the 3 separate cmds create, install, build.

To test

1. invenio-cli assets build fails for me. Running manually npm i and
   It looks like that the npm prefix is not correct and the build fails because node-pre-gyp is not searched/found in .../var/instane/assets/node_modules/.bin/
2. linking and having a JS module with watcher and autorebuild
3. all files previewer available in https://github.com/inveniosoftware/invenio-previewer
4. verify the chunks that should be part of the output, to optimize and decrease each dist js file.

To check

1. #137
2. #130

• merge catalogs with same language and domain

It can be possibly done with an additional filter that produces merged PO files.

i18n = GlobBundle(
    catalog('invenio_messages_js', 'messages-js.po'),
    'node_modules/invenio_search_js/translations/*/LC_MESSAGES/messages-js.po',
    filters=['pomerge', AngularGettextFilter(...)],
    output=...
)

It needs some deeper investigation before naive implementation (hence Size: hard).

I think we should upgrade dependencies too (run npm outdated). In particular there is an issue that Karolina encountered with Node 13, node-sass really depends on some Node versions, see here.

With Node 13, node-sass should be at least 4.13.

Maybe we should add a Travis matrix with different Node versions, from LTS (12) to 14?

Left over from #113

See supported Node.js versions compatibility matrix

cc @drjova @greut

https://travis-ci.org/inveniosoftware/invenio-assets/jobs/560026481

Package version (if known): v1.2.7

Describe the bug

Unable to install with the invenio buildall command because of a dependency resolution problem. Actually, the problem can also be seen if I clone this repository and just run npm install directly in the assets directory.

Steps to Reproduce

1. Clone the repository (git clone https://github.com/inveniosoftware/invenio-assets.git)
2. Go into the assets directory (cd invenio-assets/invenio_assets/assets)
3. Do npm install
4. See the following error:

Thomass-MacBook-Pro-2:assets thomasrambo$ npm install
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/eslint
npm ERR!   dev eslint@"^7.7.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer eslint@"^5.0.0 || ^6.0.0" from @typescript-eslint/[email protected]
npm ERR! node_modules/@typescript-eslint/parser
npm ERR!   peer @typescript-eslint/parser@"2.x" from [email protected]
npm ERR!   node_modules/eslint-config-react-app
npm ERR!     dev eslint-config-react-app@"^5.2.1" from the root project
npm ERR!   peer @typescript-eslint/parser@"^2.0.0" from @typescript-eslint/[email protected]
npm ERR!   node_modules/@typescript-eslint/eslint-plugin
npm ERR!     peer @typescript-eslint/eslint-plugin@"2.x" from [email protected]
npm ERR!     node_modules/eslint-config-react-app
npm ERR!       dev eslint-config-react-app@"^5.2.1" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/thomasrambo/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/thomasrambo/.npm/_logs/2022-03-28T16_47_53_174Z-debug-0.log

Expected behavior

I expect to be able to run the invenio buildall command (or alternatively just doing npm install directly in the assets directory).

Other things

I am using NPM version 8.5.5 installed with homebrew. I am running on MacOS Monterey version 12.1.

Assume an invalid theme is set in APP_THEME:

app.config['APP_THEME'] = ['invalid']

WebpackThemeBundle will then return None for the active theme bundle:

bundle = WebpackThemeBundle(...) bundle._active_theme_bundle is None

This means that WebpackBundleProject will have to deal with a bundle where the following calls will fail on the project:

bundle.entries
bundle.path
...

It should be investigated what impact this has if say an Invenio module does not specify a theme, and decide on what is the desired behavior.

• Waiting for #113 to be merged
• Waiting for LESS support in Webpack config to be done in inveniosoftware/invenio-theme#147

https://travis-ci.org/github/inveniosoftware/invenio-assets/jobs/664497993#L321

Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.15/bin/requirements-builder", line 8, in <module>
    sys.exit(cli())
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/requirements_builder/cli.py", line 69, in cli
    output.writelines(lines)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/click/_compat.py", line 88, in writelines
    for line in lines:
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/requirements_builder/cli.py", line 66, in <genexpr>
    '{0}\n'.format(req)
  File "/home/travis/virtualenv/python2.7.15/lib/python2.7/site-packages/requirements_builder/requirements_builder.py", line 108, in iter_requirements
    exec(setup_fp.read(), g)
  File "<string>", line 0
SyntaxError: encoding declaration in Unicode string
The command "requirements-builder -e all --level=min setup.py > .travis-lowest-requirements.txt" failed and exited with 1 during .

Moved from inspirehep/inspire-next#680

Read the following blog post:

http://blog.codinghorror.com/zopfli-optimization-literally-free-bandwidth/

And Zopfli seems like an interesting and 'not difficult' to implement improvement that could make our assets: plots, images, JS files much more efficient.

See comparison of jQuery library before/after compression:

https://mathiasbynens.be/demo/jquery-size

This is the library itself (written in C):

https://github.com/google/zopfli

This can be plugged in to different places: asset compilation, ingestion from arXiv (for plots) and so on.

The latest version 6.1.2 breaks the assets build giving the error:

ERROR in ./node_modules/semantic-ui-less/semantic.less (./node_modules/css-loader??ref--8-1!./node_modules/less-loader/dist/cjs.js??ref--8-2!./node_modules/semantic-ui-less/semantic.less)
    Module build failed (from ./node_modules/less-loader/dist/cjs.js):



    @import (multiple) '../../theme.config';
    ^
    Less resolver error:
    '../../theme.config' wasn't found. Tried - /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/node_modules/semantic-ui-less/theme.config,../../theme.config

    Webpack resolver error details:
    resolve '../../theme.config' in '/Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/node_modules/semantic-ui-less/definitions/modules'
      using description file: /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/node_modules/semantic-ui-less/package.json (relative path: ./definitions/modules)
        aliased with mapping '../../theme.config': '/Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config' to '/Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config'
          using description file: /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/node_modules/semantic-ui-less/package.json (relative path: ./definitions/modules)
            Field 'browser' doesn't contain a valid alias configuration
            using description file: /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/package.json (relative path: ./less/theme.config)
              no extension
                Field 'browser' doesn't contain a valid alias configuration
                existing file: /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config
                  /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config doesn't match the restriction /\.(le|c)ss$/i
              .less
                Field 'browser' doesn't contain a valid alias configuration
                /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config.less doesn't exist
              .css
                Field 'browser' doesn't contain a valid alias configuration
                /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config.css doesn't exist
              as directory
                /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config is not a directory

    Webpack resolver error missing:
    /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config.less,/Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config.css,/Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/less/theme.config


          Error in /Users/zzacharo/.virtualenvs/webpack-test-oPSfLRWq/var/instance/assets/node_modules/semantic-ui-less/definitions/modules/transition.less (line 19, column 0)

Hints

In the latest release, they changed the resolution logic and this might be the cause. Needs more investigation.

General thoughts

• We should probably commit a package-lock.json so we don't have these incidents when a new release is out. Now, I see that if you do that, you don't test probably new releases, taking benefit of the ^ symbol. But, I would say for when we build for production we need to lock the js dependencies otherwise we don't have a solid deployment.
• The less-loader is listed under devDependencies (See here). Shouldn't we install only dependencies when we build the project? If this is the desired behavior, then we "misuse" in a way how we declare dependencies.

There is an issue with loading assets multiple times. For example, if two templates load invenio_search_js, this will cause an error to be raised since the Angular app will be loaded twice. Assets should rather collect the requirements and then add them only once to the output.

We discussed with @jirikuncar the possbile solutions to this. There are two approaches at least. One is to make a better Bundle class that will check if the asset has already been declared and not give out the same asset twice. Other option would be to modify the assets extensions and handle the problem there.

Relevant line in the assets extension:
https://github.com/miracle2k/webassets/blob/master/src/webassets/ext/jinja2.py#L191

From conversation on Discord, seems like the homepage is including vendors for d3, but the content of the JS file is the jQuery library (detected by @ntarocco).

https://github.com/inveniosoftware/invenio-assets/blob/master/invenio_assets/ext.py#L54-L55

Add pending deprecation warnings when using the AMD/RequirejS build system.

Replace following HTTP links with HTTPS:

• docs/conf.py:141: 'invenio-assets@GitHub': 'http://github.com/inveniosoftware/invenio-assets',
• docs/conf.py:142: 'invenio-assets@PyPI': 'http://pypi.python.org/pypi/invenio-assets/',
• docs/conf.py:334: 'flask': ('http://flask.pocoo.org/docs/latest/', None),
• invenio_assets/init.py:32: * Flask-Assets <http://flask-assets.readthedocs.io/en/latest/>_
• invenio_assets/init.py:33: * Flask-Collect <http://flask-collect.readthedocs.io/en/latest/>_

Because of this join: https://github.com/inveniosoftware/invenio-assets/blob/master/invenio_assets/webpack.py#L50

Every theme has to be in their names folder. However, in many modules:

• invenio-theme
• invenio-i18n
• (maybe more)

The bootstrap assets are in the root:

js/...
scss/...
semantic-ui/

For backwards compatibility reasons this should be addressed. Some possible solutions could be:

1. Fallback:

        self.default = default
        self.themes = {}
        for theme, bundle in themes.items():
            path = os.path.join(folder, theme) if theme != self.default else folder
            self.themes[theme] = WebpackBundle(
                import_name,
                path
                **bundle
            )

2. Change the folder tree in the affected modules.

However, as it is now bootstrap three cannot be used. It will either load the application files or invenio ones, but never both.

When launching a new Invenio instance the following Webpack warnings about unused variables and bundle size are shown:

$ ./scripts/bootstrap
...
WARNING in /Users/user/.virtualenvs/my-site-GjR-0M0-/var/instance/assets/js/invenio_i18n/app.js
Module Warning (from /Users/user/.virtualenvs/my-site-GjR-0M0-/var/instance/assets/node_modules/eslint-load
er/index.js):

  ⚠  http://eslint.org/docs/rules/no-unused-vars  'setCurrentLanguage' is defined but never used  
  js/invenio_i18n/app.js:17:14
  


✘ 1 problem (0 errors, 1 warning)


Warnings:
  1  http://eslint.org/docs/rules/no-unused-vars

WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets: 
  img/fontawesome-webfont.76a4f23.svg (348 KiB)
  js/1.efb8c93ee3c6ea567422.js (954 KiB)
  
WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit
 (244 KiB). This can impact web performance.
Entrypoints:
  adminlte (961 KiB)
      js/manifest.39db3f171b1701db2cc0.js
      css/vendor.ba0e6041c836b7356017.css
      js/1.efb8c93ee3c6ea567422.js
      js/2.93ca5665ab0f06a71baf.js
  base (958 KiB)
      js/manifest.39db3f171b1701db2cc0.js
      css/vendor.ba0e6041c836b7356017.css
      js/1.efb8c93ee3c6ea567422.js
      js/3.8a97d3a6973bb03d47c9.js
  i18n_app (958 KiB)
      js/manifest.39db3f171b1701db2cc0.js
      css/vendor.ba0e6041c836b7356017.css
      js/1.efb8c93ee3c6ea567422.js
      js/4.87f184f9dd6e7df32b13.js
  search_ui_app (958 KiB)
      js/manifest.39db3f171b1701db2cc0.js
      css/vendor.ba0e6041c836b7356017.css
      js/1.efb8c93ee3c6ea567422.js
      js/6.7e50a27c4a6c9924c03f.js
  search_ui_theme (958 KiB) 
      js/manifest.39db3f171b1701db2cc0.js
      css/vendor.ba0e6041c836b7356017.css
      js/1.efb8c93ee3c6ea567422.js
      css/search_ui_theme.25db8a1dcb80ffa234a7.css
      js/7.722c098d658854b4ae41.js


WARNING in webpack performance recommendations: 
You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your appl
ication.
For more info visit https://webpack.js.org/guides/code-splitting/

...

• check existing API functionality
• add missing important API functionality
• check API function signatures and parameters
• enhance API docstrings (param, returns, raises, versionadded)
• plug API functions to existing docs

Depending on your bundles configuration, in order to run invenio assets build, having specific binaries (eg. node-sass, r.js from require.js, clean-css, etc) installed globally is now a pre-requisite, and usually a separate step of the Invenio instance setup process (eg. running some setup-npm.sh).

Since it would be a lot of trouble (if not impossible) to convert the current asset building bundles to be executed using npm run <some-build-script>, probably the easiest way would be to have these build/dev dependencies installed in node_modules and inject the static/node_modules/.bin folder to the PATH before executing the invenio assets build command.

Bonus Round
Detecting the required binaries is possible by scanning the Bundle.filters property, so automatically generating some devDependencies could be possible as well. Minimum/maximum versions for these binaries should be taken into account though.

When bootstrapping a new Invenio instance, the version in the package is set to 0.1.0.dev0. This version is also used by the npm command when generating the package.json:

{
    "name": "mysite",
    "version": "0.1.0.dev0",
    "dependencies": {
        "almond": "~0.3.1",
        "bootstrap-sass": "~3.3.5",
        "font-awesome": "~4.4.0",
        "jquery": "~1.9.1"
    }
}

The version identifier 0.1.0.dev0 is however not a valid npm version identifier, so npm install fails:

$ inveniomanage instance create mysite
$ cd mysite
$ python setup.py install
$ mysite npm
$ cdvirtualenv var/mysite-instance/static/
$ npm install
npm ERR! install Couldn't read dependencies
npm ERR! Darwin 15.0.0
npm ERR! argv "node" "/usr/local/bin/npm" "install"
npm ERR! node v0.12.2
npm ERR! npm  v2.7.5

npm ERR! Invalid version: "0.1.0.dev0"
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/lnielsen/envs/invenio3/var/mysite-instance/static/npm-debug.log

• introduce new documentation structure inveniosoftware/cookiecutter-invenio-module#56
• document user perspective
• document signals (if not done)
• document example app (if not done)

Left-over from #113

Is your feature request related to a problem? Please describe.

When wanting to create an alias for JS/CSS assets provided, a name has to be decided on.

Describe the solution you'd like

e.g. when making an alias for invenio-search-ui JS assets, one can go with either:

• @js/invenio_search_ui (actually looks like what's in assets/)
• @invenio_search_ui/js
• @invenio_search_ui