Slow webpack build time (advanced module optimization)

Question

We're having a hard time trying to migrate to webpack our project, which is currently based in requirejs.

After a couple of weeks trying to replicate our current project status with webpack, we're stuck with a performance issue.

We're using webpack version 2.3.3.

Currently we have 240 modules and 58 chunks.

Our problem is that, when we launch webpack in watch mode for development (or using webpack-dev-server), everytime we modify a file, we have to wait about 10 seconds for it.

Here is our webpack development config:

{
  context: path.resolve(__dirname),

  entry: {
      'app-std': [
        'main',
        'plugins/base-component',
        'controllers/base-controller',
        'widgets/base-widget',
        'usertiming'
      ]
  },

  output: {
    path: path.resolve('./dist/js'),
    filename: '[name].js',
    publicPath: '/js/'
  },

  resolve: {
    modules: ['public/js', 'node_modules'],
    alias: {
        'uuid': path.resolve(__dirname, 'public/vendor/uuid.js/dist/uuid.core.js'),
        'jsLogger': 'js-logger',
        'jqueryCookie': 'js-cookie',
        'jqueryValidation': path.resolve(__dirname, 'node_modules/jquery-validation/dist/jquery.validate.js'),
        'jQueryXDomainRequest': 'jquery-ajax-transport-xdomainrequest',
        'dust': 'dustjs-linkedin',
        'dust.core': 'dustjs-linkedin',
        'dustHelpers': 'dustjs-helpers',
        'bootstrapSelect': 'bootstrap-select',
        'bootstrapDropDown': path.resolve(__dirname, 'node_modules/bootstrap/js/dropdown.js')
    }
  },

module: {

    rules: [            
        {
            test: /\.jsx?$/,
            loader: 'babel-loader',
            exclude: /(node_modules)/,
            options: {
                presets: [['es2015', { modules: false }]/*, 'react'*/],
                plugins: ['syntax-dynamic-import'],
                cacheDirectory: true
            }
        }
    ]
},

plugins: [
    new webpack.DefinePlugin({
        'process.env': {
            'NODE_ENV': JSON.stringify('local')
        }
    }),
    new webpack.ProvidePlugin({
        $: 'jquery',
        jQuery: 'jquery'
    }),
    new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/)
],

devtool: 'cheap-module-eval-source-map',

devServer = {
    https: true,
    port: 7070,
    host: '0.0.0.0',
    headers: { 'Access-Control-Allow-Origin': '*' }
},

stats: {
    chunks: true,
    chunkModules: true,
    modules: true
}

}

These are the stats for the initial build:

6185ms building modules
65ms sealing
2ms optimizing
1ms basic module optimization
12ms module optimization
7906ms advanced module optimization
1ms basic chunk optimization
0ms chunk optimization
1ms advanced chunk optimization
0ms module and chunk tree optimization
12ms module reviving
2ms module order optimization
3ms module id optimization
2ms chunk reviving
6ms chunk order optimization
9ms chunk id optimization
22ms hashing
0ms module assets processing
214ms chunk assets processing
2ms additional chunk assets processing
1ms recording
0ms additional asset processing
0ms chunk asset optimization
2ms asset optimization
192ms emitting

If we modifiy one of our modules, webpack fires a rebuild and we get this numbers:

38ms building modules
38ms sealing
1ms optimizing
1ms basic module optimization
1ms module optimization
7470ms advanced module optimization
1ms basic chunk optimization
0ms chunk optimization
1ms advanced chunk optimization
0ms module and chunk tree optimization
3ms module reviving
0ms module order optimization
4ms module id optimization
3ms chunk reviving
1ms chunk order optimization
4ms chunk id optimization
14ms hashing
0ms module assets processing
1ms chunk assets processing
1ms additional chunk assets processing
0ms recording
0ms additional asset processing
1ms chunk asset optimization
0ms asset optimization
1ms emitting

In both cases, it is the advanced module optimization step which consumes most of the time. I don't understand why there is an advanced optimization in a non production build and I don't know why is taking so much time.

I would like to know if there is any way to dig deeper into that time consuming step and also, if is it possible to disable that optimization in development mode.

Thanks!

Answer 1

For anyone developing a huge app with a lot of async routes, you might need babel-plugin-dynamic-import-node. You can transform import to require in development mode with it. In this way chunk graph is pretty fast and my incremental build time from ~30s to ~3s.

Add the plugin to babel.config.js

module.exports = {
  plugins: ['dynamic-import-node']
}

If your project used vue-cli 3.x , do not need the plugin, just add a new file at root called .env.development, with the following contents:

VUE_CLI_BABEL_TRANSPILE_MODULES = true

Answer 2

After some (much) more digging, we ended up with a hack to cheat webpack. In our system we have dozens of async loaded chunks and a couple of spread circular dependencies which results in a lot of chunks having lots of parents. So, the main time consuming task was the execution of the built-in RemoveParentModulesPlugin. Because of that long module chain with many chunks having many parents, this plugin has extra work to do.

Our solution is to (only in development mode) add a new custom plugin which removes the parents of every module as we don't need this optimization when running the app in our local machines.

This is the code of our custom plugin, in case somebody finds it useful in the future:

function AvoidParentModulesOptimizationPlugin() {}
AvoidParentModulesOptimizationPlugin.prototype.apply = function(compiler) {
    compiler.plugin('compilation', function(compilation) {
        compilation.plugin(["optimize-chunks-basic", "optimize-extracted-chunks-basic"], function(chunks) {
            // We cheat webpack to think there are no parents to optimize
            // so recompilation time is quite low on development mode
            chunks.forEach(function(chunk) {
                chunk.parents = [];
            });
        });
    });
};

Answer 3

Our team also has the same issue. We've identified that the slow down is caused by using require.ensure, which provide dynamic loading for the bundles. We've flag the issue here

https://github.com/webpack/webpack/issues/4716

To work around this issue, my teammate found a workaround that strip off require.ensure in dev environment using a babel plugin. It reduced the advanced module optimization time to milliseconds. With the work around, our continuous build time reduced from 8s to 1.5s.

https://www.npmjs.com/package/babel-plugin-remove-webpack