Webpack 2 - Code splitting top-level dependencies

Question

Final Edit

The tl;dr resolution of this is that it's impossible. Though the top answer below does have some good information.

---

Consider the code below, from contacts.js. This is a dynamically loaded module, loaded on demand with System.import elsewhere in the code.

If SharedUtil1 is also used in other modules which are also dynamically loaded with System.import, how would I go about having SharedUtility1 excluded from all of these modules, and only loaded on demand the first time it's needed?

A top-level System.import of SharedUtil1 won't work, since my export depends on it: exports can only be placed in the top level of a module's code, not in any sort of callback.

Is this possible with Webpack? I'm on version 2.0.7 beta.

import SharedUtil1 from '../../SharedUtilities/SharedUtility1';

class Contacts{
    constructor(data){
        this.data = data;
        this.sharedUtil1 = new SharedUtil1();
    }
}

export default Contacts;

---

UPDATE 1

I thought the bundle loader was what I wanted, but no, that turns your imported module into a different function that you call with a callback to get to the actual module, once it's done loading asynchronously. This means you can't transparently make module X load asynchronously without making breaking changes to your code, to say nothing of the fact that you're back to the problem originally described, that if your top-level module depends on the now-asynchronously loaded dependency, there's no way to export it, since exports must be at the top level.

Is there no way in Webpack to denote that dependency X is to be loaded on-demand, if needed, and have any imported modules which import it to transparently wait out the importation process? I would think this use case would be a sine qua non for any remotely large application, so I have to think I'm just missing something.

UPDATE 2

Per Peter's answer, I attempted to get deduplication working, since the commonChunk plugin relates to sharing code between end points, as he mentioned, and since require.ensure places the loaded code into a callback, thereby preventing you from ES6 exporting any code that depends on it.

As far as deduplication, contacts.js and tasks.js both load the same sharedUtil like so

import SharedUtil1 from '../../sharedUtilities/sharedUtility1';

I tried running webpack as

webpack --optimize-dedupe

and also by adding

plugins: [
    new webpack.optimize.DedupePlugin()
]

to webpack.config. In both cases though the sharedUtil code is still placed in both the contacts and tasks bundles.

Answer 1

System.import has been deprecated in Webpack. Webpack now favors import() which requires a polyfill for promises.

Code Splitting - Using import()

Answer 2

Per the Webpack creator, this is impossible. Plain and simple. See Peter's answer for plenty of other good info regarding Webpack and ES6.

The pasted image was the result of a misunderstanding. See the same user's answer above.

Answer 3

After reading your blog post I finally understand what you intended. I got a bit confused by the word "Top-level dependencies".

You have two modules (async-a and async-b) which are loaded on-demand from anywhere (here a module main) and both have a reference on a shared module (shared).

- - -> on-demand-loading (i. e. System.import)
---> sync loading (i. e. import)

main - - -> async-a ---> shared
main - - -> async-b ---> shared

By default webpack creates a chunk tree like this:

---> chunk uses other chunk (child-parent-relationship)

entry chunk [main] ---> on-demand chunk 1 [async-a, shared]
entry chunk [main] ---> on-demand chunk 2 [async-b, shared]

This is fine when shared < async-a/b or the probability that async-a and async-b are used both by the same user is low. It's the default because it's the simplest behaviors and probably what you would expect: one System.import => one chunk. In my opinion it's also the most common case.

But if shared >= async-a/b and the probability that async-a and async-b is loaded by the user is high, there is a more efficient chunking option: (a bit difficult to visualize):

entry chunk [main] ---> on-demand chunk 1 [async-a]
entry chunk [main] ---> on-demand chunk 2 [async-b]
entry chunk [main] ---> on-demand chunk 3 [shared]

When main requests async-a: chunk 1 and 3 is loaded in parallel
When main requests async-b: chunk 2 and 3 is loaded in parallel
(chunks are only loaded if not already loaded)

This is not the default behavior, but there is a plugin to archive it: The CommonChunkPlugin in async mode. It find the common/shared modules in a bunch of chunks and creates a new chunks which includes the shared modules. In async mode it does load the new chunk in parallel to the original (but now smaller) chunks.

new CommonsChunkPlugin({
    async: true
})

// This does: (pseudo code)
foreach chunk in application.chunks
  var shared = getSharedModules(chunks: chunk.children, options)
  if shared.length > 0
    var commonsChunk = new Chunk(modules: shared, parent: chunk)
    foreach child in chunk.children where child.containsAny(shared)
      child.removeAll(shared)
      foreach dependency in chunk.getAsyncDepenendenciesTo(child)
        dependeny.addChunk(commonsChunk)

Keep in mind that the CommonsChunkPlugin has a minChunks option to define when a module is threaded as shared (feel free to provide a custom function to select the modules).

Here is an example which explains the setup and output in detail: https://github.com/webpack/webpack/tree/master/examples/extra-async-chunk

And another one with more configuration: https://github.com/webpack/webpack/tree/master/examples/extra-async-chunk-advanced

Answer 4

If I've understood you correctly, you want to prevent the same dependency being loaded multiple times when different code chunks declare it as a dependency.

Yes this is possible; how to do it depends on both context in your application and whether it is in ES6 or ES5.

ECMA Script 5

Webpack 1 was built in ECMA Script 5 and typically uses either CommonJS or RequireJS syntax for module exporting and importing. When using this syntax, the following features can be used to prevent duplicate code:

• Deduplication prevents duplicate files being included in the compiled code by creating copies of the duplciate functions instead of redefining them.
• Named Chunks allows chunks to be declared as dependencies but not immediately evaluated; all occurrences of the same chunk will use the same instance.
• CommonsChunkPlugin allows a chunk to be shared across multiple entry points (only applies to multiple page websites).

Deduplication

From the webpack documentation:

If you use some libraries with cool dependency trees, it may occur that some files are identical. Webpack can find these files and deduplicate them. This prevents the inclusion of duplicate code into your bundle and instead applies a copy of the function at runtime. It doesn’t affect semantics.

emphasis is mine, not from source

As described by the documentation, the code splitting remains unchanged; each module that needs sharedUtil1 should declare the require as normal. To prevent the same dependency being loaded multiple times, a webpack setting is enabled that causes webpack to explicitly check files for duplication before including them at runtime.

This option is enabled with --optimize-dedupe resp. new webpack.optimize.DedupePlugin()

Named Chunks

From the webpack documentation:

The require.ensure function accepts an additional 3rd parameter. This must be a string. If two split point pass the same string they use the same chunk... require.include can be useful if a module is in multiple child chunks. A require.include in the parent would include the module and the instances of the modules in the child chunks would disappear.

In short, the loading of the modules is delayed until later in the compiling. This allows duplicate definitions to be stripped before they are included. The documentation provides examples.

Common Chunk Plugin

From the webpack documentation:

The CommonsChunkPlugin can move modules that occur in multiple entry chunks to a new entry chunk (the commons chunk). The runtime is moved to the commons chunk too. This means the old entry chunks are initial chunks now.

This is very specific to sharing chunks between multiple pages, it is not relevant in other circumstances.

ECMA Script 6

Support for advanced module import features is... a work in progress. To get a feel for where things are, see the following links:

• Ongoing What's new in Webpack 2
• 2015/12/20 Tree-Shaking
• Ongoing Webpack 2 Roadmap

Here's a good summary of ES6 modules and webpack: ES6 Modules with TypeScript and Webpack

The above information is likely to become out-of-date fast.

Suggestion

For your own sanity, I suggest:

If optimisation matters: Revert to CommonJS / RequireJS syntax and upgrade to ECMA Script 6 when Webpack 2 is stable and released.

If ECMA Script 6 syntax matters: Use the standard ECMA Script 6 import export format and add optimisation features as they become available.

There is simply too much flux to try and use advanced module loading features in the sill unstable webpack 2. Wait for things to settle down and for some really good plugins to become available before even attempting it.