Webpack for granular chunking

[alicialics]

In the end of this article https://web.dev/granular-chunking-nextjs/
I've tested this briefly and it seem to be able to provide better chunking results than the current default. Should the current default be changed?

https://glitch.com/edit/#!/webpack-granular-split-chunks?path=webpack.config.js%3A1%3A0

config.optimization = {
  …
  splitChunks: {
    name: false,
    chunks: `all`,
    cacheGroups: {
      default: false,
      vendors: false,
      commons: {
        name: `commons`,
        chunks: `all`,
        // if a chunk is used more than half the components count,
        // we can assume it's pretty global
        minChunks: componentsCount > 2 ? componentsCount * 0.5 : 2,
      },
      react: {
        name: `commons`,
        chunks: `all`,
        test: /[\\/]node_modules[\\/](react|react-dom|scheduler)[\\/]/,
      },

optimization: {
    splitChunks: {
      chunks: 'all',
      maxInitialRequests: 25,
      minSize: 20000,
      cacheGroups: {
        default: false,
        vendors: false,
        framework: {
          chunks: 'all',
          name: 'framework',
          test: /[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/,
          priority: 40,
          enforce: true
        },
        lib: {
          test(module) {
            return (
              module.size() > 150000 && // 150KB cut off for any npm library (change this value to meet your requirements)
              /node_modules[/\\]/.test(module.identifier())
            );
          },
          name(module) {
            const hash = crypto.createHash(`sha1`)
            hash.update(module.libIdent({ context: 'dir' }))

            return 'lib-' + hash.digest(`hex`).substring(0, 8)          
          },
          priority: 30,
          minChunks: 1,
          reuseExistingChunk: true
        },
        commons: {
          name: 'commons',
          minChunks: 3, // define (or pass in) the total number of pages here
          priority: 20
        },
        shared: {
          name(module, chunks) {
            const hash = crypto
              .createHash(`sha1`)
              .update(chunks.reduce((acc, chunk) => acc + chunk.name, ``))
              .digest(`hex`)

            return hash
          },
          priority: 10,
          minChunks: 2,
          reuseExistingChunk: true
        }
      }
    }

[ianschmitz]

Can you be more specific? How are you measuring the difference?

[alicialics]

Obviously SPAs are different than NextJS and GatsbyJS, but I really like the idea of splitting into smaller lib bundles and increasing the framework chunk size.

Custom webpack output:

CRA output:

The lib chunks are smaller and can be better cached when there are no changes, and the framework chunk is larger than the runtime chunk in CRA. The article probably can describe this better:
https://web.dev/granular-chunking-nextjs/#improved-chunking

Splitting large libraries and framework dependencies into separate chunks reduces the possibility of cache invalidation since both are unlikely to change until an upgrade is made.

I used source-map-explorer to look at the lib chunks, one of date-fns-timezone, one was lodash, another was an internal ui library that gets changed on every release.

In Chrome lighthouse (serving CRA/custom build output via http-server) I see a small difference for Max Potential First Input Delay (1350ms vs 1920ms), I think because Chrome is parsing a bunch of smaller files and can do it in parallel. For the other metrics, I had assumed smaller chunks would result in better timing (FCP/TTI) but this doesn't show up in the runs. I believe it is because the total bundle size downloaded is still the same, and since lighthouse is throttling network traffic, downloading the chunks in parallel does not make total download time any faster.

[ianschmitz]

Thanks. The tricky thing here is each application is unique - what may be optimal for one could be suboptimal for another. We've tried to strike a balance between total # of requests and cachability. We also don't have the idea of "pages" as mentioned. When there are too many chunks that are required as part of the critical rendering path, you start to run into HTTP request concurrency limits of the browsers.

That's not to say our current configuration couldn't be improved. I'd love to hear if there's any obvious improvements that apply to projects of all shapes and sizes. We'd want to compare any chunking strategy changes across a variety of projects (varying amounts of first party code, varying amounts of third party, total number of third party libs, etc.) to compare metrics.

[alicialics]

Yep, I agree with your points. Webpack has the maxInitialRequests to configure the number of initial requests. At the end of the day, not sure whether CRA want to optimize for the 100% (all sites, including HTTP/1, maxInitialRequests =3) or whether we want to optimize for only (HTTP/2, maxInitialRequests=25).

For the first case, I think a net win is to put react libraries into it as well since they rarely change, much less than changing other packages in node_modules. That only increases the number of requests by one, and better long term caching than what we have today.

similar to:

framework: {
          chunks: 'all',
          name: 'framework',
          test: /[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/,
          priority: 40,
          enforce: true
        },

For the second case, we can split larger vendor chunks out as well, to increase browser js parsing speed. From my tests, since we only have one entry point, the number of initial bundles only increased by a small amount in any case (four in my case for an insanely large 5MB bundle). Also, since Gatsby and Next is using this as default, so it can't be that bad as an initial default. Finally HTTP/1 sites might not take a huge performance hit because the sites will load fine anyway with a slight delay if we hit the HTTP/1 connection limit.

[ianschmitz]

Optimizing for h2 makes a lot of sense to me. It's well supported by common server technologies, cloud providers, and modern browsers released in the last few years. If there's obvious wins for h2 I think we should explore them.

The write up collaborating with next and Gatsby is interesting. They were definitely optimizing a slightly different problem as they have to combat duplication across pages which we don't have. We may be able to use some of the ideas in a way that makes sense over in CRA such as the framework cache group as you suggested. The potential difference in JS parse/compile by splitting into smaller chunks is interesting.

[housseindjirdeh]

@atcastle landed the feature in Next.js, and yep the main idea was definitely to optimize across pages. That's wholly different in CRA :)

Happy to help explore if granular chunking could be useful in CRA!