Skip to content

Webpack 5 upgrade #7351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 49 commits into from
Feb 12, 2025
Merged

Webpack 5 upgrade #7351

merged 49 commits into from
Feb 12, 2025

Conversation

na9da
Copy link
Collaborator

@na9da na9da commented Dec 5, 2024

What this PR does

Fixes #6998
Depends on TerriaJS/TerriaMap#718 and #7311

Upgrades webpack and other build dependencies to the latest version.

Note: bulk of the changes are from running the saas-migrator that upgrades .scss files to modern api.

Changes summary

  • Packages upgraded
    webpack, ...
  • Babel transforms
    • Removed transformation to CJS for the default build. We still use it for node builds.
    • Other transforms removed plugin-proposal-class-properties, proposal-object-rest-spread, plugin-syntax-dynamic-import. These must now be well supported by browsers.
  • Cleaned up configureWebpack.js
    • Removed rules that are no longer necessary
    • Replaced raw-loader file-loader url-loader etc with builtin asset modules
    • Removed use of webpack specific qualifiers for imports like raw-loader!file-loader etc to make it more ESM compliant. Added extra config in webpack to correctly handle imports of static files (like csv and xml).
    • Added rules to polyfill nodejs modules that are referenced from some of our dependencies.
    • Removed hot reload config. AFAICT this was not being used and required maintaining two different code paths in webpack configuration. If required we can add it back later after testing that it works.
    • Replaced multiple configureWebpack parameters with a single options parameter.
    • Removed string-replace-webpack-plugin as it was outdated and has a few vulnerabilities.
      • TODO: Move rewrite of default Cesium API key warning to the app
  • Upgraded SASS
    • Migrated SASS files to use modern API (by running the sass-migrator script)
    • Replaced webpack aliases ~terriajs-variables and ~terriajs-mixins with respective relative path. This was necessary to run the migrator. Also it means simpler webpack config.
    • css-modules-typescript-loader that generates TS types for SASS modules is outdated and no longer works with webpack 5. I have made a repo from a fork of another similar package that can generate types. The other alternative would be to use this TS plugin - but not sure if it'll slow down compilation/editing.
  • Modified TIFFImageryProvider to work with webpack5 build.
  • TODO: Find a replacement for svg-sprite-loader. It appears to be not maintained and has a bunch of audit warnings.
  • Rewrote a couple of specs that depended on some CJS feature
  • TODO: Test in Windows.

I'll annotate the changes to make review easier.

Test me

Checklist

  • There are unit tests to verify my changes are correct or unit tests aren't applicable (if so, write quick reason why unit tests don't exist)
  • I've updated relevant documentation in doc/.
  • I've updated CHANGES.md with what I changed.
  • I've provided instructions in the PR description on how to test this PR.

na9da added 15 commits December 4, 2024 15:28
- Rewrite webpack aliased paths like ~terriajs-variables and ~terriajs-mixins to
  plain relative paths so that we can run the auto migrator.
  This was done using the following shell command:
  ```
     find lib/ -name '*.scss' -exec bash -c 'sed -i "s|~terriajs-variables|$(realpath -s --relative-to=$(dirname "$1") lib/Sass/common/_variables)|" $1' bash  {} \;
  ```
- Run auto migrator
  ```
     find lib/ -name '*.scss' -exec npx sass-migrator module {} \;
  ```
Instead of using 'raw-loader!' etc in the import path, we now
handle these import in webpack config. See webpack.config.make.js.
Removes use of 'raw-loader!' etc in import paths.
Instead handle the imports in webpack config.
Webpack literally works only for 'process.env.NODE_ENV'.
@na9da na9da changed the base branch from main to cjs-to-esm December 5, 2024 01:28
@pjonsson
Copy link
Contributor

pjonsson commented Dec 5, 2024

I looked through this and besides the things I commented on, the things that are there looks good to me. Would love to get the (approved) CJS to ESM PR merged soon so this PR doesn't end up in limbo.

@na9da na9da mentioned this pull request Dec 8, 2024
5 tasks
Base automatically changed from cjs-to-esm to main December 8, 2024 23:36
@na9da na9da marked this pull request as ready for review December 8, 2024 23:53
@zoran995 zoran995 mentioned this pull request Feb 10, 2025
4 tasks
Copy link
Collaborator

@zoran995 zoran995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have cherry-picked changes from #7450, it should be easier to review now as code-diff overthere got pretty large and we can now handle this in single review cycle

  • Fixed review comments
  • Removed unused weback plugins
  • upgraded webpack plugins
  • I didn't see any reason why we would need pmtiles alias in webpack config - marked it with review comment for double check.
  • I had to restore worker-loader as webpack native web worker support tends to get pretty complex, I will keep looking for a fix but it might be good to keep worker-loader and handle that as a follow-up. Important note: webpack worker support behaves differently in development and production builds. That's why it works on CI but fails locally. I have run into this issue in the past but still don't have a solution that always works.
  • side-effects are removed from package.json we can look into it later

Adds a custom property `Ion.defaultTokenMessage` for changing the credit message shown on the map when using default Ion token.
@na9da na9da self-assigned this Feb 11, 2025
@na9da
Copy link
Collaborator Author

na9da commented Feb 11, 2025

Regarding the web worker change, it appears to work if we set publicPath to auto instead of build/.

I think webpack will use publicPath + directory-containing-parent-script which evaluates to build/build/<downloadWorkerChunkName>.js.

We could test it out in a different PR.

This was mostly required back when using polyfill.io to perform polyfills.
@pjonsson
Copy link
Contributor

Regarding the web worker change, [..]

We could test it out in a different PR.

If this works in the current state I suggest merging it, this PR is already a significant step forward.

I'll rebase the ESLint fixes after this gets merged so the TypeScript upgrade can get merged too.

@na9da
Copy link
Collaborator Author

na9da commented Feb 12, 2025

Merging! Thanks everyone, for the contributions.

PS: will merge the terriamap PR after a new terriajs release, but i'll see if I can include the Typescript upgrade PR into this release.

@pjonsson
Copy link
Contributor

@na9da since you wrote "Merging!" I was going to rebase the ESLint fixes for TS 5.7 before I go to bed so those could be merged, but I'm guessing you got interrupted by something else. Will this PR get merged soon, or should I go to bed now and rebase things tomorrow?

Copy link
Contributor

@ljowen ljowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested building on OSX and Windows10 (arm64)

@na9da na9da merged commit f52962d into main Feb 12, 2025
9 checks passed
@na9da na9da deleted the webpack5 branch February 12, 2025 01:26
@zoran995 zoran995 mentioned this pull request Mar 7, 2025
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to webpack 5
4 participants