Automatic tree-shaking of messages

[amannn]

Messages can either be consumed in Server- or Client Components.

The former is recommended in the docs, as it has both performance benefits as well as a simpler model in general since there is only a single environment where code executes. However, for highly interactive apps, using messages on the client side is totally fine too. The situation gets a bit hairy when you need to split messages based on usage in Client Components (e.g. by page, or a mix of RSC / RCC).

Ideally, we'd provide more help with handing off translations to Client Components.

Ideas:

1. User-defined namespaces
2. Manually composed namespaces
3. Compiler-based approach

(1) User-defined namespaces

This is already possible (and shown in the docs), but unfortunately not so maintainable. For single components it can be fine however, so when you have many small interactive leaf components, this is quite ok.

(2) Manually composed namespaces

This worked well for the Pages Router and was documented, but unfortunately doesn't work with RSC.

Apollo has a similar situation with static fragments: apollographql/apollo-client-integrations#27

A potential solution could be to use a build-time macro that is evaluated separately.

(3) Compiler-based approach

See #1 (comment). Similar to fragment hoisting in Relay, if the collection of namespaces happens at build time, we could provide this list to Server Components. This seems to be the best option.

Design-wise this requires analyzing the namespace that is passed to useTranslations. Furthermore, a nice-to-have would be to analyze which keys are passed to t (if all calls are statically known).

Current progress:

• A proof-of-concept is developed in jantimon/next-poc-i18n-split (based on https://github.com/yashvesikar/next-intl-translation-bundler-example)
• APIs are being researched in https://github.com/amannn/nextjs-compiled-messages (this is a combined experiment that also includes AOT compilation for messages)

See also:

• Dependency tree crawling
• Detection of unused messages with i18n-check

[amannn]

An alternative (or possibly addition) is a concept based on the recently announced server components.

Code like this:

function LatestFollower({user}) {
  const t = useTranslations('LatestFollower');

  function onFollowClick() {
    UserAPI.follow(user.id);
  }

  return (
    <>
      <Text>{t('latestFollower', {username: user.name})}</Text>
      <IconButton
        aria-label={t('followBack')}
        icon={<FollowIcon />}
        onClick={onFollowClick}
      />
    </>
  );
}

… could be turned into:

function LatestFollowerServer({user}) {
  const t = useTranslations('LatestFollower');

  return (
    <LatestFollowerClient
      $1={<Text>{t('latestFollower', {username: user.name})}</Text>}
      $2={t('followBack')}
      $3={<FollowIcon />}
    />
  );
}

function LatestFollowerClient({$1, $2, $3, user}) {
  function onFollowClick() {
    UserAPI.follow(user.id);
  }

  return (
    <>
      {$1}
      <IconButton aria-label={$2} icon={$3} onClick={onFollowClick} />
    </>
  );
}

These are the interactive features that React currently has:

• useState
• useReducer
• use(Layout)Effect
• refs (and thus useImperativeHandle)
• event listeners

Most are easy to detect, but event listeners seem a bit tough, since it could potentially be a render prop. A convention like on[A-Z].* would definitely help here.

[amannn]

For very performance critical apps, it might make sense to use components that make use of next-intl only on the server side and to provide labels as strings to interactive client components. That way you wouldn't have to tree shake anything.

[amannn]

A manual approach seems easy and reliable: #62

[nbouvrette]

Thanks @amannn for sharing this with me. For those who might be interested, we built a similar Babel plugin solution that works on Next.js: https://github.com/Avansai/next-multilingual

Of course, as called out, one of the drawbacks is that all locales are loaded in the same bundle which increases the size. I am planning to try to find a solution as we are adding an SWC version of the plugin.

[amannn]

@yashvesikar just shared a POC for a webpack plugin that extracts messages based on page entry points with me: https://github.com/yashvesikar/next-intl-translation-bundler-example

I'm still mind blown 🤯

AFAICT what we'd have to do (assuming we'd implement this for the app directory) is:

1. Detect all entry points. In the POC, every file is treated as an "entry point", meaning that it will emit translations. This would need to be adapted to only handle modules that have the 'use client' banner as these are the boundaries where we're switching to the client side.
2. For every entry point, traverse the module graph and find all references to useTranslations/t and collect the namespaces.
3. Emit the translations per entry point.

The other part is that this is a webpack plugin, no idea if these will be supported by Turbopack or if we'd have to implement it in Rust instead (if Turbopack even supports plugins at this point).

Edit: An improved version of the webpack plugin that supports the App Router was implemented in https://github.com/jantimon/next-poc-i18n-split.

---

Another related project: https://lingui.dev/guides/message-extraction#dependency-tree-crawling-experimental

[yashvesikar]

@amannn Unfortunately, I don't have much knowledge of the app directory and all that RSC entails. I can speak to some of the advancements I implemented to this logic in the past and some of the ones I didn't get to.

• Support for colocated translations by component — Similar to CSS modules. This has a few advantages:
  • IMO it's easier to spot and fix & address translation bugs and errors.
  • Gives the translation plugin full control over the key names and allows the user to use TS type checking! (rough idea) i.e.

// src/components/my-component/i18n/en-US.json
{"title": "Title in English", "other": "Some other Key that won't be bundled"}
// src/components/my-component/i18n/fr.json
{"title": "Title in French"}

// src/components/my-component/i18n/index.ts
// loader can check that the keys match the ones in the main locale json 
let translations = {title: "title"} as const

// src/components/my-component/my-component.tsx
import { translations } from "/.i18n

export const MyComponent() => {
	// by modifying the useTranslations hook we can make t(key) have key be typeof translations
	const t = useTranslations(translations); 
	
	return(
		<div>{t(translations.title)}</div>
	)
}

• Figuring out how to get Webpack to load the translation files as dependencies.
  Right now in the POC, I made the server is loading the files from the disk using fs.readFileSync. This is probably the worst way to read that data for high-traffic scenarios because it is a blocking call. Even making that call async will be much better, even better might be to figure out how to have Webpack resolve the dependency itself such that the translations for a given page/path are available to the getServerSideProps function the same way any other import would be. I spent a little time looking into that but Webpack beat me 😅
  An open question is whether caching would be better done this way or via a separate network request with a long max-age.

• Improving the HMR, right now the HMR in dev kinda works. The POC plugin will build the right translation output files the first time the page is loaded in dev, but if the translation JSON file is changed the server needs to be restarted.

Some known limitations are that the Webpack parser is very limited and fickle. It's not a great way to traverse the AST to get all the keys, but since it's a single toolchain it is nice. I don't have any experience in SWC or TurboPack parsers but I have a hard time imagining they are worse than Webpack.

There are a few more things but I will leave it here for now. Sorry for the wall of text, happy that this may be useful to the project :)