PATH.normalize is incredibly expensive.

[Aeodyn]

I profiled a WebGL Unity game that had analytics enabled, and by far the most expensive call chain was PATH.normalize and related, in particular because of the .join("/") call. Just replacing the body of normalize with if(path[0] === "/" && path[1] === "/") return path.slice(1); return path; fixed the constant stutters that I was looking at. I cannot stress enough how much of a performance increase this was. Checking if the string is already normalized should help immensely.

I'm sorry I don't have any hard statistics ready for you, but just profiling any Unity game that has frequent virtual FS access (as with Unity's analytics enabled) with the Firefox Profiler should show it immediately.

I'm unsure what version of Emscripten was used in the game I profiled, but PATH.normalize is completely unchanged relative to now.

[kripken]

Interesting, thanks for filing this issue!

A PR would be good, as this sounds worth working around for now, even if eventually it should not be an issue with WasmFS (which won't have such code in JS at all).

[sbc100]

Do you happen to have the call stack that ended up calling PATH.normalize? Its surprising to me that this would end up being called frequently, especially in a game engine.. I'm surprised that file operations are happening in during a game render loop.

[Aeodyn]

The stack:

• normalize
• join2
• lookupPath
• stat
• getLocalSet
• syncfs
• UnityLoader["8d90908f98e90773c78cc0543d25bc04"]/syncfs/<
• forEach
• syncfs
• sync
• _JS_FileSystem_Sync
• [WASM functions]

Logging the paths indicated that the majority of them were similar to /idbfs/d7ff824e8f360056329f33e72822a2d7/Unity/local.b82aa389e23422e46828eba7d4e3f840/Analytics/ArchivedEvents/165152167800000.286f94b8/c. Additionally, manually clearing the game's IDBFS with the browser's dev tools also seemed to reduce the stuttering a lot.

[sbc100]

I wonder if _JS_FileSystem_Sync is being called more often than it needs to. I would expect that function to be called very infrequently.. most likely just once at program exit, or perhaps just once after data download.

[sbc100]

@juj?

[Aeodyn]

On the Unity end there's this mess:

var fs = {
    numPendingSync: 0,
    syncInternal: 1e3,
    syncInProgress: false,
    sync: function (onlyPendingSync) {
      if (onlyPendingSync) {
        if (fs.numPendingSync == 0) return;
      } else if (fs.syncInProgress) {
        fs.numPendingSync++;
        return;
      }
      fs.syncInProgress = true;
      FS.syncfs(false, function (err) {
        fs.syncInProgress = false;
      });
      fs.numPendingSync = 0;
    },
  };
  function _JS_FileSystem_Initialize() {
    Module.setInterval(function () {
      fs.sync(true);
    }, fs.syncInternal);
  }

syncInProgress ignores FS.syncFSRequests too. Fixing that and reducing the frequency helped, but didn't actually prevent the stutters unfortunately.

[juj]

We have seen PATH.normalize()s come up in profiles. Generally it is not that an individual PATH.normalize() would be slow, but typically when it gets slow the case is that there are a lot of files in the IDBFS to be synced.

@Aeodyn if you have a test case, I would recommend posting a bug report to Unity about that. Does removing the "Data Caching" option in the build affect the performance overhead?

[Aeodyn]

That's what it's seemed to be to me, in this case with those archived analytics events. (I found an old list of IDBFS keys I copy-pasted, and 9870 of them were analytics.)

I'm actually not a game dev, just a player, so can't say if the "Data Caching" option changes anything unfortunately. I can see about submitting a bug report to Unity, yeah.