Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workerized emscripten retroarch (WIP) #17484

Merged
merged 4 commits into from
Jan 30, 2025

Conversation

JoeOsborn
Copy link
Contributor

@JoeOsborn JoeOsborn commented Jan 28, 2025

This PR removes the need for asyncify in retroarch web, and ports the web player away from BrowserFS to the modern Emscripten WASMFS, with backends for asynchronous fetch and for OPFS (origin private file system).

This patch eliminates the need for asyncify and uses modern filesystem APIs instead of the deprecated, unmaintained BrowserFS. I also added a new emscripten graphics backend for using webgl directly instead of (buggy) egl emulation.

This is a WIP patch because it won't fully work until some Emscripten PRs land and are released:

The former fixes an offscreen canvas context recreation bug, and the latter adds an equivalent to BrowserFS's XHR filesystem (but without the hazardous running-XHR-on-the-main-thread problem).

The biggest issue is that local storage of users who were using the old version of the webplayer will be gone when they switch to the new webplayer. I don't have a good story for converting the old BrowserFS
IDBFS contents into the new OPFS filesystem (the move is worth doing because OPFS supports seeking and reading only bits of a file, and because BrowserFS is dead, and because of the performance costs of asyncify).

I've kept around the old libretro webplayer under pkg/emscripten/libretro-classic, and with these make flags you can build a non-workerized RA that uses asyncify to sleep as before:

    make -f Makefile.emscripten libretro=$CORE HAVE_WORKER=0 HAVE_WASMFS=0 PTHREAD=0 HAVE_AL=1

I also moved the default directory for core content on emscripten to not be a subdirectory of the local filesystem mount, because it's confusing to have a subdirectory that's lazily fetched and not mirrored to the local storage. I think it won't impact existing users of the classic web player because they already have a retroarch.cfg in place.

Note: I know I need to rebase this on latest master, and to integrate with the other recent emscripten patches. I'm aware of that and will work on that over the coming week.

@BinBashBanana
Copy link
Contributor

Ideally a canvas ID could be specified somehow (instead of just using #canvas), similar to specialHTMLTargets, so that multiple instances could run on the same page without iframes (#15688)

@JoeOsborn
Copy link
Contributor Author

JoeOsborn commented Jan 28, 2025

Interestingly, with the deprecated lookup behavior #canvas refers to module.canvas, so multiple RA instances should work just fine.

specialHTMLTargets actually doesn’t work in multithreaded/worker situations since it’s per-thread.

@LibretroAdmin
Copy link
Contributor

Hi, can the conflicts be resolved?

@JoeOsborn
Copy link
Contributor Author

Yep, I'll work on that tomorrow probably. I wanted to get the PR up because it's been sitting on a branch a while and I thought maintainers might want a heads up

This patch eliminates the need for asyncify and uses modern filesystem
APIs instead of the deprecated, unmaintained BrowserFS.

This is a WIP patch because it won't fully work until these two
Emscripten PRs land and are released:

emscripten-core/emscripten#23518
emscripten-core/emscripten#23021

The former fixes an offscreen canvas context recreation bug, and the
latter adds an equivalent to BrowserFS's XHR filesystem (but without
the hazardous running-XHR-on-the-main-thread problem).

The biggest issue is that local storage of users who were using the
old version of the webplayer will be gone when they switch to the new
webplayer.  I don't have a good story for converting the old BrowserFS
IDBFS contents into the new OPFS filesystem (the move is worth doing
because OPFS supports seeking and reading only bits of a file, and
because BrowserFS is dead).

I've kept around the old libretro webplayer under
pkg/emscripten/libretro-classic, and with these make flags you can
build a non-workerized RA that uses asyncify to sleep as before:

make -f Makefile.emscripten libretro=$CORE HAVE_WORKER=0 HAVE_WASMFS=0 PTHREAD=0 HAVE_AL=1

I also moved the default directory for core content on emscripten to
not be a subdirectory of the local filesystem mount, because it's
confusing to have a subdirectory that's lazily fetched and not
mirrored to the local storage.  I think it won't impact existing users
of the classic web player because they already have a retroarch.cfg in
place.
@JoeOsborn JoeOsborn force-pushed the emscripten-worker-pthread branch from 3682a56 to e1a3a11 Compare January 29, 2025 19:55
@LibretroAdmin LibretroAdmin merged commit cacd5a9 into libretro:master Jan 30, 2025
30 checks passed
@JoeOsborn
Copy link
Contributor Author

This wasn’t ready yet, can it be reverted? The wip tag was there because it’s a draft. I still have resizing and startup performance bugs to fix

@LibretroAdmin
Copy link
Contributor

My bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants