Skip to content

Loading metadata with p-limit #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

Borewit
Copy link

@Borewit Borewit commented Jul 20, 2025

This PR is build on top of:

I took a bunch of metadata files, spin op the audioMotion.js in server mode.
With the changes of #89 it metadata loaded slowly, massive parallel loading of the playlist, which took ~6 seconds to load all metadata.

First I disabled parallel fetching in the addSongToPlayQueue, retrieveMetadata , function chain.
The parallel loading is fact not done in retrieveMetadata, which is kind of mind fuck, as this method is responsible for the entire queue. Yet if a bunch of files are added to the queue, this function called for each item again and again. Hence #89 makes the parallel loading worse, and therefor performance goes down the drain.
After disabling all parallel processing, putting awaits all over the place. The increased the performance to ~1100 ms!

Then I re-enabled parallel loading, now using p-limit, to avoid the number of parallel processing, and to slowly tear down the requirement to keep calling retrieveMetadata way to ofton. Resulting to loading the metadata in 600ms.

Technical debt

  • I cut loose the promise add metadata to queue.
    retrieveMetadata(); // ToDo improve handling promise

    I maybe I should say, I kept as is on dev branch.
    There are several ways to resolve this, but it requires some refactoring.
    One solution could be to push things which require metadata in a queue. Then we consume this queue by the the metadata retrieveMetadata, which maybe could be a worker / parallel processing thingy.

Alternative solution

Update: I created #104 as probably a better alternative them, as that one queues items

@Borewit
Copy link
Author

Borewit commented Jul 20, 2025

Published this version on Netlify: https://audiomotion.netlify.app/

@Borewit Borewit force-pushed the loading-metadata-with-plimit branch from 471e947 to ef64e07 Compare July 20, 2025 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant