Default library view overloads (?) MPD causing it to disconnect

[hjri]

This is most likely due to library size being way too big (almost 3TiB with almost 200k songs and 45k albums according to mpc stats). Funnily enough neither Cantata nor M.A.L.P. have any problem displaying entire library, but myMPD "hangs" even with limit set to 25 and stops responding to further queries while MPD can still play music and Cantata/M.A.L.P. can still control it.

It seems that at least with limit set to 25 it will be successful somewhat but still show "MPD disconnected" error. This unfortunately happens when you just open library view, I have to work around it by making a shortcut to library that does some bogus search which doesn't hang the myMPD

[jcorporation]

Please provide debug logs from myMPD and MPD: https://jcorporation.github.io/myMPD/debug

[hjri]

Makes me wonder actually, it could be that cantata and malp are fast because they are runing on faster hardware? I wonder if sorting/searching could be made client-side for the myMPD UI (i.e. in JS) to capitalize on potentially faster machine running it.

[jcorporation]

I tested with my development laptop with a AMD Ryzen 7 PRO 4750U, that is much faster as your SBC.

Sorting and searching on client side requires to keep the complete albumlist with all metadata e.g. as a json array or an indexeddb database on the client. This can overburden older clients (older smartphones, tablets, etc) and makes the experience even worse.

As far as I know cantata keeps a local copy of the MPD database. I do not know how malp handles this.

[hjri]

testing by looking at mpd logs it seems that MALP just caches lists of albums and artists and whatnot, at least it did the process command "list album group albumartist group date" when just opening up album list and process command "list artist" when opening artists. When actually opening album it does find album "ALBUM NAME GOES HERE" and when opening artist it does:

list album artist "ARTIST NAME HERE" group albumartist group date
list album albumartist "ARTIST NAME HERE" group albumartist group date

I'm guessing that myMPD processes all the songs individually trying to find whatever matches the criteria, which will take long time on a very large dataset (tbh 10 seconds on a decent CPU is also quite bad) while other clients operate on smaller dataset only, i.e. albums/albumartists/artists instead of individual songs.

Interestingly enough, actually searching for something is blazing fast, i.e. if I search any contains "foo", then add any contains "bar" and then remove first one search works incredibly fast, the problem is that if there was slow query first (i.e. show all) it slows everything down (and also pegs one cpu core to 100%) and search essentially doesn't work until that query completes.

Maybe skip some processing/filtering steps and use smaller dataset directly when doing empty queries? It feels like it wastes cycles checking each song if it matches empty criteria or something.

[jcorporation]

myMPD also caches only the list of unique albums. The issue is all about sorting. The initial view without filters produces a large dataset that must be sorted. Removing the sort code makes the complete process blazing fast.

Therefore there are 2 ways to fix this issue:

• create a result set that is smaller
• implement a faster sorting algorithm / use a data structure that can be sorted faster

I think building the result set with the window in mind should speed up the process dramatically, because we have only a result set of max 500 to be sorted.

[jcorporation]

My first attempt to limit the result set limits only the length of the list to offset + limit. This reduces the response time to under one second. Code is in devel. Can you test?

[jcorporation]

myMPD v9.4.0 optimizes sorting in all views and should fix this performance problem finally.