Optimize pyinstaller build

[dromer]

As discussed here: plugdata-team/plugdata#1453

Execution of the pyinstaller version of hvcc is very slow. Next to the fact that executing from a JUCE subprocess also seems to give some performance penalty.

Converting a fairly complex patch (will link something shortly) gives the following compile times:

method	time
local hvcc	949.65ms
plugdata toolchain Heavy	33097.69ms
manual toolchain Heavy	12091.99ms
local Heavy	1312.25ms

The final result was a manual pyinstaller build of latest hvcc and is much more in line with what is expected.
It seems our build in the toolchain is particularly sluggish by over a factor difference.

It would be good to investigate how to make sure that the toolchain build gets the same performance as local pyinstaller builds.

[timothyschoen]

Interesting benchmark! That does look bad...

Any idea what I'm doing differently with my pyinstaller build?

[dromer]

Hmmm, for the Linux version this might be in part because it's using debian:bullseye which uses python3.9

I'm now trying to test automated builds using debian:bookworm and python3.11 - see if that makes a difference (my local system is bookworm with 3.11 as well)

[dromer]

Ok, this gives similar result as the local pyinstaller build: Total compile time: 1297.48ms
So hopefully this is all we need to get some better performance!

Still likely that running it from a subprocess will be much slower, but then it's ~3 seconds instead of ~30 seconds which is absolutely much better :)

I'll build the full toolchain and report back soon :)

[timothyschoen]

Sweet! One thing to keep in mind, is that that will make it link against a libc that was packaged with a distro from 2023. That's a bit recent, and could lead to compatibility issues with older distros.

Might be better to (somehow) install python 3.11 on a debian bullseye runner...

[dromer]

yeah perhaps, or we can use ubuntu-20.04 with python3.11 instead of the current privileged container runtime

[timothyschoen]

yeah perhaps, or we can use ubuntu-20.04 with python3.11 instead of the current privileged container runtime

oh yeah that'd make sense

[dromer]

Hmm, oddly enough running from plugdata it takes much longer: Total compile time: 19103.04ms
VS running the same Heavy binary locally: Total compile time: 1297.15ms

That's nearly 15 times slower! o.O

[timothyschoen]

Damn, maybe we're reading the process output too often or something?

[timothyschoen]

At line 83 of ExportingProgressView.h, you see "Time::waitForMillisecondCounter(Time::getMillisecondCounter() + 100);"

You could try to make the wait much longer, see if that improves performance?

[timothyschoen]

Alright, just updated the build server to run on Ubuntu 20.04 (oldest I could find with Python 3.11 available) with the newer python. So that should at least help a bit.

[dromer]

I think by default 20.04 uses python3.10 no?

But anyway it doesn't seem that this is our biggest bottleneck. I'll try patching ExportProgressView.h and see what that does ..

[edit: I made the wait + 1000 and it's still Total compile time: 19554.73ms]

[timothyschoen]

According to the ubuntu repo site, 22.04 comes with 3.10, 20.04 comes with 3.8.2. So we needed to install python from an external repo anyway, might as well do it on the oldest distro.

Ah, unfortunate that that wasn't it. Is this the time in ms that heavy reports? Here on macOS it's "133.29ms" for an empty patch, is that close to what you get?

[dromer]

Yeah an empty patch is Total compile time: 100.98ms so comparable.
Maybe it has something to do with disk access to parse all the included library files?

[timothyschoen]

Though if we're only getting around 100ms for an empty patch, but 19554ms for a large patch, that would suggest that this is not the overhead of starting a process. So maybe it is something about the file access? But wouldn't that be the same with regular Heavy?

[timothyschoen]

Oh wait, IIRC it unzips itself and all of its resources at startup right? That would make it take longer. But I'm not sure how that delay could be detected from inside Heavy?

I think it's also an option to not make a "zipped" package with pyinstaller. Maybe that is what you are doing with your local pyinstaller builds?