Document how to profile and optimize mypy

[JukkaL]

We now have some tooling that helps with investigating performance bottlenecks in mypy, and I've had some success using these to improve the performance of mypy. It would be good to document these workflows so that more contributors can help with this work.

[jorenham]

Is there also tooling that allows me to see if/where there are specific parts of my stubs that are a performance bottleneck?

[JukkaL]

Yes, we have something like that, but I haven't used it in a long time. I'll look into it as well.

[JukkaL]

I started writing the documentation here: https://github.com/python/mypy/wiki/Profiling-and-Optimizing-Mypy

It's still work in progress, but there is already something there.

[JukkaL]

Now it has at least the basics covered.

cc @ilevkivskyi @sterliakov who've been looking at performance bottlenecks.

[ilevkivskyi]

Nice! FWIW I started looking at why colours is so slow. It turns out more than half of all time is spent on type-checking just two expressions. Both involve a long list (few dozens) of np.array() calls nested within dict(zip()). I am currently looking at special-casing np.array() similar to how we special-case list literals and/or adding an expression cache.

[sterliakov]

Thanks, that's a nice writeup!

@ilevkivskyi then #19408 is related - also an expression cache for colour:)

[jorenham]

I am currently looking at special-casing np.array() similar to how we special-case list literals and/or adding an expression cache.

Anything I can help with on the numpy side of things?