Repository navigation
Speed up TypeQuery #7128
Description
mypy.type_visitor.TypeQuery is constructed often with the any or all builtin function as an argument. The function is used as a callable inside the query. Calling callable variables is kind of slow when compiled with mypyc. Since type queries are used a lot, switching to an integer constant or a compiled object instead of a callable could speed up runtimes measurably, perhaps by as much as 0.5% or so (random guess). Since the optimization is pretty simple, this may be worth doing.
Discussion:
- Maybe have separate type query visitor which produces boolean values. The strategy could be specified as an integer constant.
- For non-bool type queries, experiment with using an object instead of a callable for the strategy, since callables are currently pretty slow when compiled.
- Maybe also special case
self.strategy([])since it's used a lot.
Activity
mridullpandey commentedon Feb 24, 2020
hi @JukkaL i want to work on can you assign it to me
thanks :-)
mridullpandey commentedon Feb 24, 2020
can you please let me know how to get started
JukkaL commentedon Feb 28, 2020
@mridullpandey I've assigned this to you. Here are some hints:
TypeQuerythat is specialized forboolinstead of accepting an arbitrary type.strategyconstructor argument with an integer constant (corresponding to eitheranyorall).strategymethod that uses the integer constant to implement whatever is needed.vbarbaresi commentedon Oct 17, 2020
I just opened a PR with my attempt at
TypeQueryspecialized forboolIn your first description of the issue, you wrote:
I wanted to try that out too.
You mean replacing
strategytype heremypy/mypy/type_visitor.py
Line 257 in ec76cca
by a new type of object that can be compiled by mypyc? There would still have to be a callable method someplace in that object, right?
Hyperion101010 commentedon Mar 10, 2021
Hi there I am Shivam, I know Python to an intermediate level,
I am interested to solve this issue, will you assign me?
JukkaL commentedon Mar 10, 2021
@Hyperion101010 There already is an open PR here: #9604. However, it's not clear if it speeds things up significantly. If you want to help, you can try measuring the impact of the PR on performance. Note that if the difference is small, you may need a specialized environment, such as a machine where you've turned off turbo boost for a constant CPU frequency, and where you've disabled most background processes.