[mypyc] Generate introspection signatures for compiled functions #19307
Conversation
| for op in block.ops: | ||
| if isinstance(op, Assign) and op.dest.name == name: | ||
| return _extract_python_literal(op.src) | ||
| return _NOT_REPRESENTABLE |
There was a problem hiding this comment.
Do you have any ideas about how to generalize this to support more kinds of default values? I guess we could special case empty lists and dictionaries, which are somewhat common (even if you are not supposed to use them as defaults), and perhaps named constants and enum values. What about using a placeholder default value for unsupported default values, such as ... (ellipsis)?
There was a problem hiding this comment.
Other defaults are loaded with LoadStatic ops, so we'd need to look at the module's __top_level__ IR to extract the values. That isn't too tricky to do (in fact I already have a rough patch that does that), but it could be expensive performance wise, since __top_level__ can be quite big and we'd be doing potentially many linear searches. Alternatively, we could try recording information about default arguments earlier on during IR building, though I haven't figured out a simple way to do that yet
What about using a placeholder default value for unsupported default values, such as
...(ellipsis)?
I actually tried that in a previous revision :-) I was worried that it might conflict with using an actual Ellipsis default argument at runtime. I'm also not sure what effect using a placeholder might have on the tools that consume these signatures. For example, I think stubtest might complain if a runtime signature had ... as a default argument when the parameter isn't typed to actually accept EllipsisType
There was a problem hiding this comment.
Recording information about default argument values during IR building seems like the most promising approach to me, though I'm not sure how easy this would be to implement. They could perhaps be stored in FuncDecl or FuncSignature. I agree with you that analyzing __top_level__ seems too inefficient (and also inelegant).
|
@brianschubert Can you create a follow-up mypyc issue about the remaining cases, with some discussion based on the comments in this PR? |
|
I believe that this change is why I'm seeing a bunch of new errors pop up when running stubtest on scipy-stubs using mypy master ( All errors are seem to be related to
Each parameter is reported as a separate error, and aliases are counted double, which causes 75 errors to be reported here. I must admit that I know very little about cython and the C-side of cpython, so I'm not able to see why this is happening. But the fact that these only occur in case of these Oh and here are the errors: https://gist.github.com/jorenham/3d41e79607bcc1f45267acc1566833d2 update: This might also be relevant infomation: >>> from scipy.stats._unuran.unuran_wrapper import TransformedDensityRejection
>>> TransformedDensityRejection.__init__.__text_signature__
'($self, /, *args, **kwargs)'So it's kinda odd that stubtest says it's |
|
@jorenham More likely you're seeing an effect of #18259. This change only affects extension modules that are generated with mypyc, not Cython. #18259 made stubtest sensitive to signature information on C extension classes, which equally affects hand-written C extension modules and mypyc or Cython generated extension modules. |
|
Hmm, your issue seems to be due to stubtest looking at >>> from scipy.stats._unuran.unuran_wrapper import TransformedDensityRejection
>>> TransformedDensityRejection.__init__.__objclass__
<class 'object'>For comparison, this is what happens for stdlib C extension classes: >>> import pickle
>>> pickle.Pickler.__init__.__objclass__
<class '_pickle.Pickler'>and for mypyc native classes: >>> import mypy.types
>>> mypy.types.Instance.__init__.__objclass__
<class 'mypy.types.Instance'>So, it seems like there's either something funny with Cython extension classes, or a wrong assumption was made in #18259. (new issue to track this would be welcome!) |
🫡 #19732 |
Refs mypyc/mypyc#838
This PR populates
__text_signature__for compiled functions, making runtime signature introspection possible (i.e.inspect.signature(func)).While
__text_signature__is an undocumented CPython implementation detail, other extension module generators are using it in practice. For example, PyO3 and Cython both support it. I think it would be reasonable for mypyc to support it too.Some function signatures can't be represented by
__text_signature__(namely, those with complex default arguments). In those cases, no signatures are generated (same as the current behavior).