Skip to content

Reimplement the divmod() C integer optimisation using dynamic code generation #6801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Apr 25, 2025
Merged

Reimplement the divmod() C integer optimisation using dynamic code generation #6801

merged 11 commits into from
Apr 25, 2025

Conversation

scoder
Copy link
Contributor

@scoder scoder commented Apr 20, 2025

See #6717
Closes #6786

@scoder scoder added this to the 3.1 milestone Apr 20, 2025
scoder
scoder commented Apr 20, 2025
View reviewed changes
Cython/Compiler/PyrexTypes.py
Comment on lines -5165 to -5169

if function in needed_coercions and env:
arg_i, coerce_to_type = needed_coercions[function]
args[arg_i] = args[arg_i].coerce_to(coerce_to_type, env)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this was previously done twice. The (single) caller already applied the coercions at need.

@scoder scoder mentioned this pull request Apr 20, 2025
Closed
matusvalo
matusvalo approved these changes Apr 22, 2025
View reviewed changes
Copy link
Contributor

@matusvalo matusvalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had only a quick look and seems OK to me...

@matusvalo
Copy link
Contributor

One comment though. PR uses/rewrites a ranking algorithm for choosing correct type I have on comment and one question:

  1. the way how ranking mechanism works is not clear (I did not do deep investigation). Some explaining comment would be very helpful for someone who reads the code for the first time (like me)
  2. Are there any tests for this ranking mechanism that correct type is picked? The is not much in current PR but i suppose there is already something since the mechanism existed before...

@scoder
Copy link
Contributor Author

scoder commented Apr 23, 2025

the way how ranking mechanism works is not clear

There is at least a docstring in the best_match() function, but I agree that some more comments could help, e.g. for the score array. Overall, it's a rather lengthy and complex implementation. But it's not a trivial topic either, so the complexity is probably needed.

@scoder
Copy link
Contributor Author

scoder commented Apr 23, 2025

Are there any tests for this ranking mechanism that correct type is picked?

I think it's mostly tested indirectly. Type inference is an important part of this, C++ signature overloading and builtin types overloading is another.

Direct unit testing would definitely be an improvement.

da-woods
da-woods reviewed Apr 23, 2025
View reviewed changes
Copy link
Contributor

@da-woods da-woods left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No obvious issues from me - a couple of small suggestions

da-woods
da-woods reviewed Apr 24, 2025
View reviewed changes
@scoder scoder enabled auto-merge (squash) April 25, 2025 09:58
@scoder scoder merged commit 99e9df1 into cython:master Apr 25, 2025
72 checks passed
@mroeschke mroeschke mentioned this pull request Apr 25, 2025
Closed
@mroeschke
Copy link

Thank you! Confirmed that this fixes the failures we were seeing in pandas. Looking forward to seeing this in the nightly wheel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@da-woods da-woods da-woods left review comments

@matusvalo matusvalo matusvalo approved these changes

Assignees
No one assigned
Labels
Code Generation defect Optimization
Projects
None yet
Milestone
3.1
Development

Successfully merging this pull request may close these issues.

[BUG] Commit 0b866bf returns incorrect divmod result on Windows
4 participants
@scoder @matusvalo @mroeschke @da-woods