-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: dt.highest_precedence([dt.uint64, dt.int32])
gives uint64
, not int64
#7331
Comments
The correct answer is probably I think I would expect the current system to error if there's no type that can represent the set of values common to both types.
|
I think this means we would remove the ability to cast from signed to unsigned integers in all cases, which is more restrictive than the backends actually are -- although I recognize this is a bit intractable without knowing the bounds of the values in a given column. |
oh I see, my bad.
All uint32s are representable by int64s, so casting in that case should always be allowed. Are you talking about the I think there are two different functionalities here: "is x castable to y?"What needs to happen during an explicit
"find a dtype that can represent both dtypes safely"What happens during an implicit cast. Currently we are relying on uint8s = ibis.array([255]).cast("array<uint8>")
int8s = ibis.array([127, -128]).cast("array<int8>")
x = uint8s.concat(int8s)
print(x.type())
print(x)
# array<uint8>
# [255, 127, -128] Really this should return I would expect these properties: If
If If |
No, those are tractable by going up by a power of 2 in bit-width, but I think the split you've identified is good! So if a user says "just cast it" we can allow it (within reason) and then let the backend fail or succeed. Implicit casting should find a dtype that can encompass both dtypes (or the implicit cast should fail, if it cannot). |
Ah sorry I wasn't clear, I think we're on the same page I meant X was the same width for both eg uint32 to int32 |
There are valid issues here, but it's not currently blocking any of our work and fixing it is liable to be an insane 🐰 🕳️ -- if someone is interested in tackling this work without breaking the entire project, we're game to talk it through, but this isn't going to get picked up any time soon. |
What happened?
Came from exploring around exposing
highest_precedence()
publicly per #5707EDIT: this is a bit wrong, see cpclouds comment below.
gives uint64. I would expect it to be int64, because if you had a negative number in the int32 column, I would want that to still be negative in the unioned dtype. Maybe I'm not understanding this correctly though.
What version of ibis are you using?
main
What backend(s) are you using, if any?
NA
Relevant log output
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: