Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support lambda left, right: (left.col == right.col) | (left.col.isnull() & right.col.isnull()) as predicate in joins #10705

Closed
1 task done
NickCrews opened this issue Jan 22, 2025 · 1 comment
Labels
feature Features or general enhancements

Comments

@NickCrews
Copy link
Contributor

Is your feature request related to a problem?

I have several tables. I want to join them where several columns are equal or both null.

I currently do

import ibis
a = ibis.memtable({"x": [1,2,None], "y": ["a", "b", None]})
b = ibis.memtable({"x": [None, 2,None], "y": ["a", "b", None]})
c = ibis.memtable({"x": [3, 2,None], "y": ["a", "b", None]})

def equalish(left, right, col):
    return (left[col] == right[col]) | (left[col].isnull() & right[col].isnull())

ab = a.join(b, [equalish(a, b, "x"), equalish(a, b, "y")])
bc = a.join(b, [equalish(b, c, "x"), equalish(b, c, "y")])

This is annoying because I need to have two slightly different lists of join predicates. I want to be able to make this DRY, and only define the join predicates in one place. eg I want to be able to do

def equalish(col: str):
    def pred(left: ibis.Table, right: ibis.Table):
        return (left[col] == right[col]) | (left[col].isnull() & right[col].isnull())
   return pred

preds = [equalish("x"), equalish("y")]

ab = a.join(b, preds)
bc = a.join(b, preds)

What is the motivation behind your request?

No response

Describe the solution you'd like

join() should also support:

  • Callable[[ir.Table, ir.Table], ] (eg equalish("x"))
  • Iterable[Callable[[ir.Table, ir.Table], ]] (eg [equalish("x"), equalish("y")]

What version of ibis are you running?

main

What backend(s) are you using, if any?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@NickCrews NickCrews added the feature Features or general enhancements label Jan 22, 2025
@NickCrews NickCrews changed the title feat: support lambda left, right: (left.col == right.col) | (left.col.isnull() & right.col.isnull()) in joins feat: support lambda left, right: (left.col == right.col) | (left.col.isnull() & right.col.isnull()) as predicate in joins Jan 22, 2025
@NickCrews
Copy link
Contributor Author

This feature is included as part of #10703, I think we should discuss it there where these is other context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements
Projects
Status: done
Development

No branches or pull requests

1 participant