Skip to content

feat: IbisLazyFrame support #2000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
a186e7d
Initial copy of duckdb impl for ibis
rwhitten577 Feb 11, 2025
4a98c0b
Rename existing _ibis -> _ibis_old, update most of IbisNamespace
rwhitten577 Feb 11, 2025
7b25227
Update ibis namespace
rwhitten577 Feb 11, 2025
bbee1d2
Remove _ibis_old, update dataframe methods
rwhitten577 Feb 11, 2025
6c739d4
Update Expr methods
rwhitten577 Feb 12, 2025
70605ad
All ibis frame tests passing
rwhitten577 Feb 12, 2025
56a5f08
WIP ibis expr tests
rwhitten577 Feb 12, 2025
82c41f8
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 Apr 14, 2025
04a41ec
Update ibis dataframe and groupby to match latest duckdb
rwhitten577 Apr 14, 2025
9e756e5
Update ibis expr match latest duckdb
rwhitten577 Apr 14, 2025
8ccff78
Update ibis expr dt, list, str to match latest duckdb
rwhitten577 Apr 14, 2025
6140962
Update ibis namespace and selectors to match latest duckdb
rwhitten577 Apr 14, 2025
15e89c9
All ibis frame tests passing
rwhitten577 Apr 14, 2025
81a62f0
Fix more ibis tests, esp group_by, read/scan, and std/var ddof
rwhitten577 Apr 15, 2025
e71d899
All ibis tests except expr_and_series passing
rwhitten577 Apr 15, 2025
4459ca7
More ibis expr tests passing
rwhitten577 Apr 15, 2025
8eb20f6
Fix ibis type issues
rwhitten577 Apr 16, 2025
22026ac
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 Apr 16, 2025
f6bb893
Re-enable commented out test params
rwhitten577 Apr 16, 2025
5bd99df
fix(typing): Make `pyright` happy
dangotbanned Apr 16, 2025
15dc406
ci(typing): Add `ibis` to `--group typing`
dangotbanned Apr 16, 2025
0519a41
ci: Reuse `optional-dependencies` group
dangotbanned Apr 16, 2025
4b98d22
ci(typing): Remove `ibis` from `mypy` overrides
dangotbanned Apr 16, 2025
fdbd3a8
revert: Undo `mypy` override removal
dangotbanned Apr 16, 2025
6e44a8b
chore: Replace all `get_ibis` with an import
dangotbanned Apr 16, 2025
21dd5b4
the great unbanning
dangotbanned Apr 16, 2025
6f489f1
Fix ibis pyright issues on all but expr.py and namespace.py
rwhitten577 Apr 16, 2025
af77027
Add branch for converting nw enum to ibis
rwhitten577 Apr 16, 2025
a685837
Fix ibis pyright for namespace.py
rwhitten577 Apr 16, 2025
1b07dfe
refactor(typing): Simplify `concat`
dangotbanned Apr 16, 2025
25b30e0
refactor: Also same error in `concat`
dangotbanned Apr 16, 2025
dceda64
fix: Collect iterable before consuming
dangotbanned Apr 16, 2025
8f7f005
refactor: Simplify `concat` some more
dangotbanned Apr 16, 2025
55240aa
refactor: Use more `chain.from_iterable`
dangotbanned Apr 16, 2025
49e9918
a little `lit`
dangotbanned Apr 16, 2025
28b7ec5
Fix ibis pyright for expr.py
rwhitten577 Apr 16, 2025
1ed691a
Fix ibis pyright for dataframe.py
rwhitten577 Apr 16, 2025
2843b7c
Add ibis to pytest gh action
rwhitten577 Apr 17, 2025
5f3687f
Add ibis >= 10.0 requirement around `cases`, fix interval casting
rwhitten577 Apr 17, 2025
5ade4b8
Merge remote-tracking branch 'upstream/main' into pr/rwhitten577/2000
dangotbanned Apr 17, 2025
3248a50
refactor: Use `@require.backend_version`
dangotbanned Apr 17, 2025
5df6b65
refactor(typing): Remove implict `Self`
dangotbanned Apr 17, 2025
ae79ca4
ci: Make sure `ibis` is installed
dangotbanned Apr 17, 2025
ea32978
ci: Include `ibis` in coverage
dangotbanned Apr 17, 2025
d0490d7
refactor: Use `not_implemented`
dangotbanned Apr 17, 2025
89275e3
ci: omit `_ibis.typing`
dangotbanned Apr 17, 2025
88e36c0
more coverage
dangotbanned Apr 17, 2025
5467fcf
Reduce pytest 3.8 & 3.9 coverage requirement to 75
rwhitten577 May 1, 2025
95a4e4d
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 May 1, 2025
9a8788c
Add ibis to _namespace.py and update to latest nw API changes
rwhitten577 May 1, 2025
7590e57
Add ibis to _namespace LazyOnly alias
rwhitten577 May 1, 2025
5245d29
no cover on ibis decimal/u128 type conversion
rwhitten577 May 1, 2025
c5df728
Clean up ibis rolling window min_samples check
rwhitten577 May 1, 2025
08ab976
Add _NativeIbis as protocol
rwhitten577 May 1, 2025
e733e61
fix(typing): Use a real guard
dangotbanned May 1, 2025
77c91e5
fix: (`RET504`)
dangotbanned May 1, 2025
c9ca39e
chore(typing): Add some more specific members to `_NativeIbis`
dangotbanned May 1, 2025
b7b4773
refactor(typing): Avoid `mypy` even thinking about `ibis`
dangotbanned May 1, 2025
bce3412
Update ibis impl in _from_native_impl to match latest duckdb
rwhitten577 May 1, 2025
61112f5
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 1, 2025
09935bd
Clean up IbisLazyFrame branches for clear test coverage
rwhitten577 May 1, 2025
9e78593
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 1, 2025
4ac6325
Update IbisLazyFrame to latest Compliant*.to_narwhals
rwhitten577 May 1, 2025
ee4f7f7
Mark ibis expr str.replace and dt.nanosecond not_implemented
rwhitten577 May 2, 2025
c6ebfbf
Update LazyGroupBy._alias_native_expr type to NativeExprT
rwhitten577 May 2, 2025
1ce3762
Link to ibis issue for first/last ignoring nulls in window context
rwhitten577 May 2, 2025
fbe18f5
Merge remote-tracking branch 'upstream/main' into feature/initial-ibi…
rwhitten577 May 2, 2025
f60bdb5
fix(typing): Resolve `.alias()` issues
dangotbanned May 2, 2025
1ee7e48
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 2, 2025
9e96600
chore(typing): Use typing for `native_to_narwhals_dtype`
dangotbanned May 3, 2025
56d99c6
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 3, 2025
5103d3d
chore(typing): Highlight why `pyright` sees the issue now
dangotbanned May 3, 2025
39d5a82
Mark Ibis skew not_implemented
rwhitten577 May 5, 2025
ce6d334
Only raise on Ibis concat where how == diagonal
rwhitten577 May 5, 2025
bf584b4
Ibis remove name preserving logic in mean_horizonal
rwhitten577 May 5, 2025
4656742
Ibis don't cast to str in Expr.n_unique or Expr.len
rwhitten577 May 5, 2025
c1f8179
Ibis add more coverage to dtype conversion, fix Ibis Interval init
rwhitten577 May 5, 2025
ae0bb57
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 May 5, 2025
907eff5
Add Ibis dt.truncate
rwhitten577 May 5, 2025
e59efc6
Ibis expr.len use native df.count() to include nulls
rwhitten577 May 5, 2025
5940723
pyright ignore Ibis Interval init arg
rwhitten577 May 5, 2025
c43f0fa
feat(typing): Add some typing for `IbisExprDateTimeNamespace`
dangotbanned May 5, 2025
fd6ef79
Merge branch 'feature/initial-ibis-lazyframe' of https://github.com/r…
dangotbanned May 5, 2025
1e02d89
refactor Ibis concat_str to simplify
rwhitten577 May 5, 2025
0be7628
Gate ibis expr.bucket to >= 7.1
rwhitten577 May 5, 2025
7057c1e
Cast case stmt in ibis concat_str to StringValue
rwhitten577 May 5, 2025
cd36bb9
refactor: Simplifying `expr`
dangotbanned May 5, 2025
dff6236
Merge branch 'feature/initial-ibis-lazyframe' of https://github.com/r…
dangotbanned May 5, 2025
7e5764b
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 6, 2025
e3b418e
feat(typing): Make `expr_str` mostly typed
dangotbanned May 6, 2025
4a047a9
chore: Remove trailing commas
dangotbanned May 6, 2025
8eccaf7
little less keywords
dangotbanned May 6, 2025
d150b23
Remove Ibis join predicate check for l/r lens
rwhitten577 May 6, 2025
630003a
Simplify ibis concat_str
rwhitten577 May 6, 2025
c6566e3
Remove ibis version gate on concat_str
rwhitten577 May 6, 2025
809102b
Simplify dtype check
rwhitten577 May 6, 2025
c8a1296
Remove ibis .name where not needed in namespace.py
rwhitten577 May 6, 2025
efa47c6
Remove ibis .name where not needed in expr.py
rwhitten577 May 6, 2025
7af6053
Simplify ibis group by agg
rwhitten577 May 6, 2025
1c9f1de
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 6, 2025
0fbc0ec
refactor: simplify ibis df.sort
rwhitten577 May 6, 2025
661044a
refactor: simplify ibis df.sort again
rwhitten577 May 6, 2025
c6eccc4
refactor: simplify ibis df.join_asof branching & predicate handling
rwhitten577 May 7, 2025
290449d
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 May 7, 2025
ccaa9dc
refactor: add util to drop ibis duplicate join cols
rwhitten577 May 7, 2025
8c29a31
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 7, 2025
cb7dda4
Merge remote-tracking branch 'upstream/main' into pr/rwhitten577/2000
dangotbanned May 8, 2025
909243d
refactor: Reuse new default `LazyExpr.alias`
dangotbanned May 8, 2025
e336b53
test: Inline `polars` import
dangotbanned May 8, 2025
75cf279
test(typing): Fix constructor/backend
dangotbanned May 8, 2025
5dcca3d
chore: Remove unused `*args, **kwds`
dangotbanned May 8, 2025
529c196
refactor: Simplify joining
dangotbanned May 8, 2025
465ec68
test: add `ibis` to `DEFAULT_CONSTRUCTORS`
dangotbanned May 8, 2025
e1b0486
fix: actually store the result lol
dangotbanned May 8, 2025
e1cafb6
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 8, 2025
4355857
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 8, 2025
828f355
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 8, 2025
b962692
refactor: call ibis tbl.drop once during join
rwhitten577 May 12, 2025
68e7b8d
refactor: import ibis _ as col at top of expr.py
rwhitten577 May 12, 2025
6d950c1
refactor: ibis is_unique avoid casting to str
rwhitten577 May 12, 2025
a1f6a48
refactor: remove ibis redundant join predicate checks, add asserts fo…
rwhitten577 May 12, 2025
cc71730
Merge branch 'main' into feature/initial-ibis-lazyframe
rwhitten577 May 12, 2025
dd956a6
Merge branch 'main' into feature/initial-ibis-lazyframe
dangotbanned May 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions narwhals/_compliant/group_by.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,10 +159,13 @@ def _evaluate_expr(self, expr: LazyExprT_contra, /) -> Iterator[NativeExprT_co]:
if expr._is_multi_output_unnamed():
for native_expr, name, alias in zip(native_exprs, output_names, aliases):
if name not in self._keys:
yield native_expr.alias(alias)
yield self._alias_native_expr(native_expr, alias)
else:
for native_expr, alias in zip(native_exprs, aliases):
yield native_expr.alias(alias)
yield self._alias_native_expr(native_expr, alias)

def _alias_native_expr(self, native_expr: Any, alias: str) -> Any:
return native_expr.alias(alias)

def _evaluate_exprs(
self, exprs: Iterable[LazyExprT_contra], /
Expand Down
6 changes: 5 additions & 1 deletion narwhals/_compliant/selectors.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,11 @@ def matches(self, pattern: str) -> CompliantSelector[FrameT, SeriesOrExprT]:
p = re.compile(pattern)

def series(df: FrameT) -> Sequence[SeriesOrExprT]:
if is_compliant_dataframe(df) and not self._implementation.is_duckdb():
if (
is_compliant_dataframe(df)
and not self._implementation.is_duckdb()
and not self._implementation.is_ibis()
):
return [df.get_column(col) for col in df.columns if p.search(col)]

return [ser for ser, name in self._iter_columns_names(df) if p.search(name)]
Expand Down
Loading
Loading