Andrew Lamb Weekly-ish Open Source plan - 2026-01-05

[alamb]

This is my weekly plan, mostly for my own organizational need. I am making it public in the hopes that helps others to see what I am working on -- also I spend so much time in github the interface is very familiar to me and I can cross link all the issues I am working

(it is also my excuse as to why I haven't reviewed many good looking PRs)

Notes to myself: a duplicate entry unchecked means I need to go back and re-review

PR review queue (rough order)

• Remove parquet arrow_cast dependency arrow-rs#9077
• Row group limit pruning for row groups that entirely match predicates #18868
• Allow struct field access projections to be pushed down into scans #19538
• feat: Prune complex/nested predicates via statistics propagation #19609
• Record sort order when writing Parquet with WITH ORDER #19595
• Refactor PartitionedFile: add ordering field and new_from_meta constructor #19596
• Refactor cache APIs to support ordering information #19597
• Incremental group emission in HashAggregate #19562
• Timezone aware extract SQL expression #18990
• Support "pre-image" for pruning predicate evaluation #18789
• Support reverse parquet scan and fast parquet order inversion at row group level #18817
• fix: TableScan should recurse into provider logical plan in map_children #19282
• feat(memory-tracking): expose API to NullBuffer, ArrayData, and Array arrow-rs#8918 (comment)
• [DRAFT] Extension Type Registry Draft #18552
• feat: implement GroupArrayAggAccumulator attempt 3 #17915
• add specialized InList implementations for common scalar types #18832
• Row group limit pruning for row groups that entirely match predicates #18868 (rereview)

Projects I am supporting actively (high on my priority list)

• Improve DataFusion ClickBench performance (via pushdown predicates): Enable parquet filter pushdown (filter_pushdown) by default #3463
• Release arrow-rs 57.2.0: Release arrow-rs / parquet Minor version 57.2.0 (December 2025) arrow-rs#8465
• DataFusion object store caching improvements with @BlakeOrth [EPIC] ListingTable object store usage improvements #17214

Projects on my backlog

These are ones I would like to support but don't have the capacity at the moment to push, in relative order

• Help integrate Variant with @friendlymatthew [EPIC] Support VARIANT type for unstructured data #16116
• Epic: Join Order Enumeration #18249 from @NGA-TRAN
• feat: support invoking table functions with tables rather than a single expression #18535

PRs that look great but need a thorough review (looking for help here 🎣 from anyone else)

• external tables for multiple locations: feat(cli): support external tables on multiple locations #17702
• writing REE arrays to parquet: Support writing RunEndEncoded as Parquet arrow-rs#8069