Use real row-group sample to estimate partition size

[rjzamora]

Description

• Closes [BUG] CuDF-Polars produces oversized partitions for dictionary-encoded Parquet data #20566
• Follow up to Use a lower bound when estimating the partial file-size #20193

Updates cudf-polars row-group sampling to improve the ParquetSourceInfo.storage_size estimate used to make partitioning decisions.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[Matt711]

• From issue: [BUG] CuDF-Polars produces oversized partitions for dictionary-encoded Parquet data #20566

For example, when I use --blocksize 1_000_000_000 for query 1 of PDS-H, the in-memory size of each partition is actually ~3_000_000_000 bytes (3x larger than the intended partition size). This difference was even larger (3.7x) before #20193 was merged. That PR set a minimum bound on the storage-size estimate. That change improved OOM stability a bit, but the ParquetSourceInfo.storage_size estimate is still very low for query 1 of PDS-H.

How do things look after this PR?

[rjzamora]

How do things look after this PR?

This makes the partition size very accurate, because we sample a real row-group.

[rjzamora]

/merge