Skip to content

bench(functions): group the 63 bench targets into 9 - #24342

Open
Dandandan wants to merge 1 commit into
apache:mainfrom
Dandandan:perf/consolidate-function-benches
Open

bench(functions): group the 63 bench targets into 9#24342
Dandandan wants to merge 1 commit into
apache:mainfrom
Dandandan:perf/consolidate-function-benches

Conversation

@Dandandan

@Dandandan Dandandan commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

None. Found while profiling compile times (#24325, #24326, #24329, #24330,
#24338, #24339) — this is the largest remaining item I measured.

Rationale for this change

datafusion-functions declares 63 [[bench]] targets. Each one is a
separate binary that statically links this crate, arrow and criterion, so
building the crate's benchmarks means 63 link steps.

Linking, not compiling, is the bulk of it: rebuilding a single bench target
takes 10.6s, almost all of it link, while the 63 benchmark files together are
only ~10.5k lines of source.

This is paid by cargo bench, by cargo build --all-targets, and by CI's
clippy --all-targets.

What changes are included in this PR?

The targets are grouped by their required-features, which preserves that
semantics exactly — a group can only be built when its feature is on, just as
each individual bench could before:

target benches required-features
string_expressions 16 ["string_expressions"]
math_expressions 16 ["math_expressions"]
unicode_expressions 12 ["unicode_expressions"]
datetime_expressions 8 ["datetime_expressions"]
regex_expressions 4 ["regex_expressions"]
misc 4 none

crypto, encoding and dictionary_encoding stay standalone — they are the
only members of their feature groups, so grouping them would buy nothing.

Each group lives in its own directory, benches/<group>/, with main.rs
declaring the individual benchmarks as modules and listing their criterion
groups in a single criterion_main!. Cargo does not auto-discover targets
inside those directories, so no autobenches = false is needed — which also
keeps cargo machete working (it stops scanning benches/ when autodiscovery
is disabled). benches/helper.rs stays shared at the top level, reached with
#[path = "../helper.rs"].

The benchmark code itself is untouched. There is not a single change to a
bench_function, benchmark_group or bench_with_input line anywhere in the
diff — git reports every moved file as a 94–99% similarity rename. Per file the
change is only:

  • drop criterion_main! — a module cannot define main
  • drop the now-unused criterion_main import
  • drop extern crate criterion (a no-op since edition 2018, present in 4 files)
  • point the four users of benches/helper.rs at crate::helper

Are these changes tested?

Interleaved with main so machine drift cancels out:

cargo clean -p datafusion-functions then… main this PR
cargo build --benches 43.9s / 42.9s 28.4s / 28.3s
cargo check --benches 15.0s / 14.7s 12.9s / 12.8s

Every benchmark still runs, and the counts show nothing was dropped:

target benchmarks (-- --test)
string_expressions 252
unicode_expressions 251
datetime_expressions 103
dictionary_encoding 36
regex_expressions 27
misc 21
encoding 9
crypto 10 (with --features crypto_expressions)

Also clean: cargo clippy -p datafusion-functions --all-targets,
cargo fmt --check, and cargo machete --with-metadata.

Filtering a single benchmark still works — cargo bench --bench math_expressions -- power selects exactly one.

cargo metadata reports 10 bench targets, against 64 on main; both counts
include the auto-discovered helper, which is unchanged by this PR.

Are there any user-facing changes?

Only for running an individual benchmark. Where you would previously write

cargo bench -p datafusion-functions --bench lower

you now select the group and filter by name:

cargo bench -p datafusion-functions --bench string_expressions -- lower

The filter is criterion's own, matching on benchmark id, so it also narrows to a
single case within a benchmark. No library changes.

If this approach looks right, roughly 100 more bench targets across the other
crates could get the same treatment — I did this one crate first so the tradeoff
is visible on a real diff before it spreads.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the functions Changes to functions implementation label Aug 13, 2026
@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.17%. Comparing base (8c099dc) to head (a6b5f3e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24342      +/-   ##
==========================================
+ Coverage   81.13%   81.17%   +0.03%     
==========================================
  Files        1112     1109       -3     
  Lines      388220   388033     -187     
  Branches   388220   388033     -187     
==========================================
- Hits       314985   314977       -8     
+ Misses      54696    54513     -183     
- Partials    18539    18543       +4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Each `[[bench]]` target is a separate binary that statically links this crate,
arrow and criterion, so building this crate's benchmarks meant 63 link steps.
Linking, not compiling, was the bulk of it: rebuilding a *single* bench target
took 10.6s, of which the compile is a fraction.

The 63 targets are grouped by their `required-features`, which keeps that
semantics exactly as it was:

    string_expressions     16 benches
    math_expressions       16
    unicode_expressions    12
    datetime_expressions    8
    regex_expressions       4
    misc                    4   (no required features)

leaving `crypto`, `encoding` and `dictionary_encoding` standalone, since they
are the only members of their feature groups. Each group file declares the
individual benchmarks as modules and lists their criterion groups in one
`criterion_main!`.

The benchmark code itself is untouched: no `bench_function`,
`benchmark_group` or `bench_with_input` line changes anywhere in the diff. Per
file the change is only dropping `criterion_main!` (a module cannot define
`main`), dropping the now-unused `criterion_main` import, dropping
`extern crate criterion` (a no-op since edition 2018), and pointing the four
users of `benches/helper.rs` at `crate::helper`.

`autobenches = false` keeps cargo from picking the module files back up as
targets of their own.

Running a single benchmark still works, via criterion's filter:

    cargo bench --bench math_expressions -- power

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Dandandan
Dandandan force-pushed the perf/consolidate-function-benches branch from b381bac to a6b5f3e Compare August 13, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants