Skip to content

Implement special Groups for StringViews #12771

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

Part of #12680

In #12269 @jayzhan211 made significant improvements to how group values are stored in multi-column aggregations. This requires specialized implementations for different column types

His initial PR has implementations for PrimitiveArray and String/Binary. However it does not have a specialization for StringView

So that means that queries that group on multiple columns are even faster. This shows up by effectively slowing down some clickbench queries when they are run with StringView:

For example, this query is 10% slower with StringView

SELECT "SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM 'hits.parquet' WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10;

Describe the solution you'd like

I would like to make this (and similar) query faster when string view is enabled :

SELECT "SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM 'hits.parquet' WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10;

Note this is grouping by 2 columns

Here is how to reproduce the issue

Step 1. Get hits.parquet using bench.sh:

cd benchmarks
./bench.sh data clickbench_1

Step 2: Prepare a script with reproducer query:

set datafusion.execution.parquet.schema_force_view_types = true;

SELECT "SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM 'hits.parquet' WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10;

Step 3: Run query

(venv) andrewlamb@Andrews-MacBook-Pro-2:~/Downloads$ datafusion-cli -f q.sql
  • set datafusion.execution.parquet.schema_force_view_types = true; --> Elapsed 0.688 seconds.
  • set datafusion.execution.parquet.schema_force_view_types = false; --> Elapsed 0.565 seconds.

Describe alternatives you've considered

I suggest implementing something like ByteViewGroupValueBuilder following the model of ByteGroupValueBuilder

The in progress values would be u128s and some buffers (maybe 2MB?)

implementing equal_to can take advantage of the inlined prefix optimization (aka compare the prefix inlined in the u128 and only check the value in the buffer if that is already equal)

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions