LuceneSourceOperator should take advantage of LeafCollector#collectRange

[jpountz]

Lucene sometimes collects ranges of doc IDs at once via LeafCollector#collectRange. In particular this is used when there are no deletes, scores are not needed and the query is one of the following:

• a MatchAllDocsQuery
• or a conjunctive query where all clauses are dense and report matching the same long runs of doc IDs (see DocIdSetIterator#docIDRunEnd)

The second case is expected to kick in with LogsDB thanks to its index sort.

LuceneSourceOperator could then produce a block of doc IDs that would not even need to be materialized as an int[]?

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[nik9000]

Here is the default implementation. It's truly a range.

The best way to take advantage of it would be to put the range into DocVector. Either by building a IntRangeVector or special casing it in DocVector.

We need a version of lucene with apache/lucene#14401 to take advantage of it. It's not released yet so far as I can tell, but it should work.

[jpountz]

10.3 should be relased in the coming week, so hopefully very soon!