Repository navigation
Conversation
Mirrors the existing OptimizablePostingsIterator pattern: an optional interface that segment implementations backed by a block-based on-disk format (with per-block block-max metadata) can implement, and that callers type-assert for, falling back to plain Next/Advance otherwise. This is the API surface needed to wire zapx's bitpack-simd skip-list block-max metadata (minNormID/maxTF, already written but unconsumed) up into bleve's query layer for Block-Max WAND pruning.
Batch-scoring a decoded block (bleve's Block-Max WAND conjunction path) needs a norm per candidate document; without it, scoring a batch-decoded block would require a separate per-doc seek anyway, defeating the point of decoding the block in one shot.
The previous 5-method shape (SeekBlock/BlockMaxTF/BlockMinNormID/ NormFromID/CurrentBlock) conflated peeking a block's bound with moving the cursor in one SeekBlock call, which forces an implementation to invalidate its own "already positioned" fast-iteration path on every peek -- turning every ordinary scan into a full re-seek per block, even for blocks that were never actually skipped. It also returned CurrentBlock's decoded arrays as iterator-owned buffers rather than filling caller-owned ones. Replaces it with three methods, validated by a separate, independent implementation that measured a real 6.20x mean / 5.13x median speedup in production: BlockMax() (a single combined peek, never moving anything), ShallowAdvance() (the only method that moves the cursor), and NextBlock() (a direct bulk fill of caller-supplied docNums/freqs/ norms arrays, with a globalOffset parameter so a multi-segment caller never needs a separate translation pass). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.