-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix the bug when aggregation stops before a group is finished #3177
base: main
Are you sure you want to change the base?
Conversation
.build(false); | ||
|
||
// In the testing data, there are 2 groups, each group has 3 rows. | ||
// recordScanLimit = 5: scans 3 rows, and the 4th scan hits SCAN_LIMIT_REACHED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure why it only scans 4 times instead of 5, but the offset=1 seems consistent.
It looks like there's a conflict in the release notes. PRs no longer need to update the release notes file as they will be auto-generated by #3174, so I think the solution is just "take what is on main" |
// last row in last group, is null if the current group is the first group | ||
@Nullable | ||
private RecordCursorResult<QueryResult> lastInLastGroup; | ||
byte[] continuation; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
byte[] continuation; | |
@Nullable | |
private byte[] continuation; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this still hasn't marked the continuation
field as private. Is there a reason it needs package visibility?
docs/sphinx/source/ReleaseNotes.md
Outdated
@@ -20,7 +20,7 @@ Users performing online updates are encouraged to update from [4.0.559.4](#40559 | |||
<h4> New Features </h4> | |||
|
|||
* FRL respects PLAN_CACHE_*_MAX_ENTRIES - [PR #3156](https://github.com/FoundationDB/fdb-record-layer/pull/3156) | |||
<h4> Bug Fixes </h4> | |||
<h4> Bug Fixes </h4> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what happened here, but we probably want to back this out and take what's on main
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
142279c
to
14c8cfa
Compare
// last row in last group, is null if the current group is the first group | ||
@Nullable | ||
private RecordCursorResult<QueryResult> lastInLastGroup; | ||
byte[] continuation; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this still hasn't marked the continuation
field as private. Is there a reason it needs package visibility?
} else { | ||
if (Verify.verifyNotNull(previousResult).getNoNextReason() == NoNextReason.SOURCE_EXHAUSTED) { | ||
if (previousValidResult == null) { | ||
return RecordCursorResult.exhausted(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need to check to insert the empty result if isCreateDefaultOnEmpty
? It feels like that information is being lost here.
Perhaps it would be simpler if there were fewer moving parts here. If we wait until 4.2, we could drop support entirely for isCreateDefaultOnEmpty
(see #3107) which may simplify this somewhat.
currentContinuation = lastInLastGroup.getContinuation(); | ||
} | ||
previousValidResult = lastInLastGroup; | ||
return RecordCursorResult.withoutNextValue(currentContinuation, Verify.verifyNotNull(previousResult).getNoNextReason()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this also set previousResult
? I think as written, if you call getNext
a second time after getting one of these out-of-bounds values, you'll get an advanced continuation.
That is:
RecordCursor<T> aggCursor = getAggregateCursor();
RecordCursorResult<T> withValueResult = aggCursor.getNext(); // has a value
RecordCursorResult<T> withoutValueResult1 = aggCursor.getNext(); // first result with out-of-bound limit; continuation borrowed from withValueResult
RecordCursorResult<T> withoutValueResult2 = aggCursor.getNext(); // second result with out-of-bound limit; continuation may be greater than withoutValueResult1
In general, once a cursor returns a RecordCursorResult
without a next value, it should continue to return the same result if getNext()
is returned again.
This resolves #3180