Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Impose runtime limits on temp table and recursive queries #2999

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft

Impose runtime limits on temp table and recursive queries #2999

wants to merge 32 commits into from

Conversation

hatyo
Copy link
Contributor

@hatyo hatyo commented Dec 10, 2024

No description provided.

hatyo added 30 commits November 19, 2024 18:53
@hatyo
introduce RecursiveUnionExpression and RecursiveUnionQueryPlan.
ebfe5e2
@hatyo
Introduce RecursiveUnionCursor initial implementation.
8259bfb
@hatyo
Initial implementation of recursion algorithm.
3a7dc97
@hatyo
add implementation rule to RecursionUnionExpression.
e64a384
@hatyo
add recursive union test, refactor TempTableTest.
aba773e
@hatyo
Refactoring, more tests, add ordering pushdown rules for new plan ope…
b6d7fcf 
…rators.
@hatyo
Fix checkstyle violations, add documentation.
b2cab41
@hatyo
Resolve post-rebase conflicts.
1dc9552
@hatyo
add test for TempTableScan across continuations.
811d4f3
@hatyo
Factor out hierarchy plan creation logic in test.
751c04b
@hatyo
Refactor recursive union test to make it possible to test execution w…
d2ab23d 
…ith continuation
@hatyo
WIP - add method to init the temp table in evaluation context
b5c1120 
- this is to be used in the TempTableInsertPlan so it can create a new
  TempTable in the EvaluationContext if it owns it.
- recursive union plan fails because the ownership of temp tables is put
  incorrectly in the insert plans below (both sharing the same temp table
  reference which is incorrect).
@hatyo
WIP - implement Factory of TempTable and use it in EvaluationContext.
560dd0c 
- this makes testing owned temp tables much easier through dependency
  injection.
- TempTableTests works correctly, RecursiveUnionTests fails.
@hatyo
Refactor recursive union with correct temp table ownership.
9f7aa0a 
- Recursive union tests pass.
- State management is more complex due to divergence between initial
  state and recursive state transition.
@hatyo
Refactor recursive union continuation, add isInitialState flag.
e9fee48 
- This extra state is necessary as it causes a diversion in the state transition.
@hatyo
checkpoint all tests are passing.
4da95cb
@hatyo
Refactoring, recursive union tests with continuations now work.
b15c909 
- tested recursive union queries with ancestors, they give correct
  results with continuation.
- todo: more continuation tests with descendants, this is more complex
  since the continuation is covering the state of more rows.
@hatyo
WIP - edit this.
4c8b6bf
@hatyo
WIP. This must be cleaned up.
2c984b9 
- descendants test across continuation now works.
@hatyo
Algorithm is correct, refactoring, cleanup.
6b59c86
@hatyo
more hierarchy test utils.
44bcaf9 
- minor test refactoring.
@hatyo
add random hierarchy test.
80f940f
@hatyo
Use PTempTable instead of bytes where applicable.
4d89808 
- add first randomized hierarchy test.
- marked as Performance for now.
@hatyo
documentation, refactoring.
d276f3d
@hatyo
refactoring, documentation, cleanup.
8ef5b1a
@hatyo
Merge remote-tracking branch 'upstream/main' into recursive-cte
2f017fa
@hatyo
fix post-merge conflicts.
cf03c8e
@hatyo
reduce size of randmized hierarchy to 1000 elements per level.
0f25f33
@hatyo
cleanup.
660c1e5
@hatyo
add row limit to temp table insert to control its growth.
8ebc39d
@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: b96e503
  • Duration 0:12:51
  • Result: ❌ FAILED
  • Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of fdb-record-layer-pr on Linux CentOS 7

  • Commit ID: 7cbf4a2
  • Duration 0:11:54
  • Result: ❌ FAILED
  • Error: Error while executing command: ./gradlew --no-daemon --console=plain -b ./build.gradle build destructiveTest -PcoreNotStrict -PreleaseBuild=false -PpublishBuild=false -PspotbugsEnableHtmlReport. Reason: exit status 1
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hatyo @foundationdb-ci