chore(engine): add more spans inside try-assign-batch + disable JIT on a few queries - #4947
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
Benchmark resultsCompared against |
handleReplayTasks called ReplayTasks once per workflow run, and each call fetched the inputs of the whole replay subtree from the payload store while holding the transaction and the tenant replay advisory lock. With external payload storage that serialized one S3 round trip per run, so a batch of fifty replays took minutes before any message was acked. Add RetrieveReplayInputs, which expands the subtree outside of a transaction and fetches every input for the batch in one parallel fan-out. ReplayTasks takes the prefetched inputs and only retrieves tasks missing from them inside the transaction.
…after discard Reshape the previous commit: instead of prefetching inputs outside of the transaction, which reads payloads for tasks the preflight may discard, the handler now replays the whole batch in a single ReplayTasks call. Inside the transaction, after the advisory lock, subtree locks, and discard checks, only the inputs of tasks that are replayed immediately are fetched, in one parallel fan-out for the batch. Drops the per-workflow-run grouping in handleReplayTasks and the RetrieveReplayInputs repository method.
ReplayTasks held one try-lock per tenant, so every replay for a tenant was serialized through a single controller and any concurrent attempt failed and cycled through the DLQ. Lock each workflow run in the batch instead, with a blocking advisory lock taken in ascending key order so overlapping batches cannot deadlock. The batch still runs in one transaction. A concurrent replay of the same run waits, then finds the tasks already queued and discards them in the preflight check, so it degrades to a no-op rather than a duplicate replay. Tasks without a workflow run id are locked on their own external id.
mrkaye97
changed the title
try-assign-batchtry-assign-batch + disable JIT on a few queries
Contributor
Author
grutt
approved these changes
Sep 21, 2026
| Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error) | ||
| } | ||
|
|
||
| func DisableJITForTransaction(ctx context.Context, tx execer) error { |
There was a problem hiding this comment.
The exported DisableJITForTransaction function has no preceding Godoc comment. This violates the repository requirement that exported types and functions have Godoc comments, so it must be documented before merging.
Context Used: Comment and logging conventions for the Hatchet codebase (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
For more visibility in here to help debugging, and also disabling the JIT on PG on a few queries
Type of change
Checklist
Changes have been:
🤖 AI Disclosure