Add benchmarks for DefaultLTT start/stop #5595
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Start and stop are called on the critical path. We should have benchmarks for them to evaluate changes that may affect their performance.
shakuzen
added
type: task
We should get a better idea of average performance with a random sample rather than a sample in a fixed position in the active tasks collection.
Using the invocation-level setup method, we can use average-time rather than single shot.
jonatan-ivanov
approved these changes
Oct 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Start and stop are called on the critical path. We should have benchmarks for them to evaluate changes that may affect their performance.
The benchmark is done differently than most other benchmarks because we want to isolate the time taken by starting a sample with a fixed number of existing samples and stopping a sample that hasn't been stopped. These both require specific states before the benchmarked code runs and that state will be altered by running the benchmark. If the state is not reset between benchmark invocations, we are measuring the time to start a new sample with increasingly more samples present, which we expect to take increasingly longer. That did not seem like the right approach. Likewise, the stop benchmark would result in stopping an already stopped sample if multiple invocations are made without resetting the state, which we expect to be substantially faster than stopping an unstopped sample. Suggestions are welcome on any better way to handle this.
See #5591