Add test case for cache #20
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
avery-blanchard
force-pushed
the
cache-test
branch
from
9a85871 to
0e07855
Compare
JRosenkranz
reviewed
Apr 14, 2025
| @@ -287,5 +289,56 @@ def _metric_calculator(r: torch.Tensor, t: torch.Tensor): | |||
| else: | |||
| print("passed validation level 0") | |||
|
|
|||
| @pytest.mark.parametrize("model_path,batch_size,seq_length,max_new_tokens,cache_status", cache_params) | |||
| def test_cache(model_path, batch_size, seq_length, max_new_tokens, cache_status): | |||
There was a problem hiding this comment.
I don't think we need to parametrize everything here other than the miss and hit. Just specify them inside the test.
JRosenkranz
reviewed
Apr 14, 2025
tests/models/test_decoders.py
Outdated
| None, | ||
| only_last_token=True, | ||
| **padding_kwargs | ||
| ) |
There was a problem hiding this comment.
Is there something we need to assert for miss/hit?
JRosenkranz
reviewed
Apr 14, 2025
Signed-off-by: Avery Blanchard <[email protected]>
avery-blanchard
force-pushed
the
cache-test
branch
from
0e07855 to
87b3ac8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a test case for caching.
The test case runs twice with a single shape. The caching feature is enabled during the test and disabled after on reset.
cc: @JRosenkranz