Add test case for cache #20
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
avery-blanchard
force-pushed
the
cache-test
branch
from
9a85871 to
0e07855
Compare
JRosenkranz
reviewed
Apr 14, 2025
tests/models/test_decoders.py
Outdated
| print("passed validation level 0") | ||
|
|
||
| @pytest.mark.parametrize("model_path,batch_size,seq_length,max_new_tokens,cache_status", cache_params) | ||
| def test_cache(model_path, batch_size, seq_length, max_new_tokens, cache_status): |
Contributor
There was a problem hiding this comment.
I don't think we need to parametrize everything here other than the miss and hit. Just specify them inside the test.
JRosenkranz
reviewed
Apr 14, 2025
| None, | ||
| only_last_token=True, | ||
| **padding_kwargs | ||
| ) |
Contributor
There was a problem hiding this comment.
Is there something we need to assert for miss/hit?
JRosenkranz
reviewed
Apr 14, 2025
Contributor
JRosenkranz
left a comment
JRosenkranz
left a comment
There was a problem hiding this comment.
We may want to create a fixture which turns caching on, and caches the compilation. Then in the test we can re-run with caching on/off to make sure we see a hit or no-cache used
Signed-off-by: Avery Blanchard <[email protected]>
avery-blanchard
force-pushed
the
cache-test
branch
from
0e07855 to
87b3ac8
Compare
avery-blanchard
force-pushed
the
cache-test
branch
from
6a6fe67 to
2580699
Compare
JRosenkranz
reviewed
Jul 24, 2025
Contributor
JRosenkranz
left a comment
JRosenkranz
left a comment
There was a problem hiding this comment.
Is it possible to re-use the current test_common_shapes test but drive the option to use caching with an environment variable?
This was referenced Jul 28, 2025
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a test case for caching.
The test case runs twice with a single shape. The caching feature is enabled during the test and disabled after on reset.
cc: @JRosenkranz