Add Cache Miss/Hit Test #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alex-jw-brooks
force-pushed
the
test_cache_refactor
branch
2 times, most recently
from
b6e36d4 to
d2551b9
Compare
alex-jw-brooks
force-pushed
the
test_cache_refactor
branch
2 times, most recently
from
2e42e7c to
1e369b2
Compare
…_default_validation_prefix, enable sample_key Signed-off-by: kcirred <[email protected]>
…uad_v2 sampler Signed-off-by: kcirred <[email protected]>
Signed-off-by: kcirred <[email protected]>
alex-jw-brooks
force-pushed
the
test_cache_refactor
branch
from
1e369b2 to
dd355c1
Compare
…oo long Signed-off-by: kcirred <[email protected]>
Signed-off-by: kcirred <[email protected]>
Signed-off-by: kcirred <[email protected]>
…m // model.config.nheads ) Signed-off-by: Rashed Z. Bhatti, PhD <[email protected]>
…g.emb_dim // model.config.nheads )" This reverts commit b3d0f9b.
Signed-off-by: kcirred <[email protected]>
[dpp] store enforce_sizes in log name and added generic kwargs to get_default_validation_prefix
… as modeling code changed Signed-off-by: Joshua Rosenkranz <[email protected]>
…tack/update_llama_model_expectation update llama model expectations tests
…le names now sorted, testing of file names modified for new order Signed-off-by: kcirred <[email protected]>
…rite Prefix rewrite
Signed-off-by: Joshua Rosenkranz <[email protected]>
…tack/fix_test_scripts_assertions fixed test_scripts program assertions
…_cache_refactor Refactor Decoder Tests
alex-jw-brooks
force-pushed
the
rebased_cache_tests
branch
from
ad3073c to
fa5bd38
Compare
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
alex-jw-brooks
force-pushed
the
rebased_cache_tests
branch
from
fa5bd38 to
1566583
Compare
Signed-off-by: Alex-Brooks <[email protected]>
Signed-off-by: Alex-Brooks <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as foundation-model-stack#97 on top of foundation-model-stack#93 (don't have permission to make a branch in the upstream repo yet.
This PR builds on top of foundation-model-stack#20 and foundation-model-stack#93 to add a cache for testing using the refactored version of the test to allow some code reuse. foundation-model-stack#93 should probably be merged first (splitting this out for readability).
Summary of changes (wrt the original cache test PR)
- Makes sure gptq kwargs are passed through to the AIU model
- Makes sure
options={"sendnn.dynamic": COMPILE_DYNAMIC_SENDNN}is passed consistently- Clears the torch sendnn
.cache- the current PR can break if the cache test runs second since the cache paths aren't actually reset in torch sendnn. We reset the compiler settings and clear the directory, but don't clear the spyre cache object in the current PR, which causes alignment issues if the cache test doesn't run first- The current PR runs the check as two tests (cache miss -> cache hit); moves the cache miss test to run as a fixture to set things up so that we can just run cache hit as the test
Note that there is still some weirdness around how micro models are handled, mostly due to the way we configure common models paths / micro model usage and also check thresholds based on whether micro models exist.