Add unit tests for VLLMModel sampling params and config wiring#1287
Open
jayzuccarelli wants to merge 1 commit into
Open
Add unit tests for VLLMModel sampling params and config wiring#1287jayzuccarelli wants to merge 1 commit into
jayzuccarelli wants to merge 1 commit into
Conversation
Covers the pure-Python logic in VLLMModel that doesn't require a GPU or a vllm install: - _generate maps runtime args onto vllm SamplingParams (the silent-breakage surface from huggingface#721): the greedy scoring path, the generation path, and the num_samples>1 + temperature=0 guard. - __init__ wires add_special_tokens / max_length / tokenizer from the config. Closes huggingface#724
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds unit tests for
VLLMModel, as suggested in #724. These cover the pure-Python logic that doesn't need a GPU or a real vllm install (the existing@requires("vllm")gate still applies, so they run under thedev-gputest env like the other cases in this file):_generate→SamplingParamsmapping — this is the boundary with vllm that Fix revision arg for vLLM tokenizer #721 showed can break silently. Tests pin down:generate=False):temperature=0,max_tokens=1,prompt_logprobs=1,detokenize=False;n,max_tokens,stop, andlogprobsare mapped from the runtime args, withtemperaturecarried over fromgeneration_parameters;num_samples > 1+temperature == 0guard raisesValueError.__init__config wiring —add_special_tokens,max_length, and thetokenizerproperty are taken from the config (construction kept offline by patching out model/tokenizer creation).Notes
_RecordingSamplingParamsstand-in is used so the asserted values are actually checked rather than swallowed by a bareMock.pytest tests/unit/models/vllm/test_vllm_model.py→ all green;ruff check/ruff formatclean.Closes #724