Skip to content

Add unit tests for VLLMModel sampling params and config wiring#1287

Open
jayzuccarelli wants to merge 1 commit into
huggingface:mainfrom
jayzuccarelli:add-vllm-model-unit-tests
Open

Add unit tests for VLLMModel sampling params and config wiring#1287
jayzuccarelli wants to merge 1 commit into
huggingface:mainfrom
jayzuccarelli:add-vllm-model-unit-tests

Conversation

@jayzuccarelli

Copy link
Copy Markdown

Note: This resubmits #1276 (same commit). I closed it temporarily while confirming my employer's open-source contribution policy — that's now cleared.

What

Adds unit tests for VLLMModel, as suggested in #724. These cover the pure-Python logic that doesn't need a GPU or a real vllm install (the existing @requires("vllm") gate still applies, so they run under the dev-gpu test env like the other cases in this file):

  • _generateSamplingParams mapping — this is the boundary with vllm that Fix revision arg for vLLM tokenizer #721 showed can break silently. Tests pin down:
    • the greedy scoring path (generate=False): temperature=0, max_tokens=1, prompt_logprobs=1, detokenize=False;
    • the generation path: n, max_tokens, stop, and logprobs are mapped from the runtime args, with temperature carried over from generation_parameters;
    • the num_samples > 1 + temperature == 0 guard raises ValueError.
  • __init__ config wiringadd_special_tokens, max_length, and the tokenizer property are taken from the config (construction kept offline by patching out model/tokenizer creation).

Notes

  • No new dependencies; mirrors the existing mock/patch style in this file. A small _RecordingSamplingParams stand-in is used so the asserted values are actually checked rather than swallowed by a bare Mock.
  • pytest tests/unit/models/vllm/test_vllm_model.py → all green; ruff check / ruff format clean.

Closes #724

Covers the pure-Python logic in VLLMModel that doesn't require a GPU or a vllm install:
- _generate maps runtime args onto vllm SamplingParams (the silent-breakage surface
  from huggingface#721): the greedy scoring path, the generation path, and the
  num_samples>1 + temperature=0 guard.
- __init__ wires add_special_tokens / max_length / tokenizer from the config.

Closes huggingface#724
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FT] Add tests for VLLMModel base methods

1 participant