-
-
Notifications
You must be signed in to change notification settings - Fork 11.3k
Remove unused swap_space parameter #27988
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
|
Documentation preview: https://vllm--27988.org.readthedocs.build/en/27988/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request aims to remove the unused swap_space parameter throughout the codebase. The changes are comprehensive, touching documentation, tests, and core configuration files. While most of the changes are correct, I've identified a critical issue in vllm/engine/arg_utils.py that appears to be a typo. This typo will likely cause a KeyError during command-line argument parsing, breaking the CLI. I've provided a suggestion to fix this.
vllm/engine/arg_utils.py
Outdated
| ) | ||
| cache_group.add_argument("--swap-space", **cache_kwargs["swap_space"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There seems to be a typo here. The get_kwargs(CacheConfig) function returns a dictionary with keys corresponding to the fields in CacheConfig. CacheConfig has a field named cache_dtype, but not kv_cache_dtype. Accessing cache_kwargs["kv_cache_dtype"] will likely cause a KeyError. This should probably be cache_kwargs["cache_dtype"] as it was before this change.
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) | |
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
vllm/engine/arg_utils.py
Outdated
| cache_group.add_argument( | ||
| "--kv-cache-memory-bytes", **cache_kwargs["kv_cache_memory_bytes"] | ||
| ) | ||
| cache_group.add_argument("--swap-space", **cache_kwargs["swap_space"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["cache_dtype"]) | ||
| cache_group.add_argument("--kv-cache-dtype", **cache_kwargs["kv_cache_dtype"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix CLI lookup for kv cache dtype option
The new code builds the CLI by fetching cache_kwargs["kv_cache_dtype"], but the CacheConfig dataclass still exposes the field as cache_dtype. get_kwargs(CacheConfig) therefore produces a dictionary with key "cache_dtype", and this lookup now raises a KeyError as soon as the argument parser is constructed (e.g. running python -m vllm.entrypoints.llm --help). This prevents the CLI from starting at all. Keep the lookup keyed on "cache_dtype" or rename the field consistently.
Useful? React with 👍 / 👎.
85c7aaf to
2d12c02
Compare
hmellor
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see a few more references that aren't removed by this PR: 
cc @ApostaC does LMCache depend on the swap_space argument at all in vllm/distributed/kv_transfer/kv_connector/v1/lmcache_integration/vllm_v1_adapter.py?
|
Hey @hmellor , we don't use that at all. |
References: #27984
Purpose
Remove unused
swap_spaceparameter after the removal ofbest_ofsampling. Fixes #27984Test Plan
(Untested) working on it.
Help would be appreciated. I'm having quite a lot of difficulty compiling vllm and all its dependencies. This should be a simple fix with no impact anywhere else.
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.