Problem
Running python infer_split_merge.py --help currently fails on environments where torch (and/or other heavy deps) are not installed, because infer_split_merge.py
imports torch and str2bool inside main() before argparse parses args.
This makes it harder for new users to discover parameters/usage without first setting up a full GPU environment.
Proposed Fix
- Move heavy imports (e.g.,
import torch) to after args = parser.parse_args().
- Optionally keep a small helper so users can view CLI docs even without full deps.
Notes
Normal execution still requires torch/vLLM; this change is only to improve CLI discoverability.
Problem
Running
python infer_split_merge.py --helpcurrently fails on environments wheretorch(and/or other heavy deps) are not installed, becauseinfer_split_merge.pyimports
torchandstr2boolinsidemain()beforeargparseparses args.This makes it harder for new users to discover parameters/usage without first setting up a full GPU environment.
Proposed Fix
import torch) to afterargs = parser.parse_args().Notes
Normal execution still requires torch/vLLM; this change is only to improve CLI discoverability.