Remove FlashInfer benchmark backends - #2727
Conversation
There was a problem hiding this comment.
LGTM — clean, mechanical removal of the FlashInfer backend with no dangling references.
What was reviewed: the backend registry change in runner.py and the deleted flashinfer.py/flashinfer_comm.py files, confirming no remaining call sites reference them; the containers.toml entries removed match the deleted backend keys exactly; the torch.py error-message/comment edits correctly drop the now-invalid FlashInfer recommendation; and the leftover flashinfer mentions in sglang.py are unrelated internal SGLang flag comments, not dangling references to the removed backend.
Extended reasoning...
Overview
The diff (8 files, +6/-820) removes the experimental OperatorX "flashinfer" and "flashinfer_comm" backends: deletes the two backend implementation files, removes their entries from _BACKENDS in runner.py and from containers.toml, drops the flashinfer-ep workflow_dispatch choice in .github/workflows/collectivex-sweep.yml, updates example env-var strings in README.md/CLUSTERS.md, and adjusts a stale error message and comment in torch.py that referenced the now-removed backend.
Security risks
None. This is a pure deletion/config-trim of an experimental benchmarking tool; no auth, crypto, or data-handling code is touched.
Level of scrutiny
Low. The change is mechanical (delete two files, remove matching registry/config entries, fix two stale text references) and self-contained to experimental/operatorx/, which is explicitly owned by the PR author per .github/CODEOWNERS (experimental/operatorx/ @ hbarclay). I verified with a repo-wide grep that no other file under experimental/operatorx/ still references the removed flashinfer/flashinfer_comm backend keys — the only remaining flashinfer mentions are in sglang.py comments describing SGLang's own internal kernel-selection flags, unrelated to the deleted OperatorX backend, consistent with the PR description's note that the SGLang backend intentionally still uses FlashInfer kernels internally.
Other factors
The main perf-changelog.yaml (which governs benchmark configs/ recipes) is unrelated to this experimental tool and has no flashinfer entries to remove; no recipe or benchmark config under configs/ was touched, so the changelog-append requirement in AGENTS.md does not apply here. No _zh.md counterpart exists for experimental/operatorx/README.md or CLUSTERS.md, so no synchronized Chinese-doc update was needed. The bug-hunting run reported zero findings and both candidate concerns it investigated (the flashinfer-ep CI option removal being cosmetic relative to CollectiveX/sweep_matrix.py, which is out of scope for this diff) were reasonably ruled out.
Remove the FlashInfer compute and collective backends from OperatorX, their container scheduling entries, and the FlashInfer EP option from the CollectiveX sweep workflow.
The SGLang backend remains unchanged even where SGLang internally selects FlashInfer kernels.
Note
Low Risk
Benchmark and CI configuration removal only; production paths via SGLang are unchanged, though CollectiveX flashinfer-ep harness code remains but is no longer sweepable from the workflow.
Overview
Removes FlashInfer as first-class OperatorX NVIDIA backends, deleting the compute (
flashinfer) and collective (flashinfer_comm) implementations and dropping them from the runner allowlist andcontainers.tomlimage entries.Cluster docs and example
OPERATORX_BACKENDSvalues no longer listflashinfer; recommended b300 runs aretorch,deepgemm,sglangonly. SGLang remains and can still route to FlashInfer kernels internally—only the dedicated benchmark backends are gone. Torch backend error text no longer points users at FlashInfer for unsupported fp8 attention.CollectiveX CI: the
collectivex-sweepworkflow’sbackenddispatch choice no longer includesflashinfer-ep(other EP backends unchanged).Reviewed by Cursor Bugbot for commit 799e0d6. Bugbot is set up for automated code reviews on this repo. Configure here.