Skip to content

Remove FlashInfer benchmark backends - #2727

Open
hbarclay wants to merge 1 commit into
mainfrom
hbarclay/remove-flashinfer-backends
Open

Remove FlashInfer benchmark backends#2727
hbarclay wants to merge 1 commit into
mainfrom
hbarclay/remove-flashinfer-backends

Conversation

@hbarclay

@hbarclay hbarclay commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Remove the FlashInfer compute and collective backends from OperatorX, their container scheduling entries, and the FlashInfer EP option from the CollectiveX sweep workflow.

The SGLang backend remains unchanged even where SGLang internally selects FlashInfer kernels.


Note

Low Risk
Benchmark and CI configuration removal only; production paths via SGLang are unchanged, though CollectiveX flashinfer-ep harness code remains but is no longer sweepable from the workflow.

Overview
Removes FlashInfer as first-class OperatorX NVIDIA backends, deleting the compute (flashinfer) and collective (flashinfer_comm) implementations and dropping them from the runner allowlist and containers.toml image entries.

Cluster docs and example OPERATORX_BACKENDS values no longer list flashinfer; recommended b300 runs are torch,deepgemm,sglang only. SGLang remains and can still route to FlashInfer kernels internally—only the dedicated benchmark backends are gone. Torch backend error text no longer points users at FlashInfer for unsupported fp8 attention.

CollectiveX CI: the collectivex-sweep workflow’s backend dispatch choice no longer includes flashinfer-ep (other EP backends unchanged).

Reviewed by Cursor Bugbot for commit 799e0d6. Bugbot is set up for automated code reviews on this repo. Configure here.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — clean, mechanical removal of the FlashInfer backend with no dangling references.

What was reviewed: the backend registry change in runner.py and the deleted flashinfer.py/flashinfer_comm.py files, confirming no remaining call sites reference them; the containers.toml entries removed match the deleted backend keys exactly; the torch.py error-message/comment edits correctly drop the now-invalid FlashInfer recommendation; and the leftover flashinfer mentions in sglang.py are unrelated internal SGLang flag comments, not dangling references to the removed backend.

Extended reasoning...

Overview

The diff (8 files, +6/-820) removes the experimental OperatorX "flashinfer" and "flashinfer_comm" backends: deletes the two backend implementation files, removes their entries from _BACKENDS in runner.py and from containers.toml, drops the flashinfer-ep workflow_dispatch choice in .github/workflows/collectivex-sweep.yml, updates example env-var strings in README.md/CLUSTERS.md, and adjusts a stale error message and comment in torch.py that referenced the now-removed backend.

Security risks

None. This is a pure deletion/config-trim of an experimental benchmarking tool; no auth, crypto, or data-handling code is touched.

Level of scrutiny

Low. The change is mechanical (delete two files, remove matching registry/config entries, fix two stale text references) and self-contained to experimental/operatorx/, which is explicitly owned by the PR author per .github/CODEOWNERS (experimental/operatorx/ @ hbarclay). I verified with a repo-wide grep that no other file under experimental/operatorx/ still references the removed flashinfer/flashinfer_comm backend keys — the only remaining flashinfer mentions are in sglang.py comments describing SGLang's own internal kernel-selection flags, unrelated to the deleted OperatorX backend, consistent with the PR description's note that the SGLang backend intentionally still uses FlashInfer kernels internally.

Other factors

The main perf-changelog.yaml (which governs benchmark configs/ recipes) is unrelated to this experimental tool and has no flashinfer entries to remove; no recipe or benchmark config under configs/ was touched, so the changelog-append requirement in AGENTS.md does not apply here. No _zh.md counterpart exists for experimental/operatorx/README.md or CLUSTERS.md, so no synchronized Chinese-doc update was needed. The bug-hunting run reported zero findings and both candidate concerns it investigated (the flashinfer-ep CI option removal being cosmetic relative to CollectiveX/sweep_matrix.py, which is out of scope for this diff) were reasonably ruled out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant