feat(config): add GLM-5.2 NVFP4 B200 Dynamo-SGLang AgentX recipes / 新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 配方 - #2673
feat(config): add GLM-5.2 NVFP4 B200 Dynamo-SGLang AgentX recipes / 新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 配方#2673Ankur-singh wants to merge 7 commits into
Conversation
新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 聚合式与分离式配方：覆盖并发度 2/4/8/12 的聚合 TP8，以及并发度 113 的 1P2D 和 1P3D 分离式配置，使用 EAGLE MTP、HiCache DRAM 卸载和 NIXL KV 传输，吞吐任务注入 golden synthetic acceptance。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2673 链接。
将 GLM-5.2 B200 AgentX 聚合式与分离式配置改由 Nscale 集群运行：注册新的 cluster:b200-nscale 精确机群标签，扩展 launch_b200-nscale-slurm.sh 支持该模型/框架组合并对本地 NVMe 模型路径跳过 srtctl preflight，同时撤销此前对 launch_b200-dgxc.sh 的改动。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32224799614 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
sbatch_directives.cpus-per-task was ported verbatim from NVIDIA/srt-slurm#314 (a DGXC-tuned value) and does not match the b200-nscale cluster's registered per-node CPU count, causing every job routed there to fail at submission with 'sbatch: error: CPU count per node can not be satisfied'. All three recipes already request the whole node via use_exclusive_sbatch_directive: true, so the explicit cpus-per-task constraint is redundant on top of --exclusive and safe to drop entirely (precedent: sglang/glm5.2/agentic/disagg-h200-2p2d-*.yaml already omits it).
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32311694482 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32311923080 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4ef323c. Configure here.
| - b200-nscale-slurm_5 | ||
| - b200-nscale-slurm_6 | ||
| - b200-nscale-slurm_7 | ||
| - b200-nscale-slurm_8 |
There was a problem hiding this comment.
Duplicate nscale cluster runner mapping
Low Severity
cluster:b200-nscale is inserted again even though the same key and runner list already exist earlier in runners.yaml. Duplicate YAML keys are last-write-wins, so a later edit to only one block can be silently ignored.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 4ef323c. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32931114989 |
2 participants
Description
Adds Dynamo-SGLang AgentX coverage for GLM-5.2-NVFP4 on B200: one aggregated
TP8 config and one disaggregated config carrying two topologies (1P2D, 1P3D).
glm5.2-fp4-b200-dynamo-sglang-agentic-agg: single TP8 worker doing bothprefill and decode, swept at concurrency 2, 4, 8, and 12.
glm5.2-fp4-b200-dynamo-sglang-agentic-disagg: EP8/DP-attention prefillagainst either 2 or 3 TP8 decode workers (1P2D / 1P3D), both at concurrency
113, using NIXL for KV transfer between prefill and decode.
acceptance to the golden values in
golden_al_distribution/glm5.2_mtp.yaml(2.99 for the aggregate 3-step config, 2.5 for the disaggregated 2-step
decode config) for throughput runs only —
EVAL_ONLYruns keep realtarget-model verification.
benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b200-fp4/agentic/and copied into the srt-slurm checkout by a new
launch_b200-dgxc.shbranch, pinned to srt-slurm
v1.0.53(
217f94387abeddfed7149a71955dc523e07cd765).SRT_SLURM_MODEL_PREFIXfor this model/precision fromglm5.2-fp4toglm-5.2-fp4so it matches themodel.paththe recipesdeclare — this model-prefix alias was previously unused by any multi-node
config, so the fix has no effect on existing entries.
中文说明
为 B200 上的 GLM-5.2-NVFP4 新增 Dynamo-SGLang AgentX 覆盖:一个聚合式 TP8 配置,
以及一个包含两种拓扑(1P2D、1P3D)的分离式配置。
glm5.2-fp4-b200-dynamo-sglang-agentic-agg:单个 TP8 worker 同时承担 prefill和 decode,在并发度 2、4、8、12 下扫描。
glm5.2-fp4-b200-dynamo-sglang-agentic-disagg:EP8/DP-attention 的 prefill分别对接 2 个或 3 个 TP8 decode worker(1P2D / 1P3D),均在并发度 113 下运行,
prefill 与 decode 之间使用 NIXL 传输 KV。
acceptance 固定为
golden_al_distribution/glm5.2_mtp.yaml中的 golden 值(聚合3-step 配置为 2.99,分离式 2-step decode 配置为 2.5);
EVAL_ONLY运行则保留真实目标模型验证。
benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b200-fp4/agentic/,由新增的
launch_b200-dgxc.sh分支复制进 srt-slurm 检出目录,固定至 srt-slurmv1.0.53(217f94387abeddfed7149a71955dc523e07cd765)。SRT_SLURM_MODEL_PREFIX从glm5.2-fp4修正为glm-5.2-fp4,使其与配方声明的model.path一致——此模型前缀别名此前未被任何多机配置使用,因此该修正不影响现有条目。
The serving image is the upstream SGLang image
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642. TRT-LLM is only used byselected SGLang kernel backends in this recipe.
服务镜像为上游 SGLang 镜像
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642。本配方中的 TRT-LLM 仅用于部分 SGLang kernel backend。
Related Issue
N/A
Type of Change
Validation
process_changelog.pyagainst this diff generates six throughput jobs(four aggregate concurrency points, two disaggregated topologies) and two
AgentX eval jobs (one per disaggregated topology).
generate_sweep_configs.py test-configresolves both new config keys withno errors.
YAML parsing,
bash -non the launch script, andgit diff --checkallpass.
Golden synthetic-acceptance lengths (2.99 aggregate / 2.5 disaggregated)
match
golden_al_distribution/glm5.2_mtp.yaml.Synthetic acceptance is injected through the shared
inject_synthetic_acceptance.pylauncher helper, which is a no-op wheneverEVAL_ONLYis set, so eval runs keep real target-model verification.针对本次 diff 运行
process_changelog.py,生成六个吞吐任务(四个聚合并发点、两个分离式拓扑)和两个 AgentX eval 任务(每个分离式拓扑一个)。
generate_sweep_configs.py test-config可正确解析两个新增 config key,无报错。YAML 解析、launch 脚本的
bash -n,以及git diff --check均通过。Golden synthetic acceptance 长度(聚合 2.99 / 分离式 2.5)与
golden_al_distribution/glm5.2_mtp.yaml一致。Synthetic acceptance 通过共享的
inject_synthetic_acceptance.pylauncherhelper 注入,在设置
EVAL_ONLY时为 no-op,因此 eval 运行仍保留真实目标模型验证。
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.Note
Low Risk
Benchmark and Slurm launcher configuration only; no changes to core serving or auth paths, with scope limited to new matrix entries and nscale launch wiring.
Overview
Adds AgentX agentic-coding benchmark coverage for GLM-5.2-NVFP4 on B200 via Dynamo + disaggregated or aggregated SGLang, wired to the new
cluster:b200-nscalerunner pool.Three checked-in srt-slurm recipes (
glm5.2-agentx-agg,1p2d,1p3d) define TP8 serving (EAGLE MTP, HiCache DRAM offload, NIXL KV transfer for disagg, KV router session affinity) and runagentic_srt.sh.nvidia-master.yamlregistersglm5.2-fp4-b200-dynamo-sglang-agentic-agg(concurrency 2/4/8/12) and…-disagg(1P2D and 1P3D at concurrency 113), pointing each sweep point at the matching recipe and enabling golden synthetic MTP acceptance only for throughput jobs.launch_b200-nscale-slurm.shnow acceptsglm5.2/fp4/dynamo-sglang/MTP, stagesGLM-5.2-NVFP4from node-local scratch, clones srt-slurm v1.0.53, copies the vendored recipes, and skips preflight like other large on-node weights.runners.yamlandperf-changelog.yamldocument the new cluster and config keys.Reviewed by Cursor Bugbot for commit 158aafc. Bugbot is set up for automated code reviews on this repo. Configure here.