config(dsv4): constrain OpenMP threads for GB300 AgentX / 限制 GB300 AgentX 的 OpenMP 线程数 - #2711
config(dsv4): constrain OpenMP threads for GB300 AgentX / 限制 GB300 AgentX 的 OpenMP 线程数#2711RohitNagraj wants to merge 3 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32681308155 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32681308155 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32880090147 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32880090147 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32880090147 |
为 GB300 AgentX 的预填充和解码工作进程限制 OpenMP 线程数。
在性能变更日志中补充 PR #2711 链接。
让 GB300 AgentX 的前端和基准测试环境继承 KV 卸载设置。
3231cae to
872b82e
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32880090147 |
1 participant
Description
Set
OMP_NUM_THREADS=1in both the prefill and decode worker environments across all six DeepSeek-V4-Pro GB300 Dynamo-TensorRT-LLM AgentX recipes. This keeps OpenMP thread creation consistently bounded across the recipe matrix.Remove the explicit
KV_OFFLOADINGoverrides from the frontend and benchmark environments so the runtime-provided setting is inherited.Local validation covered YAML parsing, exact-key matrix generation, focused master-to-recipe parity, all six srt-slurm recipe dry runs, and the GB300 launcher contract tests.
中文说明
在全部六个 DeepSeek-V4-Pro GB300 Dynamo-TensorRT-LLM AgentX 配方的预填充和解码工作进程环境中设置
OMP_NUM_THREADS=1，从而在整个配置矩阵中统一限制 OpenMP 线程创建数量。移除前端和基准测试环境中显式设置的
KV_OFFLOADING，使其继承运行时提供的设置。本地验证包括 YAML 解析、精确配置键矩阵生成、主配置与配方的定向一致性检查、六个 srt-slurm 配方的 dry-run，以及 GB300 启动器契约测试。
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.Note
Medium Risk
Recipe-only but alters benchmark worker threading and KV offloading inheritance, which can change measured AgentX performance versus prior sweeps.
Overview
Updates all six DeepSeek-V4-Pro GB300 Dynamo–TensorRT-LLM AgentX srt-slurm recipes with two environment tweaks.
OMP_NUM_THREADS: '1'is added to bothprefill_environmentanddecode_environmentso OpenMP thread usage is capped consistently across the recipe matrix.The explicit
KV_OFFLOADING: noneentries are removed from frontend and benchmarkenvblocks so those jobs inherit the runtime-provided KV offloading setting instead of pinning it in YAML.A
perf-changelog.yamlentry documents the change under config keydsv4-fp4-gb300-dynamo-trt-agentx(agentic-coding).Reviewed by Cursor Bugbot for commit 872b82e. Bugbot is set up for automated code reviews on this repo. Configure here.