All pull requests

Pull requests

• feat(rollout): add AutoTP rollout engine with in-memory weight sync

  #9023·OkoroAchraf opened Nov 6, 2026·

  ·
  7

• SuperOffload: run CPU Adam in a thread of the training process, copy gradients to the CPU once

  #9021·xylian86 opened Nov 5, 2026Collaborator·

  ·

• CI: stop pip check from failing the AWS fallback on deepspeed-kernels

  #9020·pengdurice opened Nov 5, 2026Contributor·

  ·
  5

• ZeRO-1/2: one CPU-offload accumulator per group; zero idle gradients

  #9018·sfc-gh-truwase opened Nov 5, 2026Collaborator·

  ·
  1

  1

• ZeRO-1/2: fix offload_states() keeping the optimizer state on the device

  #9016·xylian86 opened Nov 4, 2026Collaborator·

  ·
  4

• ZeRO-3: fix offload_states() with frozen parameters and stale states after reload_states()

  #9015·xylian86 opened Nov 4, 2026Collaborator·

  ·
  5

• ZeRO-3 NVMe offload: fix "param already assigned swap buffer" in the parameter prefetch

  #9014·xylian86 opened Nov 4, 2026Collaborator·

  ·
  2

• [ROCm] Fix inference layer-norm on 64-wide wavefronts

  #9013·rraminen opened Nov 4, 2026Contributor·

  ·
  6

• [AffineIR] Check affine holders and source regions without pairwise comparisons

  #9011·jinyouzhi opened Nov 4, 2026Contributor·

  ·

• [AffineIR] Move shared affine index and box geometry into affine_geometry

  #9010·jinyouzhi opened Nov 4, 2026Contributor·

  ·

• Reuse continuous rollout graphs and release captured resources on shutdown

  #9009·jinyouzhi opened Nov 4, 2026Contributor·

  ·

• Size HybridEngine rollout KV caches from local prefill shapes under AutoTP

  #9008·jinyouzhi opened Nov 4, 2026Contributor·

  ·
  4

• [Hybrid Engine] Enable XPU graph capture behind the explicit opt-in

  #9007·delock opened Nov 4, 2026Collaborator·

  ·
  7

• Fix duplicate CSV monitor headers after restart

  #9005·To3akaRin opened Nov 4, 2026·

  ·
  1

• ZeRO-3: honor stage3_max_reuse_distance for forward-gathered parameters

  #9003·YidaWeng opened Nov 4, 2026·

  ·
  1

  4

• ZeRO-3: partition parameters of modules added after initialization

  #9002·YidaWeng opened Nov 4, 2026·

  ·
  1

  11

• ZeRO-1/2: expose original parameter layout on flat optimizer partitions

  #9001·YidaWeng opened Nov 4, 2026·

  ·
  1

  8

• Fix comms logger IndexError when the tensor is passed by keyword

  #8998·ZhuLinsen opened Nov 3, 2026·

  ·
  1

• feat(comm): one-shot P2P custom all-reduce for TP=2 inference

  #8997·delock opened Nov 3, 2026Collaborator·

  ·
  18

• Fix quantized storage of nontrainable weights in ZeRO-3

  #8995·VenusTZZ opened Nov 3, 2026Contributor·6 tasks done·

  ·
  1

  5

• DeepNVMe: fix silent I/O failures on misaligned buffers

  #8985·xylian86 opened Nov 2, 2026Collaborator·

  ·
  4

• ZeRO-3 NVMe offload: restore the optimizer state on resume

  #8984·xylian86 opened Nov 2, 2026Collaborator·

  ·
  4

• Fix DynamicLossScaler never lowering the scale when fp16 hysteresis is 0 or less

  #8983·damienen opened Nov 2, 2026·

  ·
  1

  5

• ZeRO-3: fix stale and lost gradients of unused parameters

  #8981·xylian86 opened Nov 2, 2026Collaborator·

  ·
  9

• Propagate the user script's exit code from launcher_helper

  #8978·vineethsaivs opened Nov 1, 2026Contributor·

  ·