Code

Add mixed DP + TP#3229

Code

Open

lzhangzz wants to merge 45 commits intoInternLM:mainInternLM/lmdeploy:mainfrom lzhangzz:dplzhangzz/lmdeploy:dpCopy head branch name to clipboard

Commits

Commits on Jan 17, 2025

• comm abstraction

  lzhangzz

  committed
  f5a6264

  Copy full SHA for f5a6264

Commits on Jan 26, 2025

• add custom

  lzhangzz

  committed
  25947d0

  Copy full SHA for 25947d0

Commits on Feb 11, 2025

• fused rms norm

  lzhangzz

  committed
  aa07408

  Copy full SHA for aa07408

Commits on Feb 12, 2025

• refactor

  lzhangzz

  committed
  2acd453

  Copy full SHA for 2acd453

Commits on Feb 14, 2025

• push-based kernel

  lzhangzz

  committed
  8240017

  Copy full SHA for 8240017

Commits on Feb 16, 2025

• optimize for small hidden dims

  lzhangzz

  committed
  74c169e

  Copy full SHA for 74c169e

Commits on Feb 17, 2025

• Merge remote-tracking branch 'origin/main' into comm

  lzhangzz

  committed
  3c70049

  Copy full SHA for 3c70049

Commits on Feb 20, 2025

• integration

  lzhangzz

  committed
  15b88f5

  Copy full SHA for 15b88f5

• clean up

  lzhangzz

  committed
  c10b46b

  Copy full SHA for c10b46b

• export options & fix things

  lzhangzz

  committed
  2028de6

  Copy full SHA for 2028de6

Commits on Feb 24, 2025

• allgather2d & VMM allocation

  lzhangzz

  committed
  4c48593

  Copy full SHA for 4c48593

• optimize allgather2d

  lzhangzz

  committed
  bcbf9e4

  Copy full SHA for bcbf9e4

• Merge remote-tracking branch 'origin/main' into comm

  lzhangzz

  committed
  3c96080

  Copy full SHA for 3c96080

• remove obsolete comm utils

  lzhangzz

  committed
  d67498f

  Copy full SHA for d67498f

• handle non-multi-gpu build

  lzhangzz

  committed
  cc2f364

  Copy full SHA for cc2f364

• fix lint

  lzhangzz

  committed
  9bf92b8

  Copy full SHA for 9bf92b8

• fix lint

  lzhangzz

  committed
  29fff01

  Copy full SHA for 29fff01

• avoid using mscclpp repo (some deps are not needed)

  lzhangzz

  committed
  37d661b

  Copy full SHA for 37d661b

• fix lint

  lzhangzz

  committed
  3e33fa3

  Copy full SHA for 3e33fa3

Commits on Feb 25, 2025

• fix nccl version & clean up deps

  lzhangzz

  committed
  a864d4a

  Copy full SHA for a864d4a

• fix lint

  lzhangzz

  committed
  97d8722

  Copy full SHA for 97d8722

• custom -> native

  lzhangzz

  committed
  f9f76e5

  Copy full SHA for f9f76e5

• rename

  lzhangzz

  committed
  e1e628e

  Copy full SHA for e1e628e

• fix p-lora

  lzhangzz

  committed
  75cdbe0

  Copy full SHA for 75cdbe0

• fix lm head

  lzhangzz

  committed
  0423ef1

  Copy full SHA for 0423ef1

• log fatal exception explicitly

  lzhangzz

  committed
  9987879

  Copy full SHA for 9987879

Commits on Feb 27, 2025

• initial data parallel support

  lzhangzz

  committed
  f0d42e0

  Copy full SHA for f0d42e0

Commits on Mar 7, 2025

• sync dp + tp

  lzhangzz

  committed
  78cc5ac

  Copy full SHA for 78cc5ac

Commits on Mar 10, 2025

• mixed d*t0 | t1

  lzhangzz

  committed
  ecf8e3a

  Copy full SHA for ecf8e3a

• Merge remote-tracking branch 'origin/main' into dp

  lzhangzz

  committed
  eb12b73

  Copy full SHA for eb12b73

Commits on Mar 11, 2025

• refactor

  lzhangzz

  committed
  ff61d6e

  Copy full SHA for ff61d6e

Commits on Mar 12, 2025

• refactor

  lzhangzz

  committed
  1847f8b

  Copy full SHA for 1847f8b

Commits on Mar 13, 2025

• fix ut

  lzhangzz

  committed
  2aba5e7

  Copy full SHA for 2aba5e7

Commits on Mar 14, 2025

• refactor

  lzhangzz

  committed
  8a47f4d

  Copy full SHA for 8a47f4d

• asymmetrical allreduce

  lzhangzz

  committed
  8883b56

  Copy full SHA for 8883b56

• fix

  lzhangzz

  committed
  31c3638

  Copy full SHA for 31c3638

Commits on Mar 17, 2025

• fix nccl<2.18

  lzhangzz

  committed
  4950b54

  Copy full SHA for 4950b54

• fix

  lzhangzz

  committed
  8f4039e

  Copy full SHA for 8f4039e

• fix

  lzhangzz

  committed
  6b6dbf3

  Copy full SHA for 6b6dbf3

• fix

  lzhangzz

  committed
  c63b7cb

  Copy full SHA for c63b7cb

• fix

  lzhangzz

  committed
  4f37013

  Copy full SHA for 4f37013

• fix converter

  lzhangzz

  committed
  f5538bb

  Copy full SHA for f5538bb

• fix converter

  lzhangzz

  committed
  b7502e6

  Copy full SHA for b7502e6

Commits on Mar 18, 2025

• fix tp for converted model

  lzhangzz

  committed
  c0e5c33

  Copy full SHA for c0e5c33

• fix tp for converted model

  lzhangzz

  committed
  dbba29f

  Copy full SHA for dbba29f