Code

Domino#279

Code

Open

xrsrke wants to merge 16 commits intomainhuggingface/nanotron:mainfrom xrsrke/dominohuggingface/nanotron:xrsrke/dominoCopy head branch name to clipboard

+701-75

Commits

Commits on Jan 29, 2025

• first draft of domino forward pass

  xrsrke

  committed
  d7bf8be

  Copy full SHA for d7bf8be

Commits on Jan 30, 2025

• support the backward pass

  xrsrke

  committed
  3803b19

  Copy full SHA for 3803b19

Commits on Jan 31, 2025

• the first draft for bwd overlapping

  xrsrke

  committed
  d765fd5

  Copy full SHA for d765fd5

Commits on Feb 3, 2025

• add backward pass overlapping

  xrsrke

  committed
  9924608

  Copy full SHA for 9924608

Commits on Feb 4, 2025

• fix some ops dont execute in the bwd pass

  xrsrke

  committed
  d6bc8da

  Copy full SHA for d6bc8da

Commits on Feb 5, 2025

• fix can't find an ops in fwd

  xrsrke

  committed
  93b2f10

  Copy full SHA for 93b2f10

• partially overlapping bwd pass

  xrsrke

  committed
  31db05d

  Copy full SHA for 31db05d

Commits on Feb 10, 2025

• fix stream not sync

  xrsrke

  committed
  23f2108

  Copy full SHA for 23f2108

Commits on Feb 11, 2025

• add cuda stream syncronization for the bwd pass

  xrsrke

  committed
  eac4ac5

  Copy full SHA for eac4ac5

• domino but non_async_last_batch_mlp_and_non_async_first_batch_attn

  xrsrke

  committed
  3a438ff

  Copy full SHA for 3a438ff

• non_async_last_batch_mlp_and_non_async_first_batch_attn

  xrsrke

  committed
  da948df

  Copy full SHA for da948df

Commits on Feb 12, 2025

• backup before refactoring

  xrsrke

  committed
  aa3e973

  Copy full SHA for aa3e973

• refactor

  xrsrke

  committed
  ea09a25

  Copy full SHA for ea09a25

• refactor

  xrsrke

  committed
  7761d82

  Copy full SHA for 7761d82

• remove dependency.py and comments

  xrsrke

  committed
  b11e48f

  Copy full SHA for b11e48f

Commits on Feb 14, 2025

• add unit tests for async bucket, and WaitComm

  xrsrke

  committed
  893ff07

  Copy full SHA for 893ff07