Massively reduce LayerNorm/RMSNorm training memory usage by sharing saved tensor with other parts of the networks#430
Draft
RuiWang1998 wants to merge 7 commits intoNVIDIA:mainfrom RuiWang1998:rui/dev-mem-eff-ln-operator
+987-312
Commits
Commits on Sep 11, 2023
Commits on Sep 12, 2023
Commits on Sep 13, 2023
Commits on Oct 4, 2023
Commits on Oct 5, 2023
- committed
Commits on Oct 17, 2023
- committed