Skip to content

Conversation

crcrpar
Copy link
Collaborator

@crcrpar crcrpar commented Jul 20, 2023

This is merely cosmetic as the current parametrization creates test case names such as NcclDistributedFusedLAMB.test_distributed_fused_lamb_no_copy_True_opt_kwargs_{'overlap_reductions': False, 'dwu_num_blocks': 1, 'dwu_num_chunks': 1, 'fused_norm': True, 'fuse_scale': True, 'clip_after_ar': False} whic is not ideally friendly to keyword filtering.


it might be worth thinking of removing no_copy check here

# Check if collectives have no_copy option
self._reduce_scatter_no_copy = (
'no_copy' in inspect.getfullargspec(torch.distributed.reduce_scatter).args
)
self._all_gather_no_copy = (
'no_copy' in inspect.getfullargspec(torch.distributed.all_gather).args
)

cc @ptrblck @eqy @Aidyn-A

crcrpar added 2 commits July 31, 2023 14:00
Signed-off-by: Masaki Kozuki <[email protected]>
Signed-off-by: Masaki Kozuki <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant