pyTorch supports distributed training and some example codes are provided at the official website. This repository contains more specific and concrete example codes for various distributed training scenario. The list of codes are described bellow.
- example code for multi-node multi-gpu DP with spwan : multinodeDP.py
- example code for multi-node multi-gpu MP with spwan : multinodeMP.py