Multi GPU support for vision model training in C++ with libtorch

[Bycob]

🚀 Feature

Multi GPU support for vision model training in C++ with libtorch.

Motivation

More and more systems rely only on C++ for deep learning, including ours. But at the moment the C++ version of vision models are quite unpractical to use, and it is not possible to train them on multiple GPU when using large amount of data.

Pitch

Make C++ models support multi GPU. This can be done by deriving the pytorch ICloneable class, but this might not be sufficient for optimal GPU training, as pointed out in this issue. The best option might be to implement the clone method so that data_parallel (here) can handle the models.

[linjiapro]

Thanks Louis!