Add a Heat aware DistributedSampler for torch usage.

[Berkant03]

Due Diligence

• General:
  • title of the PR is suitable to appear in the Release Notes
• Implementation:
  • unit tests: all split configurations tested
  • unit tests: multiple dtypes tested
  • benchmarks: created for new functionality
  • benchmarks: performance improved or maintained
  • documentation updated where needed

Description

Issue/s resolved: #1789

Changes proposed:

Add a Heat aware DistributedSampler for usage for PyTorch use cases

Type of change

• New feature (non-breaking change which adds functionality)

Does this change modify the behaviour of other functions? If so, which?

no

[github-actions]

Thank you for the PR!

[codecov]

Codecov Report

Attention: Patch coverage is 18.51852% with 88 lines in your changes missing coverage. Please review.

Project coverage is 91.62%. Comparing base (23e373b) to head (875643d).
Report is 58 commits behind head on main.

Files with missing lines	Patch %	Lines
heat/utils/data/datatools.py	18.51%	88 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1807      +/-   ##
==========================================
- Coverage   92.26%   91.62%   -0.64%     
==========================================
  Files          84       84              
  Lines       12447    12554     +107     
==========================================
+ Hits        11484    11503      +19     
- Misses        963     1051      +88     

Flag	Coverage Δ
unit	91.62% <18.51%> (-0.64%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[github-actions]

Thank you for the PR!

[Berkant03]

When using the normal comm.Bcast the Bcast only works the first time and the seconds time not anymore.

❯ mpirun -np 2 python test.py
0 tensor([2, 4, 3, 0, 1], dtype=torch.int32)
1 tensor([2, 4, 3, 0, 1], dtype=torch.int32)
...
1 tensor([455,   0,   0,   0,  32], dtype=torch.int32)
0 tensor([1, 4, 3, 2, 0], dtype=torch.int32)