Skip to content

Add CUDA memory pool allocation function aliases for MUSA compatibility#61

Merged
yeahdongcn merged 4 commits intoMooreThreads:mainfrom
popsiclexu:xzx/mem_pool
May 7, 2026
Merged

Add CUDA memory pool allocation function aliases for MUSA compatibility#61
yeahdongcn merged 4 commits intoMooreThreads:mainfrom
popsiclexu:xzx/mem_pool

Conversation

@popsiclexu
Copy link
Copy Markdown
Contributor

  • Add CUDA memory pool allocation function aliases (_cuda_beginAllocateCurrentThreadToPool, _cuda_endAllocateToPool, _cuda_releasePool) in the C++ extension, backed by MUSA allocator APIs, to support from torch.cuda.memory import ... usage on MUSA devices
  • Implement thread-based filtering in _musa_beginAllocateCurrentThreadToPool to match CUDA semantics (instead of the stream-based fallback from _patch.py)
  • Inject these functions into torch.musa.memory at C++ extension load time, so downstream code using CUDA memory pool APIs works transparently on MUSA

@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented May 6, 2026

🤖 Augment PR Summary

Summary: This PR improves CUDA-API compatibility on MUSA by exposing CUDA-named memory pool allocation helpers through torchada’s C++ extension.

Changes:

  • Adds C++ bindings for _cuda_beginAllocateCurrentThreadToPool, _cuda_endAllocateToPool, and _cuda_releasePool backed by MUSA caching allocator APIs.
  • Implements thread-based filtering for “current thread” semantics when allocating to a pool.
  • Injects the new functions into torch.musa.memory when the extension loads so torch.cuda.memory imports work under torchada’s CUDA->MUSA redirection.

Technical Notes: Injection happens after the extension is successfully built/loaded and marked as loaded, and is intended to make downstream CUDA memory pool API calls operate transparently on MUSA.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread src/torchada/_cpp_ops.py Outdated
Comment thread src/torchada/_cpp_ops.py Outdated
@popsiclexu popsiclexu force-pushed the xzx/mem_pool branch 2 times, most recently from 0fb07bb to 28d1545 Compare May 6, 2026 11:50
Comment thread src/torchada/_cpp_ops.py
Comment thread src/torchada/_patch.py Outdated
Comment thread src/torchada/_patch.py Outdated
yeahdongcn added 3 commits May 7, 2026 14:52
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
@yeahdongcn yeahdongcn merged commit e4054d9 into MooreThreads:main May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants