Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(fp8_cast): Add robust memory management and error handling #563

Closed
wants to merge 1 commit into from

Conversation

ajwise9
Copy link

@ajwise9 ajwise9 commented Feb 4, 2025

  • Add try-catch block for memory management operations
  • Implement graceful error handling for memory allocation failures
  • Add explicit CUDA memory cleanup
  • Protect against potential race conditions in file loading

This change improves stability when converting large models by:

  • Preventing crashes from out-of-memory conditions
  • Ensuring proper cleanup of GPU resources
  • Adding error reporting for debugging

@ajwise9
Copy link
Author

ajwise9 commented Feb 4, 2025

fix(fp8_cast): Add robust memory management and error handling

@ajwise9 ajwise9 closed this Feb 6, 2025
@ajwise9 ajwise9 reopened this Feb 6, 2025
@mowentian
Copy link
Contributor

no fallback strategy for oom error, so just raise err and failed is simple and fine

@mowentian mowentian closed this Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants