Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve batch filtering efficiency and preserve original batch type #9997

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

lcaquot94
Copy link

This PR improves DataBatch and HeteroDataBatch filtering:

  • Keeps Original Batch Type: Previously, batch[mask] used to return a data list, requiring reconversion with Batch.from_data_list(batch[mask]). Now, it directly returns a filtered Batch, preserving the original type.
  • Faster Filtering: The old approach used to split the batch in order to apply the mask, which was inefficient, especially with batch reconstruction afterward. The new implementation applies the mask directly to all batch components, achieving up to 10x speedup.

Note: If the user still needs a data list, they can convert the result with batch[mask].to_data_list(), or use the previous method: batch.index_select(mask).

@lcaquot94
Copy link
Author

Here is a discussion related to this PR : #9681

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants