Skip to content

Add zero-copy ByteBuffer image prefill API for Android LlmModule#17767

Open
psiddh wants to merge 1 commit intomainfrom
android_improv
Open

Add zero-copy ByteBuffer image prefill API for Android LlmModule#17767
psiddh wants to merge 1 commit intomainfrom
android_improv

Conversation

@psiddh
Copy link
Contributor

@psiddh psiddh commented Feb 28, 2026

The existing prefillImages(int[]) and prefillImages(float[]) APIs perform two full data copies across the JNI boundary: one via getRegion() into a temporary vector, then an element-by-element conversion into the final vector. This adds new prefillImages(ByteBuffer) and prefillNormalizedImages(ByteBuffer) overloads that use GetDirectBytes() on direct ByteBuffers to access the native memory pointer directly, eliminating the JNI array copy overhead.

Copilot AI review requested due to automatic review settings February 28, 2026 07:05
@psiddh psiddh requested a review from kirklandsign as a code owner February 28, 2026 07:05
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 28, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17767

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Unrelated Failures

As of commit 2852bd7 with merge base 5f879ca (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 28, 2026
@github-actions
Copy link

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds new Android LlmModule image prefill overloads that accept direct ByteBuffers and new corresponding JNI bindings, aiming to reduce JNI overhead compared to the existing int[] / float[] pathways.

Changes:

  • Add JNI methods to accept direct ByteBuffer image inputs (uint8 and normalized float).
  • Add Java LlmModule.prefillImages(ByteBuffer, ...) and prefillNormalizedImages(ByteBuffer, ...) overloads and register new native methods.
  • Deprecate the legacy prefillImages(int[], ...) overload.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
extension/android/jni/jni_layer_llama.cpp Adds new JNI entrypoints that read direct ByteBuffer data for image prefill.
extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.java Adds new public ByteBuffer overloads and native declarations; deprecates the int[] overload.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings February 28, 2026 08:32
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings February 28, 2026 09:26
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The existing prefillImages(int[]) and prefillImages(float[]) APIs
perform two full data copies across the JNI boundary: one via
getRegion() into a temporary vector, then an element-by-element
conversion into the final vector. This adds new prefillImages(ByteBuffer)
and prefillNormalizedImages(ByteBuffer) overloads that use
GetDirectBytes() on direct ByteBuffers to access the native memory
pointer directly, eliminating the JNI array copy overhead.

Both Java and native layers validate dimensions, buffer size, and
float alignment before accessing the buffer. The float path uses
memcpy instead of reinterpret_cast to avoid undefined behavior.
Both array-based overloads are now marked @deprecated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants