Add manifest extension AoT #14128

JacobSzwejbka · 2025-09-09T20:23:02Z

Summary:
Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later.

A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too.

Differential Revision: D82052721

pytorch-bot · 2025-09-09T20:23:05Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14128

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 4 Cancelled Jobs, 2 Unrelated Failures

As of commit 0b0880b with merge base 66639e4 ():

NEW FAILURES - The following jobs have failed:

Lint / lintrunner / linux-job (gh)
>>> Lint for extension/manifest/_manifest.py:
pull / unittest-editable / macos / macos-job (gh)
backends/xnnpack/test/ops/test_conv1d.py::TestConv1d::test_qs8_conv1d_batchnorm_seq
trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

pull / test-binary-size-linux-gcc / linux-job (gh)
##[error]The operation was canceled.
pull / test-openvino-linux / linux-job (gh)
##[error]The operation was canceled.
pull / test-samsung-models-linux / linux-job (gh)
##[error]The operation was canceled.
pull / test-setup-linux-gcc / linux-job (gh)
##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

trunk / test-llama-runner-mac (fp32, mps) / macos-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-moshi-linux / linux-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-09T20:23:26Z

This pull request was exported from Phabricator. Differential Revision: D82052721

github-actions · 2025-09-09T20:24:07Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

lucylq · 2025-09-09T20:34:59Z

extension/manifest/_manifest.py

+
+ def to_bytes(self) -> bytes:
+     """Returns the binary representation of the Manifest. Written
+     bottom up.


nit, explain why it's written bottom up

"Returns the binary representation of the Manifest. Written bottom up to allow for BC considerations. The compatibility-preserving way to make changes is to increase the header's length field and add new fields at the top. This means we can always check the last n bytes for the magic and size, and then load the full footer."

lucylq · 2025-09-09T20:36:00Z

extension/manifest/_manifest.py

+         +self.padding_size.to_bytes(4, byteorder=_MANIFEST_BYTEORDER)
+         # uint32_t: Size of this manifest. This makes it easier to add new
+         # fields to this header in the future. Always use the proper size
+         # (i.e., ignore self.length) since there's no reason to create an


What is self.length used for?

extension/manifest/_manifest.py

JacobSzwejbka · 2025-09-09T21:30:23Z

Oh the padding isnt needed since the alignment doesnt matter since we reconstruct byte by byte anyway. Ill remove

GregoryComer · 2025-09-09T23:18:29Z

Is there a strong reason to not include this in the core AOT code (as opposed to a dedicated extension?). I don't have a super strong opinion on this, but I do worry about the growing number of fine-grained extensions being confusing to users and compromising UX.

Summary: Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later. A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too. Differential Revision: D82052721

facebook-github-bot · 2025-09-10T20:39:21Z

This pull request was exported from Phabricator. Differential Revision: D82052721

Summary: Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later. A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too. Differential Revision: D82052721

facebook-github-bot · 2025-09-10T21:03:28Z

This pull request was exported from Phabricator. Differential Revision: D82052721

Summary: Pull Request resolved: pytorch#14128 Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later. A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too. Differential Revision: D82052721

JacobSzwejbka · 2025-09-10T21:03:35Z

Is there a strong reason to not include this in the core AOT code (as opposed to a dedicated extension?). I don't have a super strong opinion on this

I could put the aot stuff in core and put the runtime reader in extension? The modularity is a feature for embedded. We shouldnt expose so many options in mobile builds which is why we have the presets.

Summary: Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later. A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too. Differential Revision: D82052721

Summary: Pull Request resolved: pytorch#14128 Add some infra for us to optionally add some key structured data to the end of a pte. This diff is around enabling users to easily tag their model with a cryptographic signature. Has room to expand later. A key design motivation is it would be ideal if this is transparent to the rest of the extensions we have today. Im claiming the prime footer real estate for this which is unused today by anything in tree. This should let it be composable with other formats like bundledProgram too. Differential Revision: D82052721

facebook-github-bot · 2025-09-10T21:10:11Z

This pull request was exported from Phabricator. Differential Revision: D82052721

mergennachin · 2025-09-10T21:38:19Z

extension/manifest/_manifest.py

+
+

Write a docblock about an example usage of manifest file for higher layer consumers. Also mention that manifest is a mechanism, not a security policy. And explicitly say that consumers implements appropriate security for their threat model

# 1. Generate PTE file pte_data = serialize_pte_binary(program) # 2. Create cryptographic signature of PTE data signature = sign_with_private_key(pte_data, private_key) # e.g., RSA, ECDSA # 3. Append manifest with signature manifest = Manifest(signature=signature) pte_with_manifest = append_manifest(pte_data, manifest) Verification Process # 1. Extract manifest from end of file manifest = Manifest.from_bytes(file_data) # 2. Extract PTE data (using program_offset) pte_data = file_data[:-(manifest_length + padding)] # 3. Verify signature with public key is_valid = verify_signature(pte_data, manifest.signature, public_key)

mergennachin · 2025-09-10T21:43:12Z

extension/manifest/_manifest.py

+    # Unique ID for the data the manifest was appended to. Often this might contain
+    # a crytographic signature for the data.
+    signature: bytes
+


Add version

I can imagine people can use this for other use-cases besides security, such as saving arbitrary serializable metadata.

For instance, saving tokenizer.json file location etc.

I can imagine people can use this for other use-cases besides security, such as saving arbitrary serializable metadata.

It wasnt really the intent. I chose this impl here because I wanted a really light weight way to attach security information or other core metadata about the pte.

If we want it to store arbitrary user defined things like a json then I dont really think appending to the .pte is the correct solution, just shove it all in a zip would be my opinion.

mergennachin · 2025-09-10T21:48:25Z

extension/manifest/_manifest.py

+    # Unique ID for the data the manifest was appended to. Often this might contain
+    # a crytographic signature for the data.
+    signature: bytes
+


Should you add timestamp field too?

mergennachin · 2025-09-10T21:49:19Z

extension/manifest/_manifest.py

+
+    EXPECTED_MAGIC: ClassVar[bytes] = b"em00"
+
+    MAX_SIGNATURE_SIZE: ClassVar[int] = 512


why is this fixed?

So we can just do one load at runtime. Instead of 2 loads or a stream.

512 should also cover the vast majority of cryptographic signature algorithms I saw.

mergennachin · 2025-09-10T21:54:09Z

extension/manifest/_manifest.py

+        return data
+
+    @staticmethod
+    def from_bytes(data: bytes) -> "_ManifestLayout":


For large files you have to read the whole thing?

Could add more methods like from file that just load the last MAX_SIZE bytes. Or stream.

I dont really expect people to be verifying the signature in python though. Its mostly just there for testing.

mergennachin · 2025-09-10T21:56:05Z

extension/manifest/_manifest.py

+    # Unique ID for the data the manifest was appended to. Often this might contain
+    # a crytographic signature for the data.
+    signature: bytes
+


Something like this?

@dataclass class Manifest: type: str # "signature", "checksum", "metadata", etc. version: int payload: bytes timestamp: Optional[int] attributes: Dict[str, str]

is version the version of the manifest struct or user specified?

If a user wanted to have multiple things then would you expect them to daisy chain manifests?

What is attributes?

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 9, 2025

facebook-github-bot added the fb-exported label Sep 9, 2025

lucylq reviewed Sep 9, 2025

View reviewed changes

JacobSzwejbka force-pushed the export-D82052721 branch from e82daa2 to 9c31b7a Compare September 10, 2025 20:39

JacobSzwejbka force-pushed the export-D82052721 branch from 9c31b7a to 22ff2b7 Compare September 10, 2025 20:56

JacobSzwejbka requested review from mergennachin and GregoryComer September 10, 2025 21:01

JacobSzwejbka force-pushed the export-D82052721 branch from 22ff2b7 to 6466f05 Compare September 10, 2025 21:03

JacobSzwejbka force-pushed the export-D82052721 branch from 6466f05 to 03be89a Compare September 10, 2025 21:06

JacobSzwejbka force-pushed the export-D82052721 branch from 03be89a to 0b0880b Compare September 10, 2025 21:10

mergennachin reviewed Sep 10, 2025

View reviewed changes


		EXPECTED_MAGIC: ClassVar[bytes] = b"em00"

		MAX_SIGNATURE_SIZE: ClassVar[int] = 512

Add manifest extension AoT #14128

Are you sure you want to change the base?

Add manifest extension AoT #14128

Uh oh!

Conversation

JacobSzwejbka commented Sep 9, 2025

Uh oh!

pytorch-bot bot commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14128

❌ 3 New Failures, 4 Cancelled Jobs, 2 Unrelated Failures

Uh oh!

facebook-github-bot commented Sep 9, 2025

Uh oh!

github-actions bot commented Sep 9, 2025

This PR needs a release notes: label

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JacobSzwejbka commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GregoryComer commented Sep 9, 2025

Uh oh!

facebook-github-bot commented Sep 10, 2025

Uh oh!

facebook-github-bot commented Sep 10, 2025

Uh oh!

JacobSzwejbka commented Sep 10, 2025

Uh oh!

facebook-github-bot commented Sep 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JacobSzwejbka Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 9, 2025 •

edited

Loading

This PR needs a `release notes:` label

JacobSzwejbka commented Sep 9, 2025 •

edited

Loading

JacobSzwejbka Sep 10, 2025 •

edited

Loading