[Request Impl] Apply Segment Serialization in Bundled Program

[Gasoonjia]

🚀 The feature, motivation and pitch

BundledProgram is a primary way for users to bundle representative input and (optional) its expected outputs with serialized ExecuTorch program (.pte file). It is a critical way to run representative input on device in a unified way.

However we realize the serialization is pretty slow nowadays. The key reason is we try to reserialize the pte file again into bundled program, which has been serialized and is unnecessary here.

To make the pte data a part of serialized bundled program, while not reserialize it, we can use the new introduced NameData to handle that, which makes inserialized data as separate data blobs append after serialized flabuffer data.

#8187 here's the design doc for NameDataBlob, and

executorch/exir/_serialize/_program.py

Line 381 in 48de264

def serialize_pte_binary(

is the code pointer for pte serialization, in which include the NameData serialization.

[dillondesilva]

This is a good improvement! Could I please take this?

[Gasoonjia]

yeah sure that would be great help!

[Gasoonjia]

hey @dillondesilva how's the issue going on so far?

This one is harder and more complex than other first issues. Please let me know if anything i can support

[dillondesilva]

Thanks @Gasoonjia ! I've just been busy with some mid-semester exams but should have time to start this in about a week. Will let you know how it goes 👍

[Gasoonjia]

Great thx for your hard work!

[dillondesilva]

Hi @Gasoonjia - I had a couple things I wanted to check:

1. To avoid reserializing the .pte file, can we remove adding program bytes to bp_schema.BundledProgram and should we instead replace it with NamedDataStoreOutput (or some other more correct NamedData related type)?

@dataclass
class BundledProgram:
    """ExecuTorch program bunlded with data for verification."""

    # Schema version.
    version: int

    # Test sets and other meta datas to verify the whole program.
    # Each BundledMethodTestSuite contains the test cases for one of the Method's
    # present inside the ExecuTorchProgram of the same BundledProgram. The method_name
    # present inside the BundledMethodTestSuite is what is used to link to the appropriate Method.
    method_test_suites: List[BundledMethodTestSuite]

    # Replace program with NamedDataStoreOutput
    named_data: NamedDataStoreOutput

    # The binary data of a serialized ExecuTorchProgram.
    # program: bytes

2. Within the serialize_to_schema method, is the NamedDataStoreOutput to be accessed from self.executorch_program?

[metascroy]

cc @Gasoonjia gentle ping here