calculate the memory usage of the pte model

[imjking]

Hi, I used the following method to convert the Qwen3 model and obtained a PTE model:

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -m SA8255 --compile_only --decoder_model qwen3-0_6b --system_prompt "你是指令优化专家，负责将用户的口语化、模糊或多意图指令转化为清晰、可独立执行的指令。包括：1.上下文补全；2.多意图拆分与简写。请按此处理用户输入：" --prompt "/no_think开一下哔哩哔哩这个app赶紧把左边车窗关喽" --model_mode hybrid --max_seq_len 512 --prefill_ar_len 64 --temperature 0.8 --artifact ./qwen3_0_6b_sa8255_full_sft_batch_inference/qwen3_0_6b_sa8255_hybrid_512_64_full_sft 2>&1 |tee ./qwen3_0_6b_sa8255_full_sft_batch_inference/executorch_qwen3_0_6b_512_64_full_sft.log

Now I want to determine the theoretical memory usage of the PTE model. Is there a way to get the memory usage during or after the conversion process?

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

[lucylq]

Hi @imjking,

You can deserialize the PTE to figure out the activation memory usage.

# Get activation memory values
# Python conda env with executorch installed (pip install executorch works)

>> python environment

from executorch.exir._serialize import _deserialize_pte_binary
f = open("model.pte", "rb")
model = _deserialize_pte_binary(f.read())

# Each execution_plan corresponds to a separate method.
len(model.program.execution_plan)
# Check method name.
model.program.execution_plan[0].name
# List of buffer sizes that need to be allocated at runtime
# [0, activation_memory_size, shared_memory_size]
model.program.execution_plan[0].non_const_buffer_sizes

Besides that, the weights will take up the majority of the memory usage. For a more accurate e2e, you can try running the model and checking the RSS values.

[imjking]

@lucylq OK, I will try this. Thanks.
Additionally, I have already run the PTE model on the car device and got the actual memory usage. Now I want to check the theoretical memory of the model for comparison.

[imjking]

@lucylq Hi, When I executed the deserialization code you provided above, I got the following output result:
`
len(model.program.execution_plan)

18
model.program.execution_plan[0].name

kv_forward
model.program.execution_plan[0].non_const_buffer_sizes

[0, 0]
`
The non_const_buffer_sizes of the remaining 17 names are also [0, 0] or [0] . Is this correct?

[cccclai]

Since you run the model on Qualcomm npu, the memory profiling will be a bit different. @haowhsu-quic @shewu-quic @winskuo-quic @DannyYuyang-quic do we have documentation about memory profiling somewhere?