Text2Image pipeline export/import #2716

as-suvorov · 2025-09-09T14:34:12Z

Implements void export_model(const std::filesystem::path& blob_path) and ov::Property<std::filesystem::path> blob_path{"blob_path"} for Text2ImagePipeline.
Ticket: 171706

…exploration

as-suvorov · 2025-09-09T14:52:27Z

@TianmengChen @likholat @Wovchena Please review the sample with export/import API proposal: https://github.com/openvinotoolkit/openvino.genai/pull/2716/files#diff-7c43ee3f358ffafcb7b99c16da468002ff143722294947b8bcb22bd153894a10R67

Wovchena · 2025-09-10T06:42:05Z

LGTM. Why did NPU use a property instead of export_model()?

TianmengChen · 2025-09-10T08:14:41Z

LGTM

as-suvorov · 2025-09-10T11:45:08Z

@dmatveev We want to enable export/import blobs for Image2Image pipeline and potentially for other pipelines as well.
There is existing functionality for some NPU pipelines. NPU implementation uses EXPORT_BLOB property: https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/utils.cpp#L525
My current proposal is to use export_model, it correlates with the core.export_model approach, allows to delay export after pipeline ctor. Sample: c7cb70e#diff-7c43ee3f358ffafcb7b99c16da468002ff143722294947b8bcb22bd153894a10R71
What's your thoughts on EXPORT_BLOB property vs export_model API?

…exploration

JohnLeFeng · 2025-09-15T04:07:12Z

samples/cpp/image_generation/text2image_export_import.cpp

+}
+
+void test_npu_request_size(const std::filesystem::path& models_path) {
+    ov::AnyMap properties{{"NPU_USE_NPUW", "YES"}, {"NPUW_DEVICES", "CPU"}, {"NPUW_ONLINE_PIPELINE", "NONE"}};


Sorry for bothering. Just a question here, should we use NPUW for image generation pipeline? I think NPU should be able to handle image generation pipeline without enable NPUW.

It should work without NPUW.

…exploration

Copilot

Pull Request Overview

This PR adds export/import functionality for Text2Image UNet model components, enabling users to export compiled models to blob files and later import them for faster loading. The implementation supports exporting individual model components (UNet, CLIP text encoders, VAE) and provides Python bindings for the export functionality.

Adds export/import methods to core image generation model classes (UNet2DConditionModel, CLIPTextModel, AutoencoderKL)
Implements blob path property handling to enable model import during construction
Provides Python bindings and documentation for the new export/import API

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/python/py_utils.cpp	Adds support for PosixPath objects in Python property conversion
src/python/py_image_generation_pipelines.cpp	Exposes export_model method for Text2ImagePipeline in Python
src/python/py_image_generation_models.cpp	Adds Python bindings for export_model methods on individual model classes
src/python/openvino_genai/py_openvino_genai.pyi	Type stub definitions for the new export_model methods
src/cpp/src/utils.hpp	Declares utility functions for blob export/import operations
src/cpp/src/utils.cpp	Implements blob export/import utility functions
src/cpp/src/image_generation/text2image_pipeline.cpp	Adds export_model method to Text2ImagePipeline
src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp	Implements export/import logic for SDXL pipeline components
src/cpp/src/image_generation/models/	Adds export/import methods to UNet inference classes
src/cpp/src/image_generation/models/unet2d_condition_model.cpp	Implements export/import for UNet2DConditionModel
src/cpp/src/image_generation/models/clip_text_model.cpp	Implements export/import for CLIPTextModel
src/cpp/src/image_generation/models/autoencoder_kl.cpp	Implements export/import for AutoencoderKL
src/cpp/src/image_generation/diffusion_pipeline.hpp	Adds virtual export_model method to base DiffusionPipeline
src/cpp/include/openvino/genai/	Header updates with export_model method declarations
src/cpp/include/openvino/genai/common_types.hpp	Defines blob_path property for specifying import directory
samples/	Documentation and examples for using the export/import functionality

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/python/py_utils.cpp

src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp

src/cpp/src/image_generation/models/unet_inference_static_bs1.hpp

src/cpp/src/utils.cpp

src/cpp/src/image_generation/models/unet2d_condition_model.cpp

as-suvorov · 2025-09-18T15:23:30Z

Export import doesn't work on NPU, potential fix: openvinotoolkit/openvino#32054

OV PR merged. export/import on NPU device with fallback {"NPUW_DEVICES", "CPU"} works fine.

samples/cpp/image_generation/README.md

Wovchena · 2025-09-23T12:44:14Z

src/python/py_image_generation_pipelines.cpp

Add tests. I think the test should check that a new pipeline can be constructed form the blob. Save the blob to a temp dir instead of cache to reduce share usage

There are no image generation python tests for the moment. I propose to implement it in a separate PR, I'll create a ticket for that.

I know.

Adding a file shouldn't be a big deal. Anna wouldn't be able to implement it easily because she isn't familiar with GenAI's tests because of the reason you named, so adding export test here should be a good start

as-suvorov · 2025-09-23T14:45:55Z

@TianmengChen could you please test this export/import API on NPU device?

TianmengChen · 2025-09-24T06:11:08Z

@TianmengChen could you please test this export/import API on NPU device?

Hi @as-suvorov , can you give me example codes, when I use ov::CacheMode::OPTIMIZE_SPEED it return
Exception from src\inference\src\cpp\core.cpp:112:
Exception from src\inference\src\dev\plugin.cpp:53:
Exception from src\plugins\intel_npu\src\common\src\filtered_config.cpp:37:
[ NOT_FOUND ] Option 'CACHE_MODE' is not supported for current configuration

here are my test codes:

    std::string device = "NPU";
    ov::AnyMap import_blob_properties{{ov::cache_mode.name(),ov::CacheMode::OPTIMIZE_SPEED}};
    ov::genai::Text2ImagePipeline pipeline(models_path, device, import_blob_properties);
    pipeline.export_model(models_path / "blobs");

and can you give me some example codes like heterogeneous_stable_diffusion.cpp example, that text_encoder on CPU
UNet on NPU and vae_decoder on GPU and their blob can be exported according to their device.

as-suvorov · 2025-09-24T08:41:17Z

@TianmengChen could you please test this export/import API on NPU device?

Hi @as-suvorov , can you give me example codes, when I use ov::CacheMode::OPTIMIZE_SPEED it return Exception from src\inference\src\cpp\core.cpp:112: Exception from src\inference\src\dev\plugin.cpp:53: Exception from src\plugins\intel_npu\src\common\src\filtered_config.cpp:37: [ NOT_FOUND ] Option 'CACHE_MODE' is not supported for current configuration

here are my test codes:
    std::string device = "NPU";
    ov::AnyMap import_blob_properties{{ov::cache_mode.name(),ov::CacheMode::OPTIMIZE_SPEED}};
    ov::genai::Text2ImagePipeline pipeline(models_path, device, import_blob_properties);
    pipeline.export_model(models_path / "blobs");
and can you give me some example codes like heterogeneous_stable_diffusion.cpp example, that text_encoder on CPU UNet on NPU and vae_decoder on GPU and their blob can be exported according to their device.

This is NPU plugin issue, we need help from NPU team.
Could you please try to use GPU device:

std::string device = "GPU";
ov::genai::Text2ImagePipeline pipeline(models_path, device, ov::cache_mode(ov::CacheMode::OPTIMIZE_SPEED));
pipeline.export_model(models_path / "blobs");

ov::genai::Text2ImagePipeline imported_pipeline(models_path, device, ov::genai::blob_path(models_path / "blobs"));

Let's also try to remove cache mode for NPU:

std::string device = "NPU";
ov::genai::Text2ImagePipeline pipeline(models_path, device);
pipeline.export_model(models_path / "blobs");

as-suvorov · 2025-09-24T11:55:38Z

@TianmengChen please check heterogeneous_stable_diffusion with export/import sample: https://gist.github.com/as-suvorov/7cd131b6f42a4326cfb75c1bab8dae6d

TianmengChen · 2025-09-25T05:53:08Z

@TianmengChen please check heterogeneous_stable_diffusion with export/import sample: https://gist.github.com/as-suvorov/7cd131b6f42a4326cfb75c1bab8dae6d

Thanks @as-suvorov , I first tried with this code on NPU and remove xml and bin file under unet, vae, text_encoder, and it works.

    std::string device = "NPU";
    // ov::genai::Text2ImagePipeline pipeline(models_path, device);
    // pipeline.export_model(models_path / "blobs");
    ov::genai::Text2ImagePipeline imported_pipeline(models_path, device, ov::genai::blob_path(models_path / "blobs"));

But for heterogeneous_stable_diffusion example, it return this:

text_encoder_device: CPU
unet_device: NPU
vae_decoder_device: GPU
Generating image 0
Check 'encoder_hidden_states_bs == m_native_batch_size' failed at C:\chen\openvino.genai\src\cpp\src\image_generation/models/unet_inference_static_bs1.hpp:80:
UNetInferenceStaticBS1::set_hidden_states: native batch size is 1, but encoder_hidden_states has batch size of 2

But this model can run with original heterogeneous_stable_diffusion example.

FYI @JohnLeFeng

as-suvorov · 2025-09-25T08:53:16Z

Thanks @as-suvorov , I first tried with this code on NPU and remove xml and bin file under unet, vae, text_encoder, and it works.
    std::string device = "NPU";
    // ov::genai::Text2ImagePipeline pipeline(models_path, device);
    // pipeline.export_model(models_path / "blobs");
    ov::genai::Text2ImagePipeline imported_pipeline(models_path, device, ov::genai::blob_path(models_path / "blobs"));
But for heterogeneous_stable_diffusion example, it return this:

text_encoder_device: CPU unet_device: NPU vae_decoder_device: GPU Generating image 0 Check 'encoder_hidden_states_bs == m_native_batch_size' failed at C:\chen\openvino.genai\src\cpp\src\image_generation/models/unet_inference_static_bs1.hpp:80: UNetInferenceStaticBS1::set_hidden_states: native batch size is 1, but encoder_hidden_states has batch size of 2

But this model can run with original heterogeneous_stable_diffusion example.

FYI @JohnLeFeng

There is a limitation of NPU export/import unet model implementation. The batch size is not preserved and model always imported with batch size 1: https://github.com/openvinotoolkit/openvino.genai/pull/2716/files#r2355059791
But expected batch size is calculated with guidance scale and equal to 2 for the default config.
https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp#L176
https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/include/openvino/genai/image_generation/unet2d_condition_model.hpp#L96
The workaround I can suggest is to use guidance_scale=1 to set unet model batch size to 1. I updated gist to use guidance_scale=1: https://gist.github.com/as-suvorov/7cd131b6f42a4326cfb75c1bab8dae6d
@likholat Maybe you have other suggestions?

likholat · 2025-09-25T09:29:39Z

Thanks @as-suvorov , I first tried with this code on NPU and remove xml and bin file under unet, vae, text_encoder, and it works.
    std::string device = "NPU";
    // ov::genai::Text2ImagePipeline pipeline(models_path, device);
    // pipeline.export_model(models_path / "blobs");
    ov::genai::Text2ImagePipeline imported_pipeline(models_path, device, ov::genai::blob_path(models_path / "blobs"));
But for heterogeneous_stable_diffusion example, it return this:
text_encoder_device: CPU unet_device: NPU vae_decoder_device: GPU Generating image 0 Check 'encoder_hidden_states_bs == m_native_batch_size' failed at C:\chen\openvino.genai\src\cpp\src\image_generation/models/unet_inference_static_bs1.hpp:80: UNetInferenceStaticBS1::set_hidden_states: native batch size is 1, but encoder_hidden_states has batch size of 2
But this model can run with original heterogeneous_stable_diffusion example.
FYI @JohnLeFeng
There is a limitation of NPU export/import unet model implementation. The batch size is not preserved and model always imported with batch size 1: https://github.com/openvinotoolkit/openvino.genai/pull/2716/files#r2355059791 But expected batch size is calculated with guidance scale and equal to 2 for the default config. https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/image_generation/stable_diffusion_xl_pipeline.hpp#L176 https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/include/openvino/genai/image_generation/unet2d_condition_model.hpp#L96 The workaround I can suggest is to use guidance_scale=1 to set unet model batch size to 1. I updated gist to use guidance_scale=1: https://gist.github.com/as-suvorov/7cd131b6f42a4326cfb75c1bab8dae6d @likholat Maybe you have other suggestions?

This is the only workaround I see at the moment

as-suvorov added 5 commits August 22, 2025 09:59

Add measure time sample

5ec1c77

Merge remote-tracking branch 'upstream/master' into as/export_import_…

9070cbd

…exploration

Add text2image export import poc

c7cb70e

Revert sample

eb7067d

clean up

6673bed

as-suvorov changed the title ~~Image2Image Unet model export/import~~ Text2Image Unet model export/import Sep 9, 2025

as-suvorov added do_not_merge do_not_review labels Sep 9, 2025

github-actions bot added category: image generation Image generation pipelines category: cmake / build Cmake scripts category: CPP API Changes in GenAI C++ public headers no-match-files category: Image generation samples GenAI Image generation samples labels Sep 9, 2025

add file structure repr

6271714

as-suvorov requested review from Wovchena, likholat and TianmengChen September 9, 2025 14:47

as-suvorov added 2 commits September 11, 2025 13:36

Remove export blob property

09616ed

Merge remote-tracking branch 'upstream/master' into as/export_import_…

a2471ff

…exploration

JohnLeFeng reviewed Sep 15, 2025

View reviewed changes

Remove export import sample

6fab3c2

github-actions bot removed category: cmake / build Cmake scripts category: Image generation samples GenAI Image generation samples labels Sep 15, 2025

as-suvorov added 2 commits September 15, 2025 12:54

Add comments

3557ae3

Merge remote-tracking branch 'upstream/master' into as/export_import_…

b9e4236

…exploration

as-suvorov removed the do_not_review label Sep 15, 2025

as-suvorov added 6 commits September 17, 2025 16:03

Add clip model export/import

ce6445e

Merge remote-tracking branch 'upstream/master' into as/export_import_…

32ab698

…exploration

Add autoencoder export import

a2e3738

Add pybindings

3bfd5e5

Update cpp readme

de1e272

Update readmes

218ac3a

github-actions bot added the category: Image generation samples GenAI Image generation samples label Sep 18, 2025

Update readmes

54d7904

as-suvorov marked this pull request as ready for review September 18, 2025 14:50

Copilot AI review requested due to automatic review settings September 18, 2025 14:50

Copilot AI reviewed Sep 18, 2025

View reviewed changes

as-suvorov changed the title ~~Text2Image Unet model export/import~~ Text2Image pipeline export/import Sep 18, 2025

as-suvorov requested review from smirnov-alexey and PatrikStepan September 18, 2025 14:56

Fix blob_path convertion for binbinds

e6dfb9e

as-suvorov requested a review from dmatveev September 19, 2025 09:23

Merge branch 'master' into as/export_import_exploration

176588b

Wovchena reviewed Sep 22, 2025

View reviewed changes

samples/cpp/image_generation/README.md Show resolved Hide resolved

as-suvorov removed the do_not_merge label Sep 22, 2025

as-suvorov added 2 commits September 22, 2025 16:04

Update docstrings

f993b81

Ref blob_path

483602e

Wovchena reviewed Sep 23, 2025

View reviewed changes

Text2Image pipeline export/import #2716

Are you sure you want to change the base?

Text2Image pipeline export/import #2716

Conversation

as-suvorov commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

as-suvorov commented Sep 9, 2025

Uh oh!

Wovchena commented Sep 10, 2025

Uh oh!

TianmengChen commented Sep 10, 2025

Uh oh!

as-suvorov commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohnLeFeng Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

as-suvorov Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

as-suvorov commented Sep 18, 2025

Uh oh!

Uh oh!

Wovchena Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

as-suvorov Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

Wovchena Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

as-suvorov commented Sep 23, 2025

Uh oh!

TianmengChen commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

as-suvorov commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

as-suvorov commented Sep 24, 2025

Uh oh!

TianmengChen commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

as-suvorov commented Sep 25, 2025

Uh oh!

likholat commented Sep 25, 2025

Uh oh!

Uh oh!

as-suvorov commented Sep 9, 2025 •

edited

Loading

as-suvorov commented Sep 10, 2025 •

edited

Loading

TianmengChen commented Sep 24, 2025 •

edited

Loading

as-suvorov commented Sep 24, 2025 •

edited

Loading

TianmengChen commented Sep 25, 2025 •

edited

Loading