[OpenVINO backend] supporting inference for Gemma, Mistral and GPT2 with ov backend

[Mohamed-Ashraf273]

Description of the change

As a part of my GSoC25 project to support inference with the openvino backend for Gemma , Mistral and GPT-2,
This is my PR for supporting Gemma , Mistral and GPT-2 pipelines.

import os
os.environ["KERAS_BACKEND"] = "openvino"
import keras_hub

model = keras_hub.models.GPT2CausalLM.from_preset(
    "gpt2_large_en", dtype="float16"
)
model.summary()
output = model.generate("Keras is ", max_length=20)
print("Generated text:", output)

Reference

https://docs.openvino.ai/2025/index.html
https://keras.io/api/
https://keras.io/keras_hub/

Colab Notebook

Checklist

• I have added all the necessary unit tests for my change.
• I have verified that my change does not break existing code and works with all backends (TensorFlow, JAX, and PyTorch).
• My PR is based on the latest changes of the main branch (if unsure, rebase the code).
• I have followed the Keras Hub Model contribution guidelines in making these changes.
• I have followed the Keras Hub API design guidelines in making these changes.
• I have signed the Contributor License Agreement.

[Mohamed-Ashraf273]

Hi @fchollet ,
I'd appreciate any feedback on my PR.
thanks

[divyashreepathihalli]

this should be removed for final merge

[divyashreepathihalli]

Keras does not guarantee the stability of its src layout. A future refactoring in Keras could easily break this code. The goal should be to rely only on public Keras APIs.

[divyashreepathihalli]

maybe move the nested helper functions out of make_generate_function. They can be defined at the module level (e.g., in a keras_hub.src.backend.openvino_utils file if necessary) to make the core make_generate_function logic cleaner and more readable.

[divyashreepathihalli]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for the OpenVINO backend for several models (Gemma, Mistral, GPT-2). The changes are extensive and well-structured, including adding OpenVINO as a dependency, implementing a robust test-skipping mechanism, and adding OpenVINO-specific code paths to handle backend differences. The new openvino_utils.py module is a great way to encapsulate backend-specific logic.

My review focuses on improving robustness and configurability. I've identified a few issues:

• A hardcoded "CPU" device for OpenVINO compilation, which limits hardware acceleration options.
• A minor inefficiency in the test configuration file.
• Potential data issues in the test exclusion lists (trailing whitespace, missing newline).

Overall, this is a great contribution. The changes are thoughtful and address the complexities of adding a new backend.

[gemini-code-assist]

This line has a trailing whitespace. This could cause issues with string comparisons in the test skipping logic, potentially preventing this test from being skipped as intended. Please remove the trailing space.

RandomSamplerTest::test_stateful_call

[gemini-code-assist]

This logic for reading and processing the skipped tests file can be made more concise and memory-efficient by iterating over the file object directly instead of using readlines().

Suggested change

	openvino_skipped_tests = file.readlines()
	# it is necessary to check if stripped line is not empty
	# and exclude such lines
	openvino_skipped_tests = [
	line.strip() for line in openvino_skipped_tests if line.strip()
	]
	openvino_skipped_tests = [
	line.strip() for line in file if line.strip()
	]

[gemini-code-assist]

The OpenVINO compilation device is hardcoded to "CPU". This prevents users from leveraging other OpenVINO-supported hardware accelerators like iGPUs or NPUs. To improve flexibility, consider making the device configurable, for example, by reading from a keras.config setting or a dedicated environment variable.

                    ov_model, keras.config.backend_device()

[gemini-code-assist]

This file is missing a newline character at the end. It's a best practice to end text files with a newline to ensure they are processed correctly by various command-line tools.

keras_hub/src/utils/pipeline_model_test.py

[Mohamed-Ashraf273]

@fchollet
@mattdangerw
@rkazants
@divyashreepathihalli