Docker Model Runner: gemma4 model fails to load with "unknown model architecture: 'gemma4'"

[evittorini-harmonicinc]

Environment: - Docker Desktop: 4.67.0 - OS: Windows - Model: gemma4 (GGUF V3, Q4_K - Medium, 4.74 GiB) - Backend: llama.cpp (ec2b787)

Steps to reproduce:

1. Pull gemma4:latest from Docker Hub via Docker Desktop
2. Run: docker model run gemma4 "hello" Expected: Model loads and responds

Actual: Error - "unknown model architecture: 'gemma4'" Error logs: llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4'
llama_model_load_from_file_impl: failed to load model

[eavelardev]

for vllm it needs to be updated to latest v0.19.0

[doringeman]

Soon a new llama.cpp version with Gemma4 will be pushed to https://hub.docker.com/r/docker/docker-model-backend-llamacpp/tags. Once it's pushed, Docker Model Runner will automatically pull it on restart. 🙌

[ilopezluna]

@evittorini-harmonicinc its already pushed so if you restart Docker Desktop it will work

@eavelardev for vllm is not working yet, we have to also bump Transformers lib. I'll keep you updated

[TomLBZ]

Same issue, here are my terminal outputs:

$ docker model version
Client:
 Version:    v1.1.29
 OS/Arch:    linux/amd64

Server:
 Version:    v1.1.28
 Engine:     Docker Engine

$ docker model status
Docker Model Runner is running

BACKEND    STATUS         DETAILS                          
llama.cpp  Running        llama.cpp 3191462                
sglang     Error          Python not found                 
diffusers  Not Installed                                   
mlx        Not Installed  only supported on Apple Silicon  
vllm       Not Installed  binary not found        
         
$ docker model run ai/gemma4
> background model preload failed: preload failed: status=500 body=unable to load runner: error waiting for runner to be ready: llama.cpp terminated unexpectedly: llama.cpp failed: failed to load model

Verbose output:
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4'
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model '/models/bundles/sha256/cc379c5c3638cf5e862821110b27e32d11bf92865275c2a90b203b0d509a0671/model/model.gguf'
srv    load_model: failed to load model, '/models/bundles/sha256/cc379c5c3638cf5e862821110b27e32d11bf92865275c2a90b203b0d509a0671/model/model.gguf'
srv    operator(): operator(): cleaning up before exit...
main: exiting due to model loading error
uantize.imatrix.chunks_count u32              = 141
llama_model_loader: - type  f32:  423 tensors
llama_model_loader: - type q4_K:  193 tensors
llama_model_loader: - type q5_K:   27 tensors
llama_model_loader: - type q6_K:   61 tensors
llama_model_loader: - type iq4_xs:   15 tensors
llama_model_loader: - type bf16:    1 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 4.74 GiB (5.41 BPW)