Cannot pull a stable-diffusion model from docker hub

[factor3]

I have tried pulling the ai/stable-diffusion:Q4 model from the docker hub:

docker model pull ai/stable-diffusion

and I am getting the following error message:

Failed to pull model: pulling ai/stable-diffusion:Q4 failed with status 415 Unsupported Media Type: safetensors models are not currently supported - this runner only supports GGUF format models

I understand this, of course, because I am using llama.cpp as my backend runner and that runner only supports gguf formatted models.

There is, however, a problem. When I do a search for the stable-diffusion models using the search feature:

 docker model search stable-diffusion

the first line it shows is the following:

NAME                                                                                   DESCRIPTION                                         BACKEND          DOWNLOADS  STARS  SOURCE
ai/stable-diffusion                                                                    Image generation model, uses a base latent diff...  llama.cpp        18.9K      3      Docker Hub

the backend column indicates that ai/stable-diffusion is compatible with llama.cpp and is a gguf formatted model!

Obviously, this is incorrect, as the error line shows.

I have two issues:

1. Why is the docker hub providing a model that is supposedly llama.cpp compatible but isn't?
2. Why isn't there an actual gguf version of ai/stable-diffusion?

[ilopezluna]

Hello @factor3 I have opened #754 to have a more robust backend resolution.

1. Why is the docker hub providing a model that is supposedly llama.cpp compatible but isn't?
2. Why isn't there an actual gguf version of ai/stable-diffusion?

1. Because the our current process to determine which backend can run which model is very fragile, as you can see here it's based on the repository name. Unfortunately ai/stable-diffusion can not run on llama.cpp so the search command output was missleading.
2. Because as far as I know, llama.cpp does not support image generation Adding image generation ggml-org/llama.cpp#4408

So to use the model you have to use diffusers backend:
docker model install-runner --backend diffusers (if you are using DD it will be available soon, please let me know if you are using DD and I ping you as soon as diffusers is ready)