/models/pull

[pelikhan]

I am trying to use the /models/pull api to download a model on the fly based a chat completion request.

The API (in 5.12 rc2) takes 3 mandatory parameters (model,id,name) but it's unclear whether they actually have to be provided.

I have tried a few combinations of them and ended up with bad request conflict.

Also, does this API support streaming progress of the download?

[louis-jan]

Hi @pelikhan
model: The download URL of the gguf model file.
id: a random Unique ID string
name: just a display name

Here is an example payload:

{
  "model":"https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q8_0.gguf",
  "id":"llama3.2-3b-instruct",
  "name":"Llama 3.2 3B Instruct Q8"
}

[pelikhan]

is it possible to resolve the GUFF url from known models? For example, for id = llama3.2-1b-instruct (list in the models page), jan.ai could resolve the name and GUFF location.