-
Notifications
You must be signed in to change notification settings - Fork 15.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🎞️ add json format to OllamaCommon class #13848
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
cc @jmorganca |
Additionally, I added ChatLlamaCpp now. Not sure if this was the right move to add it to the same PR but its booth similar. |
preference would be to make two separate pr's. also llama cpp has a chat interface that we should probably use for chat model https://github.com/abetlen/llama-cpp-python#chat-completion |
I tried to use create_chat_completion but i realized that its a bit more complex because the Dict Messages Schema changed because of the new multimodal interface and didn't know how to handle this best because LlamaCpp supports multimodal stuff but there is not a lot documentation on this... Reverted and caused a git chaos but a squash should solve this :D |
The Ollama Client now allows a pure json mode which I just added to the OllamaCommon class.
This allows using it like in this example: