Add base API URL field for Ollama and OpenAI embedding models #1136
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Jupyter AI currently allows the user to call a model at a URL (location) different from the default one by specifying a selected Base API URL. This can be done for Ollama, OpenAI provider models. However, for these providers, there is no way to change the API URL for embedding models when using the `/learn` command in RAG mode. This PR adds an extra field to make this feasible. Tested as follows for Ollama: [1] Start the Ollama system from port 11435 instead 11434 (the default): `OLLAMA_HOST=127.0.0.1:11435 ollama serve` [2] Set the Base API URL: [3] Check that the new API URL works:
for more information, see https://pre-commit.ci
|
I noticed a bug on I'm pushing a change that alters the |
dlqqq
changed the title
srdas
commented
Dec 9, 2024
There was a problem hiding this comment.
- Tested
/learnand/askwith haiku-3.5 and titan-embed-v1. ✅ (No Base API URL) - Tested
/learnand/askwith ollama-llama3.2 and ollama-mxbai-embed-large. ✅ (No Base API URL) -- all next tests with Ollama (to test changes in ollama.py ). - Tested again with Base API URL = 11434 (the default explicitly). ✅
- Tested again after clearing out the Base API URL, still works ✅
- Restarted Ollama with port=12345. Added this to the Base API URL and it all works as expected. ✅
- Removed the custom Base API URL (blank field) and the /learn and /ask commands now fail, as they should, because Ollama is still running on the custom port. :gr-checkmark:
- Leaving custom fields blank, restarted Ollama to return to default API URL and everything works as expected. ✅
- With OpenAI embeddings, left the Base API URL blank (it works ✅), then added the URL (it works ✅) and then deleted the URL (and it still works ✅, confirms the change in
config_manager.pyis implemented).
Code looks good as well.
dlqqq
approved these changes
Dec 9, 2024
|
Kicking CI since the RTD workflow has stalled. |
|
@meeseeksdev please backport to v3-dev |
meeseeksmachine
pushed a commit
to meeseeksmachine/jupyter-ai
that referenced
this pull request
Dec 9, 2024
…enAI embedding models
dlqqq
pushed a commit
that referenced
this pull request
Dec 10, 2024
…ding models (#1149) Co-authored-by: Sanjiv Das <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
BedrockEmbeddingsProvider#493Jupyter AI currently allows the user to call a model at a URL (location) different from the default one by specifying a selected Base API URL. This can be done for Ollama, OpenAI provider models. However, for these providers, there is no way to change the API URL for embedding models when using the
/learncommand in RAG mode. This PR adds an extra field to make this feasible.Testing instructions
Testing as follows for Ollama:
[1] Start the Ollama system from port 11435 instead 11434 (the default):
OLLAMA_HOST=127.0.0.1:11435 ollama serve[2] Set the Base API URL:
[3] Check that the new API URL works for completions and
/learn: