Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make API endpoint URL configurable in OpenAiClient #3923

Open
mysticfall opened this issue Nov 10, 2024 · 0 comments
Open

Make API endpoint URL configurable in OpenAiClient #3923

mysticfall opened this issue Nov 10, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@mysticfall
Copy link

What is the problem this feature would solve?

Several LLM backends, like OpenRouter or Text Generation WebUI, provide an OpenAI-compatible HTTP API.

However, the current implementation of OpenClient in the ai package doesn't allow the user to change the default endpoint URL, which is hardcoded as https://api.openai.com/v1.

It could be a major limitation for many users since OpenAI only provides their own proprietary LLM models, while many different base models and finetunes exist that may provide better features for them.

What is the feature you are proposing to solve the problem?

Make API endpoint URL configurable in OpenAiClient.

What alternatives have you considered?

Add a baseURL or url argument to the OpenAIClient.make function.

@mysticfall mysticfall added the enhancement New feature or request label Nov 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant