-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: adding InferenceType enum #1186
feat: adding InferenceType enum #1186
Conversation
936dc6b
to
e389061
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM
2 things that we should improve (even on separate PRs) are
- we should add a label displayign the backend in the models table
- i don't like to see the error " no enabled provider could be found.." when clicking on create service using a non-supported model. I would prefer to not seeing the "create new inference server" button at all. Otherwise i may think that i'm doing something wrong and there is some way to make it work
Yeah I totally agree, this PR is the first step #1111, we can add more items on their to improve how we deal with this relation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Signed-off-by: axel7083 <[email protected]>
Signed-off-by: axel7083 <[email protected]>
Signed-off-by: axel7083 <[email protected]>
Signed-off-by: axel7083 <[email protected]>
Signed-off-by: axel7083 <[email protected]>
d9e2fb4
to
3caacd7
Compare
What does this PR do?
Adding the
InferenceType
enum, allowingInferenceProvider
(see #1161) to precise their type, in this context, it could bellama-cpp
,whisper-cpp
etc.Notable change
InferenceType
backend
string property toModelInfo
interfacebackend
string property toRecipe
interfacegetByType
method to theInferenceProviderRegistry
backend
property of models to select theInferenceProvider
InferenceProvider
constructorScreenshot / video of UI
N/A no visual change
What issues does this PR fix or reference?
Fixes #1181
Part of #1111
How to test this PR?
Side effect
Now, trying to start an InferenceServer with the WhisperModel will raise an error as we do not have any InferenceProvider for it. Same for
facebook/detr-resnet-101
model.ℹ️ This is good, this is what we expect !