Skip to content

Issues: kubernetes-sigs/llm-instance-gateway

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

12 Open 3 Closed
12 Open 3 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add FetchMetricLatency metric
#50 opened Nov 20, 2024 by liu-cong
Validate model/adapter is available on the model server before sending requests to a model server
#49 opened Nov 20, 2024 by liu-cong
Support traffic split between target models
#45 opened Nov 19, 2024 by liu-cong
Validate latency of EPP when rolling out a new deployment
#44 opened Nov 18, 2024 by kfswain
Merge podMetrics map and datastore map
#43 opened Nov 18, 2024 by kfswain
Identify owning SIG
#41 opened Nov 14, 2024 by BenTheElder
17
Update sample yaml with new ext proc image once that's built
#34 opened Oct 28, 2024 by liu-cong
Switch to upstream vLLM
#22 opened Oct 21, 2024 by liu-cong
1
@coolkp
4
vLLM Fork: RuntimeError: CUDA error
#21 opened Oct 20, 2024 by guydc
2
Gateways supporting LLMServerPool as a an HTTPRoute BackendRef
#19 opened Oct 10, 2024 by kfswain
1
Add more knobs to the LLMServerPool
#16 opened Oct 4, 2024 by liu-cong
Dynamically select pod instead of statically passing Pods/PodIPs into ext proc
#12 opened Sep 29, 2024 by Xunzhuo
2
ProTip! What’s not been updated in a month: updated:<2024-10-23.