[Feature request]: Allow LiteLLM to track cost when accessing models from LiteLLM Proxy

[xingyaoww]

What problem or use case are you trying to solve?

Currently, litellm cannot track cost when invoking a model from a LiteLLM proxy.

Describe the UX of the solution you'd like

Do you have thoughts on the technical implementation?

LiteLLM .completion call's response will have an x-litellm-response-cost in the response header when calling models from an LLM proxy.

We should modify openhands/llm/llm.py so that we first check if the response header is present, if so, we skip any cost calculation and use that directly as completion cost.

[github-actions]

OpenHands started fixing the issue! You can monitor the progress here.

[github-actions]

An attempt was made to automatically fix this issue, but it was unsuccessful. A branch named 'openhands-fix-issue-4196' has been created with the attempted changes. You can view the branch here. Manual intervention may be required.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[xingyaoww]

This should already been addressed in self.model_info in the LLM class.