Skip to content

Commit

Permalink
format: changes from auto-formatter
Browse files Browse the repository at this point in the history
Signed-off-by: Travis Johnson <[email protected]>
  • Loading branch information
tjohnson31415 committed May 8, 2024
1 parent 742d6d4 commit e2fd7ab
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions vllm/tgis_utils/metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,13 @@ def log(self, stats: Stats) -> None:
self.tgi_batch_current_size.set(stats.num_running_sys)

for ttft in stats.time_to_first_tokens_iter:
self.tgi_batch_inference_duration.labels(
{"method": "prefill"}
).observe(ttft)
self.tgi_batch_inference_duration.labels({
"method": "prefill"
}).observe(ttft)
for tpot in stats.time_per_output_tokens_iter:
self.tgi_batch_inference_duration.labels(
{"method": "next_token"}
).observe(tpot)
self.tgi_batch_inference_duration.labels({
"method": "next_token"
}).observe(tpot)

for input_len in stats.num_prompt_tokens_requests:
self.tgi_request_input_length.observe(input_len)
Expand Down

0 comments on commit e2fd7ab

Please sign in to comment.