vLLM model handler efficiency improvements #27495
Job | Run time |
---|---|
10s | |
16m 53s | |
7m 11s | |
13m 59s | |
9m 47s | |
15m 33s | |
9m 55s | |
13m 59s | |
6m 48s | |
14m 30s | |
2m 24s | |
15m 42s | |
2m 30s | |
14m 24s | |
2m 49s | |
15m 1s | |
2m 57s | |
3m 32s | |
3m 35s | |
3m 26s | |
3m 46s | |
0s | |
0s | |
2h 58m 51s |