Skip to content

Commit

Permalink
opendatahub/vllm: include adapter smoke test for main (openshift#52662)
Browse files Browse the repository at this point in the history
* opendatahub-io/vllm: use Dockerfile.ubi for main branch

* opendatahub-io/vllm: add grpc test to main branch smoke test
  • Loading branch information
dtrifiro authored and Vivek Shakya committed Jun 12, 2024
1 parent 038bd13 commit ee66498
Showing 1 changed file with 32 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -56,13 +56,15 @@ tests:
# we will need to download test models off HF hub
unset HF_HUB_OFFLINE
# spin up the server and run it in the background, allowing for images download
# spin up the OpenAPI server in the background
python -m vllm.entrypoints.openai.api_server &
server_pid=$!
# wait for the server to be up
sleep 60
# OpenAI API tests
curl -v --no-progress-meter --fail-with-body \
localhost:8000/v1/models | python -m json.tool || \
(kill -9 $server_pid && exit 1)
Expand All @@ -76,7 +78,35 @@ tests:
localhost:8000/v1/completions | python -m json.tool || \
(kill -9 $server_pid && exit 1)
echo "success"
echo "OpenAI API success"
kill -9 $server_pid
# spin up the grpc server in the background
python -m vllm_tgis_adapter &
server_pid=$!
# wait for the server to be up
sleep 60
# get grpcurl
curl --no-progress-meter --location --output grpcurl.tar.gz \
https://github.com/fullstorydev/grpcurl/releases/download/v1.9.1/grpcurl_1.9.1_linux_x86_64.tar.gz
tar -xf grpcurl.tar.gz
# get grpc proto
curl --no-progress-meter --location --remote-name \
https://github.com/opendatahub-io/text-generation-inference/raw/main/proto/generation.proto
# GRPC API test
./grpcurl -v \
-plaintext \
-proto generation.proto \
-d '{ "requests": [{"text": "A red fedora symbolizes "}]}' \
localhost:8033 \
fmaas.GenerationService/Generate || (kill -9 $server_pid && exit 1)
echo "GRPC API success"
kill -9 $server_pid
container:
clone: false
Expand Down

0 comments on commit ee66498

Please sign in to comment.