Skip to content

Commit

Permalink
Revert "Update vllm colab to work with asyncio requirements (#33122)" (
Browse files Browse the repository at this point in the history
…#33170)

This reverts commit ca92726.
  • Loading branch information
damccorm authored Nov 19, 2024
1 parent ca92726 commit baa1591
Showing 1 changed file with 1 addition and 26 deletions.
27 changes: 1 addition & 26 deletions examples/notebooks/beam-ml/run_inference_vllm.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -117,35 +117,10 @@
"source": [
"!pip install openai>=1.52.2\n",
"!pip install vllm>=0.6.3\n",
"!pip install apache-beam[gcp]==2.61.0\n",
"!pip install nest_asyncio # only needed in colab\n",
"!pip install apache-beam[gcp]==2.60.0\n",
"!pip check"
]
},
{
"cell_type": "markdown",
"source": [
"## Colab only: allow nested asyncio\n",
"\n",
"The vLLM model handler logic below uses asyncio to feed vLLM records. This only works if we are not already in an asyncio event loop. Most of the time, this is fine, but colab already operates in an event loop. To work around this, we can use nest_asyncio to make things work smoothly in colab. Do not include this step outside of colab."
],
"metadata": {
"id": "3xz8zuA7vcS3"
}
},
{
"cell_type": "code",
"source": [
"# This should not be necessary outside of colab.\n",
"import nest_asyncio\n",
"nest_asyncio.apply()\n"
],
"metadata": {
"id": "sUqjOzw3wpI3"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
Expand Down

0 comments on commit baa1591

Please sign in to comment.