TEMP: no shared tokenizer from PR-3512

Gotta get conflicts with the main line figured resolved first. Just don't use the shared tokenizer for now. Signed-off-by: Travis Johnson <[email protected]>
opendatahub-io · May 23, 2024 · 4458631 · 4458631
1 parent d976df3
commit 4458631
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/vllm/entrypoints/grpc/grpc_server.py b/vllm/entrypoints/grpc/grpc_server.py
@@ -118,7 +118,8 @@ def __init__(self, engine: AsyncLLMEngine, args: argparse.Namespace):
 
     async def _post_init(self):
         self.config = await self.engine.get_model_config()
-        self.tokenizer_group = await self.engine.get_tokenizer_group()
+         # self.tokenizer_group = await self.engine.get_tokenizer_group()
+        self.tokenizer_group = self.engine.engine.tokenizer
         self.tokenizer = await self.engine.get_tokenizer()
 
         # Swap in the special TGIS stats logger