From e41da812b370610597c85476feae685f76b74e09 Mon Sep 17 00:00:00 2001 From: Praveen Kumar Date: Wed, 4 Dec 2024 13:21:34 +0530 Subject: [PATCH] Fix outdated containerfile link and add link about vulkan Signed-off-by: Praveen Kumar --- model_servers/llamacpp_python/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/model_servers/llamacpp_python/README.md b/model_servers/llamacpp_python/README.md index 046b7fe4..13ed16c0 100644 --- a/model_servers/llamacpp_python/README.md +++ b/model_servers/llamacpp_python/README.md @@ -57,7 +57,7 @@ Finally, you will also need to add `--device nvidia.com/gpu=all` to your `podman ### Vulkan (experimental) -The [Vulkan image](../llamacpp_python/vulkan/Containerfile) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental". +The [Vulkan](https://docs.vulkan.org/guide/latest/what_is_vulkan.html) image ([amd64](../llamacpp_python/vulkan/amd64/Containerfile)/[arm64](../llamacpp_python/vulkan/arm64/Containerfile)) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental". To build the Vulkan model service variant image: @@ -174,4 +174,4 @@ Running tests ```bash make -f Makefile test -``` \ No newline at end of file +```