Skip to content

Commit

Permalink
Fix outdated containerfile link and add link about vulkan
Browse files Browse the repository at this point in the history
Signed-off-by: Praveen Kumar <[email protected]>
  • Loading branch information
praveenkumar committed Dec 4, 2024
1 parent c300271 commit e41da81
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions model_servers/llamacpp_python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Finally, you will also need to add `--device nvidia.com/gpu=all` to your `podman

### Vulkan (experimental)

The [Vulkan image](../llamacpp_python/vulkan/Containerfile) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental".
The [Vulkan](https://docs.vulkan.org/guide/latest/what_is_vulkan.html) image ([amd64](../llamacpp_python/vulkan/amd64/Containerfile)/[arm64](../llamacpp_python/vulkan/arm64/Containerfile)) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental".

To build the Vulkan model service variant image:

Expand Down Expand Up @@ -174,4 +174,4 @@ Running tests

```bash
make -f Makefile test
```
```

0 comments on commit e41da81

Please sign in to comment.