Skip to content

Commit

Permalink
Merge pull request #832 from praveenkumar/fix_doc_vulkan
Browse files Browse the repository at this point in the history
Fix outdated containerfile link and add link about vulkan
  • Loading branch information
rhatdan authored Dec 4, 2024
2 parents c300271 + e41da81 commit 6d90ca3
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions model_servers/llamacpp_python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Finally, you will also need to add `--device nvidia.com/gpu=all` to your `podman

### Vulkan (experimental)

The [Vulkan image](../llamacpp_python/vulkan/Containerfile) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental".
The [Vulkan](https://docs.vulkan.org/guide/latest/what_is_vulkan.html) image ([amd64](../llamacpp_python/vulkan/amd64/Containerfile)/[arm64](../llamacpp_python/vulkan/arm64/Containerfile)) is experimental, but can be used for gaining partial GPU access on an M-series Mac, significantly speeding up model response time over a CPU only deployment. This image requires that your podman machine provider is "applehv" and that you use krunkit instead of vfkit. Since these tools are not currently supported by podman desktop this image will remain "experimental".

To build the Vulkan model service variant image:

Expand Down Expand Up @@ -174,4 +174,4 @@ Running tests

```bash
make -f Makefile test
```
```

0 comments on commit 6d90ca3

Please sign in to comment.