From ad7480c384343c428ff901c63d5008a0c3982f6f Mon Sep 17 00:00:00 2001 From: hydai Date: Sat, 4 Nov 2023 13:58:10 +0800 Subject: [PATCH] [Example] Update the installation guide of ggml interactive example (#56) Signed-off-by: hydai --- wasmedge-ggml-llama-interactive/README.md | 29 ++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/wasmedge-ggml-llama-interactive/README.md b/wasmedge-ggml-llama-interactive/README.md index 59785b8..0b3f48e 100644 --- a/wasmedge-ggml-llama-interactive/README.md +++ b/wasmedge-ggml-llama-interactive/README.md @@ -17,9 +17,32 @@ source $HOME/.zshenv ### For Ubuntu (>= 20.04) -Because we enabled OpenBLAS on Ubuntu, you must install `libopenblas-dev` by `apt update && apt install -y libopenblas-dev`. +#### CUDA enabled -Install WasmEdge 0.13.4+WASI-NN ggml plugin(OpenBLAS enabled) via installer +The installer from WasmEdge 0.13.5 will detect cuda automatically. + +If CUDA is detected, the installer will always attempt to install a CUDA-enabled version of the plugin. + +Install WasmEdge 0.13.4+WASI-NN ggml plugin via installer + +```bash +curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- --plugin wasi_nn-ggml +# After install the wasmedge, you have to activate the environment. +# Assuming you use bash (the default shell on Ubuntu), you will need to run the following command +source $HOME/.bashrc +``` + +This version is verified on the following platforms: +1. Nvidia Jetson AGX Orin 64GB developer kit +2. Intel i7-10700 + Nvidia GTX 1080 8G GPU + +#### CPU only + +If CPU is the only available hardware on your machine, the installer will install OpenBLAS version of plugin instead. + +You may need to install `libopenblas-dev` by `apt update && apt install -y libopenblas-dev`. + +Install WasmEdge 0.13.4+WASI-NN ggml plugin via installer ```bash apt update && apt install -y libopenblas-dev # You may need sudo if the user is not root. @@ -36,7 +59,7 @@ Install WasmEdge 0.13.4+WASI-NN ggml plugin via installer ```bash curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- --plugin wasi_nn-ggml # After install the wasmedge, you have to activate the environment. -# Assuming you use bash (the default shell on Ubuntu), you will need to run the following command +# Assuming you use bash (the default shell on Linux), you will need to run the following command source $HOME/.bashrc ```