chore: update run.sh script (#11)

* chore: update run.sh script Signed-off-by: axel7083 <[email protected]> * fix: removing config file related code Signed-off-by: axel7083 <[email protected]> * Apply suggestions from code review Co-authored-by: Florent BENOIT <[email protected]> Signed-off-by: axel7083 <[email protected]> * Update chat/run.sh Signed-off-by: axel7083 <[email protected]> --------- Signed-off-by: axel7083 <[email protected]> Co-authored-by: Florent BENOIT <[email protected]>
containers · Apr 11, 2024 · 91431f5 · 91431f5
1 parent 026c855
commit 91431f5
Show file tree

Hide file tree

Showing 2 changed files with 8 additions and 2 deletions.
diff --git a/chat/requirements.txt b/chat/requirements.txt
@@ -1 +1 @@
-llama-cpp-python[server]
+llama-cpp-python[server]==0.2.57
diff --git a/chat/run.sh b/chat/run.sh
@@ -15,4 +15,10 @@
 # limitations under the License.
 #
 # SPDX-License-Identifier: Apache-2.0
-python -m llama_cpp.server --model ${MODEL_PATH} --host ${HOST:=0.0.0.0} --port ${PORT:=8000} --n_gpu_layers 0
+if [ ${MODEL_PATH} ]; then
+    python -m llama_cpp.server --model ${MODEL_PATH} --host ${HOST:=0.0.0.0} --port ${PORT:=8001} --n_gpu_layers ${GPU_LAYERS:=0} --clip_model_path ${CLIP_MODEL_PATH:=None} --chat_format ${MODEL_CHAT_FORMAT:="llama-2"}
+    exit 0
+fi
+
+echo "This image should not be used outside of Podman Desktop AI Lab extension. Missing required MODEL_PATH environment variable."
+exit 1
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		llama-cpp-python[server]
		llama-cpp-python[server]==0.2.57