Model service not running on MAC applehv and linux

[lstocchi]

Philippe discovered that the model service cannot be actually run after creation with #100 on MAC applehv.

Error

Traceback (most recent call last):
  File "/locallm/chat_service.py", line 9, in <module>
    llm = Llama(os.getenv('MODEL_PATH',
  File "/opt/app-root/lib64/python3.9/site-packages/llama_cpp/llama.py", line 276, in __init__
    raise ValueError(f"Model path does not exist: {model_path}")
ValueError: Model path does not exist: /llama-2-7b-chat.Q5_K_S.gguf

Also broken on Linux

[lstocchi]

Start investigating, if someone has any info pls share here

[jeffmaury]

I would guess that's because volume mounts is not implemented on applehv

[lstocchi]

But if i execute it from the cli it works

[lstocchi]

Ok so i'm able to replicate the error we face using the CLI if i forget to add the Z flag to the command
Working command

podman container create -v localSource:/model.gguf:Z --env MODEL_PATH=/model.gguf modelimage

Non-working command -> same result as running with Desktop

podman container create -v localSource:/model.gguf --env MODEL_PATH=/model.gguf modelimage

[lstocchi]

closing as #231 has been merged and the model mounting works on applehv/linux