Getting error for model when using vicuna model

[djaffer]

2023-04-18 17:03:51 Traceback (most recent call last):
2023-04-18 17:03:51 File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
2023-04-18 17:03:51 return _run_code(code, main_globals, None,
2023-04-18 17:03:51 File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
2023-04-18 17:03:51 exec(code, run_globals)
2023-04-18 17:03:51 File "/app/basaran/main.py", line 38, in
2023-04-18 17:03:51 stream_model = load_model(
2023-04-18 17:03:51 File "/app/basaran/model.py", line 318, in load_model
2023-04-18 17:03:51 tokenizer = AutoTokenizer.from_pretrained(name_or_path, **kwargs)
2023-04-18 17:03:51 File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py", line 657, in from_pretrained
2023-04-18 17:03:51 config = AutoConfig.from_pretrained(
2023-04-18 17:03:51 File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/configuration_auto.py", line 916, in from_pretrained
2023-04-18 17:03:51 config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
2023-04-18 17:03:51 File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 573, in get_config_dict
2023-04-18 17:03:51 config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
2023-04-18 17:03:51 File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 628, in _get_config_dict
2023-04-18 17:03:51 resolved_config_file = cached_file(
2023-04-18 17:03:51 File "/usr/local/lib/python3.8/dist-packages/transformers/utils/hub.py", line 380, in cached_file
2023-04-18 17:03:51 raise EnvironmentError(
2023-04-18 17:03:51 OSError: /models/vicuna does not appear to have a file named config.json. Checkout 'https://huggingface.co//models/vicuna/None' for available files.

Maybe the documentation can improve on running a custom model. It is pretty vague right now.

FROM hyperonym/basaran:0.15.3

# Copy model files
COPY ./model /models/vicuna

# Provide default environment variables
ENV MODEL="/models/vicuna"
ENV MODEL_LOCAL_FILES_ONLY="true"
ENV MODEL_HALF_PRECISION="true"
ENV SERVER_MODEL_NAME="vicuna"

[fardeon]

It looks like the config.json file was not found in /models/vicuna. You can enter the container to check if the file has been copied correctly. This may be due to the different working directory when running docker build.

[djaffer]

it quits before it starts.

[fardeon]

it quits before it starts.

You may temporarily remove ENV MODEL="/models/vicuna" to let it download and start with the default model, meanwhile bash into the container to see if the files were correctly copied.

[fardeon]

Vicuna is now tested and is working properly (#160). This issue seems to be a configuration problem within the model repository.

[peakji]

I guess it is now safe to close this issue as vicuna has been confirmed to work with Basaran (#160, #180).

For choices of working vicuna models, please refer to #160 (comment).