Why this error？ ValueError: We need an offload_dir to dispatch this model according to this device_map, the following submodules need to be offloaded: base_model.model.model.layers.3, base_model.model.model.layers.4, base_model.model.model.layers.5, base_model.model.model.layers.6, base_model.model.model.layers.7, base_model.model.model.layers.8, base_model.model.model.layers.9, base_model.model.model.layers.10, base_model.model.model.la

[hzbhh]

when I python my generate.py， this error ocurred！ help， I am a rookie~~

You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in huggingface/transformers#24565
Loading checkpoint shards: 100%|██████████| 33/33 [00:22<00:00, 1.46it/s]
C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\accelerate\utils\modeling.py:1365: UserWarning: Current model requires 33562624 bytes of buffer for offloaded layers, which seems does not fit any GPU's remaining memory. If you are experiencing a OOM later, please consider using offload_buffers=True.
warnings.warn(
Traceback (most recent call last):
File "D:\Desktop\temp\alpaca-lora-main\generate.py", line 223, in
fire.Fire(main_interactive)
File "C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\fire\core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\fire\core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\fire\core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "D:\Desktop\temp\alpaca-lora-main\generate.py", line 210, in main_interactive
initialize_model(load_8bit, "../decapoda-research-llama-7B-hf", './lora-alpaca-zh', "")
File "D:\Desktop\temp\alpaca-lora-main\generate.py", line 50, in initialize_model
model = PeftModel.from_pretrained(
File "C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\peft\peft_model.py", line 177, in from_pretrained
model = dispatch_model(model, device_map=device_map)
File "C:\Users\ROG\AppData\Roaming\Python\Python39\site-packages\accelerate\big_modeling.py", line 376, in dispatch_model
raise ValueError(
ValueError: We need an offload_dir to dispatch this model according to this device_map, the following submodules need to be offloaded: base_model.model.model.layers.3, base_model.model.model.layers.4, base_model.model.model.layers.5, base_model.model.model.layers.6, base_model.model.model.layers.7, base_model.model.model.layers.8, base_model.model.model.layers.9, base_model.model.model.layers.10, base_model.model.model.layers.11, base_model.model.model.layers.12, base_model.model.model.layers.13, base_model.model.model.layers.14, base_model.model.model.layers.15, base_model.model.model.layers.16, base_model.model.model.layers.17, base_model.model.model.layers.18, base_model.model.model.layers.19, base_model.model.model.layers.20, base_model.model.model.layers.21, base_model.model.model.layers.22, base_model.model.model.layers.23, base_model.model.model.layers.24, base_model.model.model.layers.25, base_model.model.model.layers.26, base_model.model.model.layers.27, base_model.model.model.layers.28, base_model.model.model.layers.29, base_model.model.model.layers.30, base_model.model.model.layers.31, base_model.model.model.norm, base_model.model.lm_head.