Cannot copy out of meta tensor; no data! when half process

[Paxwell-Paxwell]

README.md: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 167/167 [00:00<00:00, 2.09MB/s]
Repo card metadata block was not found. Setting CardData to empty.
val.jsonl.zst: 100%|███████████████████████████████████████████████████████████████████████████████████████| 471M/471M [00:01<00:00, 241MB/s]
Generating validation split: 100%|█████████████████████████████████████████████████████████| 214670/214670 [00:07<00:00, 28813.68 examples/s]
AWQ: 0%| | 0/80 [00:00<?, ?it/s]
AWQ: 61%|████████████████████████████████████████████████████████████▋ | 49/80 [46:39<29:31, 57.14s/it]
Traceback (most recent call last):
File "/workspace/convert_awq.py", line 43, in
model.quantize(tokenizer, quant_config=quant_config)
File "/workspace/myenv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/awq/models/base.py", line 238, in quantize
self.quantizer.quantize()
File "/workspace/myenv/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 159, in quantize
input_feat = self._get_input_feat(self.modules[i], named_linears)
File "/workspace/myenv/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 633, in _get_input_feat
self.inps = self._module_forward(self.inps, layer, module_kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 247, in _module_forward
module_output = module(x, **module_kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/hooks.py", line 364, in pre_forward
return send_to_device(args, self.execution_device), send_to_device(
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 175, in send_to_device
return honor_type(
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 82, in honor_type
return type(obj)(generator)
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 176, in
tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor)
File "/workspace/myenv/lib/python3.10/site-packages/accelerate/utils/operations.py", line 156, in send_to_device
return tensor.to(device, non_blocking=non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!

A100 80gb
ram 128

print("Loading model with offloading...")
model_init_kwargs = {"max_memory": {0: "70GIB", "cpu": "120GiB"}}
model = AutoAWQForCausalLM.from_pretrained(model_path ,
use_cache=False,
**model_init_kwargs)

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

quantization_config = AwqConfig(
bits=quant_config["w_bit"],
group_size=quant_config["q_group_size"],
zero_point=quant_config["zero_point"],
version=quant_config["version"].lower(),
)

config = AutoConfig.from_pretrained(model_path)
config.quantization_config = quantization_config

model.quantize(tokenizer, quant_config=quant_config)