Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

about infer error? #17

Open
zhanghongyong123456 opened this issue Aug 21, 2024 · 0 comments
Open

about infer error? #17

zhanghongyong123456 opened this issue Aug 21, 2024 · 0 comments

Comments

@zhanghongyong123456
Copy link

zhanghongyong123456 commented Aug 21, 2024

when run 👍python demo_biwi.py
--cfg configs/diffusion/biwi/diffspeaker_hubert_biwi.yaml
--cfg_assets configs/assets/biwi.yaml
--template datasets/biwi/templates.pkl
--example demo/wavs/speech_obama.wav
--ply datasets/biwi/templates/BIWI.ply
--checkpoint checkpoints/biwi/diffspeaker_hubert_biwi.ckpt
--id F3
You are using a model of type hubert to instantiate a model of type wav2vec2. This is not supported for all configurations of models and can yield errors.
Traceback (most recent call last):
File "D:/2023project/project/0win_os/05speech_Pro/DiffSpeaker/demo_biwi.py", line 100, in
main()
File "D:/2023project/project/0win_os/05speech_Pro/DiffSpeaker/demo_biwi.py", line 32, in main
model = get_model(cfg, dataset)
File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\get_model.py", line 5, in get_model
return get_module(cfg, datamodule)
File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\get_model.py", line 16, in get_module
return Model(cfg=cfg, datamodule=datamodule)
File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\modeltype\diffusion_bias.py", line 74, in init
self.scheduler = instantiate_from_config(cfg.model.scheduler)
File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\config.py", line 31, in instantiate_from_config
return get_obj_from_str(config["target"])(**config.get("params", dict()))
File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\config.py", line 21, in get_obj_from_str
return getattr(importlib.import_module(module, package=None), cls)
File "H:\Anaconda3\envs\multitalk\lib\importlib_init_.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in load_unlocked
File "", line 843, in exec_module
File "", line 219, in call_with_frames_removed
File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers_init
.py", line 3, in
from .configuration_utils import ConfigMixin
File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\configuration_utils.py", line 34, in
from .utils import DIFFUSERS_CACHE, HUGGINGFACE_CO_RESOLVE_ENDPOINT, DummyObject, deprecate, logging
File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\utils_init
.py", line 22, in
from .import_utils import (
File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\utils\import_utils.py", line 125, in
_transformers_available = importlib.util.find_spec("transformers") is not None
File "H:\Anaconda3\envs\multitalk\lib\importlib\util.py", line 114, in find_spec
raise ValueError('{}.spec is None'.format(name))
ValueError: transformers.spec is None

about infer vocaset
python demo_vocaset.py
--cfg configs/diffusion/vocaset/diffspeaker_wav2vec2_vocaset.yaml
--cfg_assets configs/assets/vocaset.yaml
--template datasets/vocaset/templates.pkl
--example demo/wavs/speech_long.wav
--ply datasets/vocaset/templates/FLAME_sample.ply
--checkpoint checkpoints/vocaset/diffspeaker_wav2vec2_vocaset.ckpt
--id FaceTalk_170809_00138_TA

Some weights of the model checkpoint at J:\SD_huggingface\wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.bias', 'lm_head.weight']

  • This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  • This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
    Some weights of Wav2Vec2Model were not initialized from the model checkpoint at J:\SD_huggingface\wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    Traceback (most recent call last):
    File "D:/2023project/project/0win_os/05speech_Pro/DiffSpeaker/demo_vocaset.py", line 174, in
    main()
    File "D:/2023project/project/0win_os/05speech_Pro/DiffSpeaker/demo_vocaset.py", line 56, in main
    model = get_model(cfg, dataset)
    File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\get_model.py", line 5, in get_model
    return get_module(cfg, datamodule)
    File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\get_model.py", line 16, in get_module
    return Model(cfg=cfg, datamodule=datamodule)
    File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\models\modeltype\diffusion_bias.py", line 74, in init
    self.scheduler = instantiate_from_config(cfg.model.scheduler)
    File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\config.py", line 31, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
    File "D:\2023project\project\0win_os\05speech_Pro\DiffSpeaker\alm\config.py", line 21, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
    File "H:\Anaconda3\envs\multitalk\lib\importlib_init_.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
    File "", line 1014, in _gcd_import
    File "", line 991, in _find_and_load
    File "", line 975, in _find_and_load_unlocked
    File "", line 671, in load_unlocked
    File "", line 843, in exec_module
    File "", line 219, in call_with_frames_removed
    File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers_init
    .py", line 3, in
    from .configuration_utils import ConfigMixin
    File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\configuration_utils.py", line 34, in
    from .utils import DIFFUSERS_CACHE, HUGGINGFACE_CO_RESOLVE_ENDPOINT, DummyObject, deprecate, logging
    File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\utils_init
    .py", line 22, in
    from .import_utils import (
    File "H:\Anaconda3\envs\multitalk\lib\site-packages\diffusers\utils\import_utils.py", line 125, in
    _transformers_available = importlib.util.find_spec("transformers") is not None
    File "H:\Anaconda3\envs\multitalk\lib\importlib\util.py", line 114, in find_spec
    raise ValueError('{}.spec is None'.format(name))
    ValueError: transformers.spec is None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant