youngseng / speech-driven-expressions Goto Github PK

epoch 226
Some weights of the model checkpoint at /xhzyssd2/molingqiang/repo/Speech-driven-expressions/data/chinese-roberta-wwm-ext were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 322, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 136, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "synthesize.py", line 233, in <module>
    main(args.ckpt_path, args.transcript_path, args.wav_path)
  File "synthesize.py", line 64, in main
    checkpoint_path, device)
  File "/xhzyssd2/molingqiang/repo/Speech-driven-expressions/Tri/scripts/utils/train_utils.py", line 187, in load_checkpoint_and_model
    generator.load_state_dict(checkpoint['gen_dict'])
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1498, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PoseGenerator:
        Unexpected key(s) in state_dict: "gru.weight_ih_l0", "gru.weight_hh_l0", "gru.bias_ih_l0", "gru.bias_hh_l0", "gru.weight_ih_l0_reverse", "gru.weight_hh_l0_reverse", "gru.bias_ih_l0_reverse", "gru.bias_hh_l0_reverse", "gru.weight_ih_l1", "gru.weight_hh_l1", "gru.bias_ih_l1", "gru.bias_hh_l1", "gru.weight_ih_l1_reverse", "gru.weight_hh_l1_reverse", "gru.bias_ih_l1_reverse", "gru.bias_hh_l1_reverse", "gru.weight_ih_l2", "gru.weight_hh_l2", "gru.bias_ih_l2", "gru.bias_hh_l2", "gru.weight_ih_l2_reverse", "gru.weight_hh_l2_reverse", "gru.bias_ih_l2_reverse", "gru.bias_hh_l2_reverse", "out.0.weight", "out.0.bias", "out.2.weight", "out.2.bias", "mutimodal_fusiom.weight", "mutimodal_fusiom.bias", "mutimodal_fusiom_.layers.0.self_attn.in_proj_weight", "mutimodal_fusiom_.layers.0.self_attn.in_proj_bias", "mutimodal_fusiom_.layers.0.self_attn.out_proj.weight", "mutimodal_fusiom_.layers.0.self_attn.out_proj.bias", "mutimodal_fusiom_.layers.0.linear1.weight", "mutimodal_fusiom_.layers.0.linear1.bias", "mutimodal_fusiom_.layers.0.linear2.weight", "mutimodal_fusiom_.layers.0.linear2.bias", "mutimodal_fusiom_.layers.0.norm1.weight", "mutimodal_fusiom_.layers.0.norm1.bias", "mutimodal_fusiom_.layers.0.norm2.weight", "mutimodal_fusiom_.layers.0.norm2.bias", "con1d_pitch.0.weight", "con1d_pitch.0.bias", "con1d_pitch.1.weight", "con1d_pitch.1.bias", "con1d_pitch.1.running_mean", "con1d_pitch.1.running_var", "con1d_pitch.1.num_batches_tracked", "con1d_pitch.3.weight", "con1d_pitch.3.bias", "con1d_pitch.4.weight", "con1d_pitch.4.bias", "con1d_pitch.4.running_mean", "con1d_pitch.4.running_var", "con1d_pitch.4.num_batches_tracked", "con1d_pitch.6.weight", "con1d_pitch.6.bias", "con1d_pitch.7.weight", "con1d_pitch.7.bias", "con1d_pitch.7.running_mean", "con1d_pitch.7.running_var", "con1d_pitch.7.num_batches_tracked", "con1d_erengy.0.weight", "con1d_erengy.0.bias", "con1d_erengy.1.weight", "con1d_erengy.1.bias", "con1d_erengy.1.running_mean", "con1d_erengy.1.running_var", "con1d_erengy.1.num_batches_tracked", "con1d_erengy.3.weight", "con1d_erengy.3.bias", "con1d_erengy.4.weight", "con1d_erengy.4.bias", "con1d_erengy.4.running_mean", "con1d_erengy.4.running_var", "con1d_erengy.4.num_batches_tracked", "con1d_erengy.6.weight", "con1d_erengy.6.bias", "con1d_erengy.7.weight", "con1d_erengy.7.bias", "con1d_erengy.7.running_mean", "con1d_erengy.7.running_var", "con1d_erengy.7.num_batches_tracked", "con1d_volume.0.weight", "con1d_volume.0.bias", "con1d_volume.1.weight", "con1d_volume.1.bias", "con1d_volume.1.running_mean", "con1d_volume.1.running_var", "con1d_volume.1.num_batches_tracked", "con1d_volume.3.weight", "con1d_volume.3.bias", "con1d_volume.4.weight", "con1d_volume.4.bias", "con1d_volume.4.running_mean", "con1d_volume.4.running_var", "con1d_volume.4.num_batches_tracked", "con1d_volume.6.weight", "con1d_volume.6.bias", "con1d_volume.7.weight", "con1d_volume.7.bias", "con1d_volume.7.running_mean", "con1d_volume.7.running_var", "con1d_volume.7.num_batches_tracked", "transformer_encoder_1.layers.0.self_attn.in_proj_weight", "transformer_encoder_1.layers.0.self_attn.in_proj_bias", "transformer_encoder_1.layers.0.self_attn.out_proj.weight", "transformer_encoder_1.layers.0.self_attn.out_proj.bias", "transformer_encoder_1.layers.0.linear1.weight", "transformer_encoder_1.layers.0.linear1.bias", "transformer_encoder_1.layers.0.linear2.weight", "transformer_encoder_1.layers.0.linear2.bias", "transformer_encoder_1.layers.0.norm1.weight", "transformer_encoder_1.layers.0.norm1.bias", "transformer_encoder_1.layers.0.norm2.weight", "transformer_encoder_1.layers.0.norm2.bias", "fusion_1.fusion_layer_1.weight", "fusion_1.fusion_layer_1.bias", "fusion_1.fusion_layer_3.weight", "fusion_1.fusion_layer_3.bias", "fusion_wo_emo.fusion_layer_1.weight", "fusion_wo_emo.fusion_layer_1.bias", "fusion_wo_emo.fusion_layer_3.weight", "fusion_wo_emo.fusion_layer_3.bias". 
        size mismatch for AIWIN.0.layers.0.self_attn.in_proj_weight: copying a param with shape torch.Size([192, 64]) from checkpoint, the shape in current model is torch.Size([198, 66]).
        size mismatch for AIWIN.0.layers.0.self_attn.in_proj_bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([198]).
        size mismatch for AIWIN.0.layers.0.self_attn.out_proj.weight: copying a param with shape torch.Size([64, 64]) from checkpoint, the shape in current model is torch.Size([66, 66]).
        size mismatch for AIWIN.0.layers.0.self_attn.out_proj.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN.0.layers.0.linear1.weight: copying a param with shape torch.Size([2048, 64]) from checkpoint, the shape in current model is torch.Size([2048, 66]).
        size mismatch for AIWIN.0.layers.0.linear2.weight: copying a param with shape torch.Size([64, 2048]) from checkpoint, the shape in current model is torch.Size([66, 2048]).
        size mismatch for AIWIN.0.layers.0.linear2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN.0.layers.0.norm1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN.0.layers.0.norm1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN.0.layers.0.norm2.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN.0.layers.0.norm2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.self_attn.in_proj_weight: copying a param with shape torch.Size([192, 64]) from checkpoint, the shape in current model is torch.Size([198, 66]).
        size mismatch for TransformerDecoder.layers.0.self_attn.in_proj_bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([198]).
        size mismatch for TransformerDecoder.layers.0.self_attn.out_proj.weight: copying a param with shape torch.Size([64, 64]) from checkpoint, the shape in current model is torch.Size([66, 66]).
        size mismatch for TransformerDecoder.layers.0.self_attn.out_proj.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.multihead_attn.in_proj_weight: copying a param with shape torch.Size([192, 64]) from checkpoint, the shape in current model is torch.Size([198, 66]).
        size mismatch for TransformerDecoder.layers.0.multihead_attn.in_proj_bias: copying a param with shape torch.Size([192]) from checkpoint, the shape in current model is torch.Size([198]).
        size mismatch for TransformerDecoder.layers.0.multihead_attn.out_proj.weight: copying a param with shape torch.Size([64, 64]) from checkpoint, the shape in current model is torch.Size([66, 66]).
        size mismatch for TransformerDecoder.layers.0.multihead_attn.out_proj.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.linear1.weight: copying a param with shape torch.Size([2048, 64]) from checkpoint, the shape in current model is torch.Size([2048, 66]).
        size mismatch for TransformerDecoder.layers.0.linear2.weight: copying a param with shape torch.Size([64, 2048]) from checkpoint, the shape in current model is torch.Size([66, 2048]).
        size mismatch for TransformerDecoder.layers.0.linear2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm1.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm2.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm3.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for TransformerDecoder.layers.0.norm3.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([66]).
        size mismatch for AIWIN_.0.weight: copying a param with shape torch.Size([37, 64]) from checkpoint, the shape in current model is torch.Size([37, 66]).