simonalexanderson / listendenoiseaction Goto Github PK

Hi!

I was trying to set up the environment for the code, but I could not get mpic package when running conda install -c conda-forge mpi4py mpic . I then checked the conda-forge repo and found out that there is no mpic pakcage under the conda-forge channel.

Instead, there is a mpich package available. Should I install mpich instead of mpic?

In addition, after conda install -c conda-forge mpi4py, conda automatically downloaded and installed python 3.11. Should the python version be 3.11 instead of 3.9?

Hello, authors. If I want to use my own audio file and the offered bvh file to generate motion, how to generate feature file in pkl format? Can you provide the script?

Hi,
According to my understanding, the pkl files contain the characteristic information of the sound, how can I get the pkl files from the wav files.

Hi, could you provide the co-speech gestures synthesis pre-trained model and the instruction for the co-speech gestures synthesis data-processing and training processing?

错误描述是results/generated/dance_LDA/kthmisc_gCA_sFM_cAll_d01_mCA_ch24_8_gCA.mp4: No such file or directory
rm: cannot remove 'results/generated/dance_LDA/kthmisc_gCA_sFM_cAll_d01_mCA_ch24_8_gCA.mp4': No such file or directory请问这是怎么回事

In models/LightningModel.py L85, L86 are they doing the same thing?

Hi, thanks for the great work. May I know why there are two pkl files corresponding to one audio file?
What's the difference between the two pkl files, like kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_00.audio29_30fps.pkl and kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001_01.audio29_30fps.pkl . If they are both extracted from kthstreet_gKR_sFM_cAll_d01_mKR_ch01_chargedcableupyour_001.wav, why are they different?

Thank you for providing your great work!

I found that Hips_Yposition is also selected as a motion feature for training.

However, there are already reference_dXposition, reference_dZposition, and reference_dYrotation here, which indicate Hips_Xposition, Hips_Yposition , and Hips_Zposition (I think) because the hip joint is the root joint.

So what is the Hips_Yposition? Or what are the reference_d?positions?

Could you tell me the difference between the reference_d?positions and root positions (hip)?

Hi, I read your paper and got a good insight from your work. Thank you for sharing your work and code.

I'm not familiar with diffusion models. In your paper, you set diffusion steps 100 for gesture generation and 150 for dance generation.
I thought it is too much compared to other domain's diffusion models work.

I wonder if what kind of problem happens when you reduce the diffusion steps and how you determine the number.

Hi, thanks for sharing this impressive work!
I would like to try the pretrained model, but it seems that the download link https://zenodo.org/record/8156769 is broken. Is it possible to fix it? Thanks.

Thanks for sharing this project.
How to preprocess the raw dataset and re-train the model.
Could you provide some operation steps?

In the inference stage, it seems that the mean and standard deviation used for standardization are extracted from the model instead of calculating the input input_tensor, so the normalized result is not a distribution with a mean of 0 and a standard deviation of 1. , which is inconsistent with the description in the first paragraph of the appendix of the paper. Did I miss something?
/model/BaseModel.py line 37

    # standarize input
    def standardizeInput(self, input_tensor):
        return ((input_tensor - self.input_means.type_as(input_tensor)) / self.input_scales.type_as(input_tensor))

    # standarize output
    def standardizeOutput(self, output_tensor):
        return ((output_tensor - self.output_means.type_as(output_tensor)) / self.output_scales.type_as(output_tensor))

    # Add scale and means to output
    def destandardizeInput(self, input_tensor):
        return (input_tensor * self.input_scales.type_as(input_tensor) + self.input_means.type_as(input_tensor))

    # Add scale and means to output
    def destandardizeOutput(self, predictions):
        return (predictions * self.output_scales.type_as(predictions) + self.output_means.type_as(predictions))

Really appreciate the work! Could you provide the feature extraction script so we can evaluate on custom music? Or inform us the precise package and function to retrieve the 29 dim features. Thanks!

Hello, would you like to ask how the data of ZeroEGGs is converted to smplx representation? I found that the bvh skeleton with all 0 seems to be neither T-Pose nor A-Pose.

Hi,

When I run ./experiments/dance_LDA.sh, the sliced wav forms can be seen in the results folder but the generated motion and rendered videos are missing. I just followed the instructions in the README. Is there something missing that I should install or download?