Thank you for your great work. I'm looking forward to the open source of your code.

so, waht time is it, when will you open the code

Hi Author,
Thanks for sharing very excellent work. Would you share the details of converting Yang-style Tai Chi for training data? Thanks

Hi! What's the difference between the tomato representation and humanml3d of the body-only part? As I understand, from the supplementary, that the difference is there's rotation regularization, but I can't find it in your code.
And when will the pretrained TMR model be released?

Hi! I have done something similar to you for my motion processing. I have also used 000021 as the example motion similar to humanml3d. The raw offset is the general direction of the joint from the parent. However, if you visualize the motion (as shown below) you can see that the relative offset of the first finger joint is at -1 y from the wrist similar to lines 94 - 99 for t2m_raw_body_offsets. While finger joints 2 and 3 are on the x-axis. However you have the raw offsets of all finger joints on the x-axis, denoting the fingers are pointing in the x-axis for 000021. Can you recheck again?

This is what I have:

p - pinky, r - ring, m - middle, i - index, t - thumb
left hand
[0, -1, 0], # lp1
[-1, 0, 0], # lp1
[-1, 0, 0], # lp1
[0, -1, 0], # lr1
[-1, 0, 0], # lr1
[-1, 0, 0], # lr1
[0, -1, 0], # lm1
[-1, 0, 0], # lm1
[-1, 0, 0], # lm1
[0, -1, 0], # li1
[-1, 0, 0], # li1
[-1, 0, 0], # li1
[0, -1, 0], # lt1
[0, -1, 0], # lt1
[0, -1, 0], # lt1
right hand
[0, -1, 0], # rp1
[1, 0, 0], # rp1
[1, 0, 0], # rp1
[0, -1, 0], # rr1
[1, 0, 0], # rr1
[1, 0, 0], # rr1
[0, -1, 0], # rm1
[1, 0, 0], # rm1
[1, 0, 0], # rm1
[0, -1, 0], # ri1
[1, 0, 0], # ri1
[1, 0, 0], # ri1
[0, -1, 0], # rt1
[0, -1, 0], # rt1
[0, -1, 0], # rt1

If my dataset stores frame-by-frame images in each folder, with images containing only upper body movements, and each folder represents an action sequence with a corresponding text description.
How should I use your code for such a dataset?
Also, which part of your code is related to text-motion alignment? I couldn't find it, please point it out. Thank you.
If I have only extracted the 2D skeletal keypoints of human images, how do I use your code?

Hi,
in appendix, I see the motion representation is defined below:

Specifically, the i-th frame pose is defined by a tuple of root angular velocity ( ̇ra ∈ R) along Y-axis, root linear velocities ( ̇rx, ̇rz ∈ R) on XZ-plane, root height ry ∈ R, local joints positions (jp ∈ R3N −1), and velocities (jv ∈ R3N ), where N denotes the number of joints.

Obviously, we can recover 3D global position of root from XZ-plane velocities and root height. We can also recover the Y rotation of root by using Y-rotation velocity . My question is how the X-rotation and Z-rotation of root can be recovered from the representation defined above.

https://swanhub.co/demo/Evan/OpenTMR

Hi, I have some questions about the Pretrained Evaluation Checkpoints of OpenTMA. I would be very grateful if you can provide some help~

• OpenTMA itself should be the motion encoder and text encoder optimized by contrastive learning. Why are these pre-trained weights read in for OpenTMA evaluation? What role do they play in OpenTMA evaluation?
• Additionally, I would like to confirm that the "smpl212" in OpenTMA is based on the 322-dimensional SMPLX of MotionX with the 100-dimensional face_shape and 10-dimensional betas removed?
• Last, the currently provided Pretrained Evaluation Checkpoints of OpenTMA do not seem to include pre-trained weights for the smpl212 format. Will it be provided later, or will the corresponding training code be provided?
  Looking forward to your reply~

How do you control the number of frames generated for each action? Where is the corresponding code? Please point it out. Thank you.

This is great work! I have a quick question. After I have processed the motions as described, I have three folders. joint new_joints new_joint_vecs. When training a generative model here, as described in your paper, you would use the full 623 dimensional vector in new_joint_vecs

If you wanted to extract the rotations in new_joint_vecs to apply to an fbx - how would you do this?

I mean to say, it seems the joint order has changed, there is no Jaw joint, and directly removing the continuous 6d rotations and transforming them into quaternions doesn't yield the desired effect.

I'm curious if you have insights here

Thanks for the awesome work! Do you have a estimated release date for when the code and models(including mld, t2m-gpt) will be released?

Good idea!When will the code be released approximately?I really need your code as a reference.Thank you!

MPJPE of RVQ on HumanML3D from MoMask is 29.5,

but the statistic from your paper is 63.1.

Maybe RVQ is not that bad? Can you provide more details about table 2 of your paper?

Where can I find the code for contrastive learning of action encoders? Where is the corresponding training code?

Hi, I am reading the original paper, but I notice that in Fig2(b) the text-motion alignment is finished during inference since it is marked with dotted line, why is that ?

你好，感谢公开这么优秀的工作，想请教一下论文代码是否计划开源？期待你的回复！

Awesome work!

But I met a problem:

When I use the motionx checkpoints, Line249 and Line251 in 'OpenTMA/tma/models/modeltype/temos.py', there is a "size mismatch for main.0.weight: copying a param with shape torch.Size([512, 309, 4]) from checkpoint, the shape in current model is torch.Size([512, 619, 4]). Copying a param with shape torch.Size([512, 309, 4]) from checkpoint, the shape in current model is torch.Size([512, 619, 4])."

I don't know if it's a problem with my Motion-X data processing or with the checkpoint.

Hi! I am very interested in your work, especially the text-motion alignment pre-trained model. Hope to see your model and codes soon.

Hi, I test the demo code https://github.com/LinghaoChan/OpenTMA-demo.

I am referring the code to know how to generate them more conveniently. Maybe you can provide the corresponding scripts.

How much VRAM do I need to run the pre-trained model after release?Thank you！

Great contribution! When I try to preprocess the Motion-X dataset, I find the motionx_face_motion.zip is not avaliable in their provided google drive and Baidu Disk. Does it necessary to perform face motion augmentation in https://github.com/IDEA-Research/Motion-X/tree/main/mocap-dataset-process? Could you provide the motionx_face_motion.zip?

Thanks for the great work. In the data preparation instruction, it is mentioned to check the 623-dim visualization run plot_feature.py. But it seems this file is missing from the repository. Could please share this code? thanks

I would like to convert the tomato representation to SMPL-X format for visualization, similar to Figures 4 and 8 in your paper and the demo on your project page. Could you provide the necessary scripts for this? Thank you!