akanazawa / motion_reconstruction Goto Github PK

Hello author, thanks for your work! I have a little question to ask you:
What is the json file here and how can I get it?
https://github.com/akanazawa/motion_reconstruction/blob/6ff7fc867e9e185d547ada11713a60fbf3caa902/run_openpose.py#LL116C4-L116C4

where are the ketpoints files(.json file)?

Thanks a ton for the great work!

I am trying to apply the motion reconstruction to the output of HMMR as a smoothing stage, as well as converting the repo to python3. Though I have a few questions regarding the refinement pipeline. 1. It is mentioned in the paper that the optimization is done in the latent space of HMR's. Does this corresponds to the f_movie output in HMMR, and one can directly optimize that part of the output?
2. I am wondering how is the optimization stepped, as in is there a regressor trained here to revise the previous output? Just reading the paper which describes the losses confuses me how this is done (or maybe it is due to my lack of knowledge in this field :(

Appreciate it!

Hi,

first of all, thanks for releasing the code.

Do you have any suggestion about how to integrate HMMR instead of HMR with this code?
Something like a todo list or steps to go through

There is a example output data in your Readme, link.

What's the unit for the root position? Is it meter?

Thanks for your code~ I'm confused about your body model in 2D pose estimation, do you use coco as your model?

I have read your code, but I have no idea how did you smooth your result. I just notice you used HMR as your output, Thx~

I can not find ''openpose/build/examples/openpose/openpose.bin'' in openpose repo, could you tell me where this file or folder is?

I havn't run these two line code because I havn't configure the environment.

python -m run_openpose

python -m refine_video

But I guess the first one is to get the json file storing all the keypoints and the second one is to get the bvh and h5 file storing the animation..

The question is, how to train the policy? Is the code for training not uploaded yet?
I'm a newbee about Deep Learning and I will be very appreciated if you have time to answer me.

When I try to run refine_video.py on the demo videos and .h5 files, I get errors like:

!!!./demo_data/openpose_output/run_bboxes.h5 doesnt exist!!!

What exactly am I supposed to do with the .h5 and the .bvh files in the demo data for the demo to work? As there is no *_bboxes.h5 file given in the demo data.

Could you tell me how do you do the rotation augmentation and share the code? Thanks so much!

Hello,

I wanted to ask if anybody has implemented the write2bvh function or has another way of importing the results of the project into Unity as an animation. I have successfully imported the pre-existing .bvh files provided in the data folder and I wanted to make my own bvh files by running the project. How can I achieve that?

Thank you!

Hi, I noticed that '# from jason.bvh_core import write2bvh' this line has been commented out, so how to write to bvh?

in refiner.py, you need a pretrained_model which name should be Feb12_2100_save75_model.ckpt-667589,
but in hmr the model isn't this so I wonder how can I get it ?

I've build both models( SMPL and DeepMimic) in Unity, and restore them in T-pose.
But the T-pose is very different, especially in arms. Besides, their joint count is different( 24 vs 14),
So in SFV paper, how to transfer the data from SMPL into DeepMimic?

Any idea would be very helpful!

How to get root position?
Not rotation or anything, but root position, I saw in your bvhs, you have root position, but in your code, the model didn't predict that thing, so how did you get it?

may i ask How to use motion data calculated or collected in is [motion_reconstruction] in the environment [deepmimic]？