@taoranyi Hello,

I am actually facing the issue during training - Producing Empty Images. It's a new dataset . Do you potentially know the cause for this? It seems a value mismatch between the camera rays on the image, BUT how can I debug this step-wise?

Thank you in advanced !!

[Dataset Path] dataset/genebody/barry
 -- Total Frames: 16
************************************
genebody_barry_train
core.data.human_nerf.train
[Dataset Path] dataset/genebody/barry
 -- Total Frames: 600
Epoch: 1 [Iter 20, 19/600 (3%), 10.47 sec] Loss: 0.8744 [lpips: 0.8236 mse: 0.0508 ]
Epoch: 1 [Iter 40, 39/600 (6%), 8.81 sec] Loss: 0.2724 [lpips: 0.2506 mse: 0.0218 ]
Epoch: 1 [Iter 60, 59/600 (10%), 8.90 sec] Loss: 0.4936 [lpips: 0.4518 mse: 0.0418 ]
Epoch: 1 [Iter 80, 79/600 (13%), 8.84 sec] Loss: 0.5618 [lpips: 0.5370 mse: 0.0248 ]
Epoch: 1 [Iter 100, 99/600 (16%), 8.84 sec] Loss: 0.2704 [lpips: 0.2668 mse: 0.0036 ]
Evaluate Progress Images ...
  0%|                                                                                                                                                                                          | 0/16 [00:50<?, ?it/s]
Produce empty images; reload the init model.
Load checkpoint from experiments/human_nerf/genebody/barry/single_gpu/init.tar ...
Epoch: 1 [Iter 20, 119/600 (20%), 59.33 sec] Loss: 0.3576 [lpips: 0.3395 mse: 0.0180 ]

Hello @taoranyi,

I appreciate your work here. I also planning to train humannerf on h36m dataset, atleast for 1 subject.
Could you please tell me, how can we get the h36m smpl parameters and the necessary mask

I tried to use the Easymocap file, since h36m provides the 3d groundtruth https://github.com/zju3dv/EasyMocap/blob/master/apps/demo/smpl_from_keypoints.py (Is this the correct way?)

So I got the smpl parameters like this as an example: I used the h36m Positions_3D_mono files (are these correct files)

  {  "id": 1611,
    "Rh": [
      [-1.109, 4.132, -2.318]
    ],
    "Th": [
      [-246.794, 285.948, 657.016]
    ],
    "poses": [
      [0.000, 0.000, 0.000, 0.917, -1.202, 1.189, 0.836, 1.005, 0.382, 2.144, -0.020, -0.395, -0.627, -1.443, 0.152, ...]
    ],
    "shapes": [
      [11905.026, -2553.326, 698.123, -111.642, 616.076, 136.446, -253.905, -107.915, 117.896, 356.510]
    ]
}

then I generated the mask using SAM -

I used the prepare_dataset.py for the humannerf wild/monocular since I am using a single camera from h36m.
after all that, I got the data loader error, when I placed num_workers to 0,

The near, far list are empty which are generated from the "joints". The bbox values are vast:
min x: -247
max x: +256

So, I need little help on what I could be doing wrong here, is it the masking issue or smpl issue or something? Really need your help, since you already trained on h36m.

Thank you once again. I hope you can help me out

Hi, The repository doesn't have code for pre-training, only fine tuning code is released. Would you have some pointers on how to implement the pre-training pipeline with a custom dataset?

Is there a way to generate masks directly from the easymocap processing code?

Can you provide a file for generating mesh？

Thank you for the excellent work!

When experimenting with The People Snapshot Dataset, there were some bugs when loading the data, which should be related to the dataset format. I don't know whether I did not find the right way to do it, but after making some simple changes to the code, it can run successfully.

img_paths = list_files(os.path.join(self.dataset_path, 'images'), exts=['.png'])

subject = self.now_subject
time_int = int(frame_name)

time_int = int(frame_name)

I want to know the basic format of the data set, because it is not clearly written in the framework of easymocap. Thanks a lot.

Thanks for the great work~ I wonder how to train the pretrained model?

Hello @taoranyi,

I have trained all my models from scratch. For people-snapshot, The rendering result is good but the camera angle is not accurate.
for z-axis

for y-axis

I have processed the file from EasyMocap "demo for motion capture",
I am not understanding what could be the error.

I checked your cameras.pkl, it is the same as mine. Have you faced any issues like this?

@taoranyi, Hello

Since it is mentioned in the repo, that the People-snapshot dataset was processed using easymocap "Demo motion capture on monocular videos". I tried to do the same but for some reason, I encountered dataloader issue.

Steps I followed:

1. Extracted the images using "extract_images.py" and stored it in " /home/ndip/EasyMocap/images/01/ folder" and 2dkeypoints in /home/ndip/EasyMocap/keypoints/01/
2. Running the following command - python apps/demo/mocap.py /home/ndip/EasyMocap/ --work internet --subs 01
3. I have placed the SPIN, MANO and SMPL-H in the respective folders
4. I did not pass the intrinsic and extrinisic yml files - so the files were generated automatic
5. There is an output folder named - output-smpl-3d which has 3 yml files

Why is the "exp" always smpl-3d, is there a way to change the name? Since there are many modes available.

It will be helpful if you can provide any information, how you processed the Snapshot dataset using Easymocap