Hi, thanks for your work.
Here I use your model weight and source code to do a prediction, when I use the model of two_hands_seg, it's ok.
however, when I use obj1, obj2,etc, it is shown an Error about the input channel（swin input expect 5, but got 3）,
is there anythin i need to do before input image to the model?

There is no contact boundary information in this dataset downloaded from the link. Where it is?

Is there a way to map the available frames back to the original video sources?

I am working on some video understanding tasks and would be interested in analyzing the annotated frames in this dataset in the context of their respective videos. Any advice on how I can create this mapping will also help.

Thank you!

Hi, when I was downloading the dataset, it was always interruptted at 50%. dunno how to handle this. could u help with that? thanks

Hi! I really like your paper and the high-quality code+data.

You mention in the dataset portion of the paper that:

We think both types of labels are useful depending on the application, so we provide both types of hand mask labels for all images in our dataset, where one for hands and another one for the rest of the arms.

I was checking the data, and I'm not sure where to find the hands-only. Is that a separate download? Maybe I missed it. Thanks so much!

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

Attention: please create a new virtual environment for OpenMMLab 2.0.

Thanks for sharing this.
As I was running download_checkpoints.sh I needed to add --fuzzy for the download to start!

 gdown https://drive.google.com/file/d/1DEJBeQ3cR1q7cjjzwDUIQVSoptT-y9U7/view?usp=drive_link
/../envs/openmmlab/lib/python3.8/site-packages/gdown/parse_url.py:48: UserWarning: You specified a Google Drive link that is not the correct link to download a file. You might want to try `--fuzzy` option or the following url: https://drive.google.com/uc?id=1DEJBeQ3cR1q7cjjzwDUIQVSoptT-y9U7
  warnings.warn(
Downloading...
From: https://drive.google.com/file/d/1DEJBeQ3cR1q7cjjzwDUIQVSoptT-y9U7/view?usp=drive_link
To: /../EgoHOS/mmsegmentation/view?usp=drive_link
82.4kB [00:00, 66.9MB/s]

It seems like the google drive is not accessible. I am not able to download the weights/images etc and get an error when I try to open the drive. Would it be possible to fix this?

I am very interested in an application to classify hand state with EgoHOS.
Do you have any resources on that?

After downloading the dataset, I noticed the format is not as mentioned on the repo description (it does not contain the contact info) and the labels are all black masks (arrays of 0s). Was wondering the reason or if extra processing is required.

I transferred my own images in the "testimages/images" directory and ran inference with the following command:

config_file = "./EgoHOS/mmsegmentation/work_dirs/twohands_to_cb_ccda/twohands_to_cb_ccda.py"
checkpoint_file = "./EgoHOS/mmsegmentation/work_dirs/twohands_to_cb_ccda/best_mIoU_iter_76000.pth"
pred_seg_dir = "./EgoHOS/testimages/pred_cb"
model = init_segmentor(config = config_file, checkpoint=checkpoint_file, device='cuda:0')

img_dir = "./EgoHOS/testimages/images"
file = img_dir + "/IMG_8776.jpg"
fname = os.path.basename(file).split('.')[0]
print(fname)
seg_result = inference_segmentor(model, file)[0]
seg_result

Resulting in the following error:

FileNotFoundError: [Errno 2] No such file or directory: './EgoHOS/testimages/pred_twohands/IMG_8776.png'

It appears the model object attempts to read the image with a "png" extension when calling the "whole_inference" function in order to get the image height and width. I don't understand why this would be the case..

The code works fine when using the default test images.

Great paper!! Do you have any steps for training the model using the context-aware compositional data augmentation technique?

Hello, could you please explain why the results in the paper show that the performance of the 1st left-hand object is better than the 1st right-hand object, both in parallel and sequential methods? When I reproduced your result, I found the performances of these two categories are similar, and there is no unbalanced data distribution for these two categories in the EgoHOS dataset.

when the time to upload the checkpoint?