Is the generation text guided? about arldm HOT 9 CLOSED

I've already understood. I didn't notice before that there are two tasks to choose.

from arldm.

@lll-zy Great! You can refer to our paper to the task setting, feel free to address any further question

from arldm.

Thank you for your reply! But I still don't understand how test data is applied in visualization task. In the appendix of the paper, each ground truth image correspends to one text, and there are five texts and five outputs, right? However each sample in test dataset has five frames, they correspond to the same text. I don't understand what is the ground truth and its description when generating.

from arldm.

@lll-zy Hi, it is not correct that "However each sample in test dataset has five frames, they correspond to the same text.". If this happen, there must exist some bug in the implementation. Could you please share some case to me?

from arldm.

Maybe I misunderstood. I downloaded the flintstones dataset, which includes many .npy files. And the shape is (5, 128, 128, 3). In flintstones_annotations_v1. json, each .npy corresponds to a text.

from arldm.

@lll-zy yep, that's true, and each text should contains 5 captions. Each caption is corresponding to a frame

from arldm.

Are these 5 captions also in flintstones_annotations_v1. json? I can only find 'description' for each npy. And, the content of these 5 frames in each npy seems similar, like

from arldm.

@lll-zy It makes sense because every frames is sampled from video, the caption is in "flintstones_annotations_v1-0.json", as you can see in:

from arldm.

I'll take a closer look. Thanks again for your answer.

from arldm.