question about fashionbert about easytransfer HOT 8 OPEN

(1) We do not attempt to train fashionbert only with image-text-matching task. Since we evaluate fashionbert model with fashion-gen dataset, we donot perform fine-tuning again.

(2) Need to resize the image patch to fit the feature extraction.

(3) Resnet50

from easytransfer.

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

from easytransfer.

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

1. randomly mask the image patches
2. fashionbert is continuely pretrained based on google bert(base) with fashionGen dataset

from easytransfer.

I am confused about the pertained model you loaded in the code

where the 'pretrain_model_name_or_path=pai-imagebert-base-en' instead of ‘= google-bert-base-en’

from easytransfer.

When you run_train.sh first time, ez will automatically download model to modelZooBasePath. Pleas check your os.getenv("HOME") / eztransfer_modelzoo.
flags.DEFINE_string("modelZooBasePath", default=os.path.join(os.getenv("HOME"), ".eztransfer_modelzoo"), help="eztransfer_modelzoo")

from easytransfer.

Yes. What I am confused is about the downloaded model.

In the readme file, the command for pretrained model is 'pretrain_model_name_or_path=pai-imagebert-base-en' instead of ‘= google-bert-base-en’. Should I change it to googlebert when making pretrain on FashionGen?

from easytransfer.

You can conduct two experiments to compare the two pretrained models.

from easytransfer.

Thanks a lot!
Do you use the specific masks for image patche in training set? Besides, in your code, a pertained imagebert is used for initializing the fashionbert. Which dataset the imagebert is pretrained based on?

1. randomly mask the image patches
2. fashionbert is continuely pretrained based on google bert(base) with fashionGen dataset

Thank you for your excellent work. May I ask one following-up question about FashionBERT?
Could you please give some details about the ratios of positive pairs and negative pairs during the training process? Like in one mini-batch, 50% of examples are positive and the rest are negative. Thanks a lot!

from easytransfer.