wissingchen / vlci Goto Github PK

Hi!

Thanks for the paper and implemantation. Both are perfect! I am trying to run inference on my X-Ray dataset, however tokenizer module needs annotation.json files for both MIMIC-CXR and IU X-Ray datasets. I found the IU X-Ray's, but could not find the MIMIC-CXR. I requested, but it would be wonderful if you provide ONLY annotation json file.

Thanks

Hi, Weixing,
I extend my gratitude for your generosity in sharing the open-source code. However, I have encountered challenges in replicating the clinical metrics (i.e., F1) outlined in your paper using the provided checkpoint on the MIMIC-CXR dataset.

In the process of computing CE metrics, I felt that I might have gone wrong at one step or another.

To elaborate, when utilizing your pretrained VLCI model on the MIMIC-CXR dataset, our obtained NLP and clinical metrics are as follows:
BLEU4: 0.113
METEOR: 0.144
ROUGE_L: 0.276
CIDEr: 0.174

Precision: 0.314
Recall: 0.181
F1: 0.179

Here are a few key points:

1. I use cheXbert to extract entities from gt and pred reports. (I will use cheXpert to extract and compute CE metrics later.)
2. I use compute_ce.py from R2Gen (https://github.com/zhjohnchan/R2Gen/blob/main/compute_ce.py) to compute CE metrics.
   The extracted entity csv files of gt and pred are attached.

labeled_reports_gts.csv
labeled_reports_res.csv

I eagerly await your insights on this matter.
Best

非常抱歉打扰你，请问运行训练文件是通过python main.py -c config/vlp.json指令来运行吗？我运行的结果为什么是负无穷呢？
epoch: 30 32/33 lv: 0.146 lt: 0.670
Epoch 30 mean_loss: 0.1476 0.7979 time: 22.8444s
Saving checkpoint: results/pretrain/current_checkpoint.pth ...
Best results (w.r.t BLEU_4) in validation set:
val_BLEU_4 : -inf
Best results (w.r.t BLEU_4) in test set:
test_BLEU_4 : -inf
希望你能抽空回答我的问题，非常感激！

Hi!

Thanks for the implementation. I am working on your repository and I realized that your MixTokenizer reads only IU Xray annotations. Even Mimic-CXR annotation.json path is defined above code lines, IU Xray's json path is given for Mimic-CXR too. Also your pretrained models were trained with this bug. They are only loaded with this configuration. You can see the wrong line from the link below.

https://github.com/WissingChen/VLCI/blob/216038fe28e1fb9e3d2fee5b7c76b8dc843c4397/utils/tokenizers.py#LL132C53-L132C53

Hi, thanks for your work. Do you provide the code for the attention visualisation in your work, I could not find it. If so, could you provide the code for this section, please?

Nice work! Can you release the code for training? Thank you very much!

This is a good job that has inspired me greatly. However, I'm not clear on how the CE metrics (Precision, Recall, F1-Score) in the paper are calculated, and I haven't been able to find it in the source code either. I would like to ask how missing values are handled after CheXpert extracts labels. From what I've searched, there are two methods: one is to treat missing values as 0 and treat the remaining values (1,-1,0) as 1 (as shown in https://github.com/MIT-LCP/mimic-cxr/blob/master/txt/validation/compare_negbio_and_chexpert.ipynb), while the other is to treat missing values as 0 and keep the remaining labels unchanged for calculation. Alternatively, there may be other ways of handling. Could you please let me know which method was used or provide the relevant code for this?

“We provide the well-trained models of VLCI for inference, and you can download from [here] https://pan.baidu.com/s/1qlWmUfUXsvHyPaqaOtJ-fQ?pwd=m5aq).”

作者您好，我的预训练后，再验证测试的效果不太好'task_name': 'funtune_vlci_iuxray', 'val_BLEU_1': 0.1380630630630553, 'val_BLEU_2': 0.09158504914607858, 'val_BLEU_3': 0.06221934861401921, 'val_BLEU_4': 0.04247527165566198, 'val_METEOR': 0.08422837893406326, 'val_ROUGE_L': 0.15350846384780967, 'val_CIDEr': 0.003667854710952635, 'test_BLEU_1': 0.13562146892654986, 'test_BLEU_2': 0.08884478297779994, 'test_BLEU_3': 0.06049948761282417, 'test_BLEU_4': 0.041740465382958794, 'test_METEOR': 0.09046227977818676, 'test_ROUGE_L': 0.15533803930844284, 'test_CIDEr': 0.0033052960085075836}能加您个联系方式，耽误您一点时间，请教一下您吗

Hi there, I'm trying to download the well-trained models of VLCI for inference from the link provided in the README, but the link appears to be broken. When I click on it, I receive an error message saying the page cannot be found. Could you please update the download link or provide an alternate method for accessing the models? Thanks!