The implementation of Medical Visual Question Answering via Conditional Reasoning [ACM MM 2020]
We evaluate our proposal on VQA-RAD dataset.

We propose QCR [Question-Conditioned Reasoning Module] and TCR [Type-Conditioned Reasoning] modules, which guide the importance selection over multimodal fusion features and learn different reasoning skills for different types of tasks separately. The detailed architecture is shown in the figure below.

Our proposal achieves significantly increased accuracy in predicting answers to both closed-ended and open-ended questions, especially for open-ended questions.

pip install -r requirements.txt

All pre-prepared data can be found in data package

git clone https://github.com/Awenbocc/Med-VQA.git
cd ./Med-VQA
python main.py --gpu 0 --seed 88

MIT License

Please cite following in your publications if they help you.

@inproceedings{zhan2020medical,
  title={Medical visual question answering via conditional reasoning},
  author={Zhan, Li-Ming and Liu, Bo and Fan, Lu and Chen, Jiaxin and Wu, Xiao-Ming},
  booktitle={Proceedings of the 28th ACM International Conference on Multimedia},
  pages={2345--2354},
  year={2020}
}