The repository contains the majority of the code to reproduce the experimental results of the paper Structured Attentions for Visual Question Answering on the VQA-1.0 and VQA-2.0 dataset. Currently only the accelerated version of Mean Field is provided, which is used in the VQA 2.0 challenge.

To reproduce the experimental results,

• Clone and compile mxnet, with mxnet@c9e252, cub@89de7ab, dmlc-core@3dfbc6, nnvm@d3558d, ps-lite@acdb69, mshadow@8eb1e0. There has been some modification on optimizers (and others) in later versions of mxnet, and code in this repository has not been adapted yet.

• ResNet-152 feature of MS COCO images: extracted with MCB's preprocess code.

• Our training question and answer data for VQA2.0: Baidu Pan.

Set the arguments and run train_VQA.py.

The best single model accuracy on test-dev of VQA-1.0 and VQA-2.0 with skip-thought vector initialization and Visual Genome training data are 67.19 and 64.78 respectively. Here is the model on VQA-2.0.

If you found this repository helpful, you could cite

@article{chen2017sva,
  title={Structured Attentions for Visual Question Answering},
  author={Chen, Zhu and Yanpeng, Zhao and Shuaiyi, Huang and Kewei, Tu and Yi, Ma},
  journal={IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
}

This code is distributed under MIT LICENSE.