Giter VIP home page Giter VIP logo

bias-gzsl's Introduction

bias-gzsl

This is the source code for reproducing the experiments from the paper:

Bias-Awareness for Zero-Shot Learning the Seen and Unseen
William Thong and Cees G.M. Snoek
British Machine Vision Conference (BMVC), 2020

TL;DR We mitigate the classifier bias towards classes seen during training in generalized zero-shot learning (GZSL). To achieve this, we calibrate the model with temperature and apply entropy regularization to balance out seen and unseen class probabilities.

We validate the proposed approach on four benchmarks (CUB, SUN, AWA, and FLO) in two scenarios:
(a) stand-alone classifier with only seen class features,
(b) classifier with real seen class samples and generated unseen class features.

In other words, our bias-aware classifier behaves like a traditional compatibility functions for (G)ZSL and can integrate unseen class features from any existing generative model.

Datasets

Features We rely on the features extracted by [Xian, et al, CVPR 2018] and use the same train/test splits. Sentences for CUB and FLO were collected by [Reed et al, CVPR 2016].

  • CUB, SUN, and AWA features are here.
  • CUB sentences are here.
  • FLO features and sentences are here.

Stand-alone classification with seen classes only

GZSL Run runs/stand-alone.sh. This will train a bias-aware model with real seen features only. It will perform GZSL at the end of the training for each of the four benchmarks.

Swapping representations Run runs/swapping.sh. This will train one model with attributes representations, and one with sentence representations. Then, it will extract the features and perform a least squares estimation for swapping the representations between seen and unseen classes.

Classification with both seen and unseen classes

Generating unseen class features We rely on existing models to generate unseen class features. We select f-CLSWGAN and cycle-CLSWGAN, which are publicly available and reproducible.

We train both and extract the features to be used in our bias-aware classifier. Note that our bias-aware classifier is not limited to these two methods to generate features, and can be used for any other method that produces generated features! For this, you just need to save into an .npz file the following:

train_seen_X
train_seen_Y
train_unseen_X
train_unseen_Y
test_seen_X
test_seen_Y
test_unseen_X
test_unseen_Y

or alternatively with seen and unseen merged in the training set:

train_X  # includes both real and generated features
train_Y
test_seen_X
test_seen_Y
test_unseen_X
test_unseen_Y

For convenience, here are features from cycle-CLSWGAN for each dataset (900MB file). Note that it corresponds to features for one seed point. In the paper we report the average over 3 runs, so numbers might be different.

GZSL Run runs/with-generated.sh. This will train a bias-aware model with real seen features and generated unseen features from cycle-CLSWGAN. It perform GZSL at the end of the training for each of the four benchmarks.

Requirements

The source code relies on python 3.6 and pytorch 0.4:

python=3.6.9
pandas=0.25.2
numpy=1.17.3
pytorch=0.4.1
torchvision=0.2.1
scikit-learn=0.22.1

Citation

If you find these scripts useful, please consider citing our paper:

@inproceedings{
    Thong2020BiasGZSL,
    title={Bias-Awareness for Zero-Shot Learning the Seen and Unseen},
    author={Thong, William and Snoek, Cees G.M.},
    booktitle={BMVC},
    year={2020},
    url={https://arxiv.org/abs/2008.11185}
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.