pedrodiamel / ferattention Goto Github PK

View Code? Open in Web Editor NEW

74.0 8.0 25.0 57.24 MB

FERAtt: Facial Expression Recognition with Attention Net

License: MIT License

Python 1.81% Jupyter Notebook 98.14% Shell 0.04% Dockerfile 0.01%

pytorch fer facial-expression-recognition deep-learning

ferattention's Introduction

FERAtt: Facial Expression Recognition with Attention Net

This repository is under construction ...

Paper | arXiv

Pedro D. Marrero Fernandez¹, Fidel A. Guerrero-Peña¹, Tsang Ing Ren¹, Alexandre Cunha²

1 Centro de Informatica (CIn), Universidade Federal de Pernambuco (UFPE), Brazil
2 Center for Advanced Methods in Biological Image Analysis (CAMBIA) California Institute of Technology, USA

Introduction

Pytorch implementation for FERAtt neural net. Facial Expression Recognition with Attention Net (FERAtt), is based on the dual-branch architecture and consists of four major modules: (i) an attention module $$G_{att}$$ to extract the attention feature map, (ii) a feature extraction module $G_{ft}$ to obtain essential features from the input image $I$, (iii) a reconstruction module $G_{rec}$ to estimate a good attention image $I_{att}$, and (iv) a representation module $G_{rep}$ that is responsible for the representation and classification of the facial expression image.

Prerequisites

Linux or macOS
Python 3
NVIDIA GPU + CUDA cuDNN
PyTorch 1.5

Installation

$git clone https://github.com/pedrodiamel/pytorchvision.git
$cd pytorchvision
$python setup.py install
$pip install -r installation.txt

Docker:

docker build -f "Dockerfile" -t feratt:latest .
./run_docker.sh

Visualize result with Visdom

We now support Visdom for real-time loss visualization during training!

To use Visdom in the browser:

# First install Python server and client
pip install visdom
# Start the server (probably in a screen or tmux)
python -m visdom.server -env_path runs/visdom/
# http://localhost:8097/

How use

Step 1: Train

./train_bu3dfe.sh
./train_ck.sh

Citation

If you find this useful for your research, please cite the following paper.

@InProceedings{Fernandez_2019_CVPR_Workshops,
author = {Marrero Fernandez, Pedro D. and Guerrero Pena, Fidel A. and Ing Ren, Tsang and Cunha, Alexandre},
title = {FERAtt: Facial Expression Recognition With Attention Net},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2019}
}

Acknowledgments

Gratefully acknowledge financial support from the Brazilian government agency FACEPE.

ferattention's People

Contributors

Stargazers

Watchers

ferattention's Issues

Cannot start training

Hi, I checked out your master branch, installed your pytorchvision package and tried running the bu3dfe training script, but it failed with the following error:

Traceback (most recent call last):
  File "../train.py", line 221, in <module>
    main()
  File "../train.py", line 143, in main
    network.resume( os.path.join(network.pathmodels, args.resume ) )
  File "/home/spectre/anaconda3/envs/thesis/lib/python3.6/site-packages/pytvision-0.0.0-py3.6.egg/pytvision/neuralnet.py", line 287, in resume
  File "/home/spectre/anaconda3/envs/thesis/lib/python3.6/site-packages/torch/nn/modules/module.py", line 535, in __getattr__
    type(self).__name__, name))
AttributeError: 'FERAttentionGMMNet' object has no attribute 'module'

I am using python 3.6 and pytorch 1.0.1

training.csv

From where can I get the training.csv file, I downloaded the dataset from link http://mohammadmahoor.com/affectnet/ but they haven't provided training.csv file.

Unable to open object (object 'data' doesn't exist)

Hi I got this error. And I wonder how you preprocess your images

No protocol specified
=> no checkpoint found at '../out/attnet/feratt_attnet_ferattention_attloss_adam_ck_dim32_bbpreactresnet_fold0_000/models/model_best.pth.tar'
Traceback (most recent call last):
File "/home/haixuanguo/Documents/ferattention/train.py", line 296, in
main(params=args)
File "/home/haixuanguo/Documents/ferattention/train.py", line 154, in main
data = FactoryDataset.factory(
File "/home/haixuanguo/Documents/ferattention/torchlib/datasets/factory.py", line 176, in factory
data = fer.FERClassicDataset(pathname, 'ck', idenselect=idenselect, train=btrain )
File "/home/haixuanguo/Documents/ferattention/torchlib/datasets/fer.py", line 126, in init
self.data = np.array(f["data"])
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "/usr/lib/python3/dist-packages/h5py/_hl/group.py", line 264, in getitem
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object 'data' doesn't exist)"

the variable "loss_att" is overrided

in the class AttMSEloss, in the forward function.

loss_att = ((( (x_org*y_mask[:,1,...].unsqueeze(dim=1)) - att ) ** 2)).mean()
this result is overrided by the next sentence,
loss_att = ((( x_org - att ) ** 2)).mean()

is this a bug?

question about Structured Gaussian Manifold Loss

hi, dear Pedro Diamel Marrero Fernández, Recently, I am reading the paper "ferratt: facial expression recognition with attentionnet". Then I have a question about the loss function. I don't understand why to set structured Gaussian manifold loss. hope you could answer it. Thank you very much

Affect load data

Can you explain the numbers in this line?
ifilter[np.array([235929, 315313] if org else [235929, 315313, 87432, 126295])] = False

Thank you.

Fix Installation Guide

There is no installation.txt file in the pytorchvision folder which is cloned

Datasets link

Hi,
Would you be able to provide link of datasets used to train.
Thank you.

No checkpoint

Hi, I got this error.
=> no checkpoint found at '../out/attnet/feratt_attnet_ferattention_attloss_adam_ck_dim32_bbpreactresnet_fold0_000/models/model_best.pth.tar'

"pytvision" package doesn't exist

Hi there! I've been exploring the repository and couldn't locate the "pytvision" package mentioned in the code or documentation. It seems to be missing or perhaps named differently. Could you please clarify the availability and location of the "pytvision" package? Thank you!

Object 'data' doesn't exis

After disabling the parallel thing, I got this error
Do you have any idea how to resolve it

KeyError: "Unable to open object (object 'data' doesn't exist)"

pedrodiamel / ferattention Goto Github PK

ferattention's Introduction

FERAtt: Facial Expression Recognition with Attention Net

Paper | arXiv

Introduction

Prerequisites

Installation

Visualize result with Visdom

How use

Step 1: Train

Citation

Acknowledgments

ferattention's People

Contributors

Stargazers

Watchers

Forkers

ferattention's Issues

Recommend Projects

Recommend Topics

Recommend Org