wiibrew / eac-net Goto Github PK

Hello,

As you say the different subjects for the 3-fold cross validation are found in the README. If I follow those files and extract the subject, this is what I found:

• Test for fold 2 BP4D_ts_fold2.txt:
  ['F008', 'F009', 'F010', 'F011', 'F013', 'F014', 'M007', 'M008', 'M009', 'M010', 'M011', 'M012']
• Test for fold 3 BP4D_ts_fold3.txt:
  ['F001', 'F002', 'F003', 'F004', 'F005', 'F006', 'F007', 'M001', 'M002', 'M003', 'M004', 'M005', 'M006']

However, they do not match with the subjects used for DRML. With personal communication with @zkl20061823, they say that they use these subjects. For instance:
Fold1: ['F001', 'F002', 'F008', 'F09', 'F010', 'F016', 'F018', 'F023', 'M001', 'M004', 'M007', 'M008', 'M012', 'M014']
Fold2: ['F003', 'F005', 'F011', 'F013', 'F020', 'F022', 'M002', 'M005', 'M010', 'M011', 'M013', 'M016', 'M017', 'M018']
Fold3: ['F004', 'F006', 'F007', 'F0012', 'F0014', 'F015', 'F017', 'F019', 'F021', 'M003', 'M006', 'M009', 'M015']

So, could you please clarify this?

Could you please share the features for the LSTM inputs? I am trying to reproduce them but I cannot reach 10% of improvement only by using LSTM as you report in the paper. For instance, I cannot improve my results using LSTM at all.

I am asking this because I am having a bad time trying to install Theano under Cuda 9.1 and all your coding dependencies.

Why do you add 0.5 to the predictions here?

Is it like doing a threshold of 0.5 and then calculating f1_score over two binary variables? Is there any reason to choose 0.5 as threshold?

Which is train script, how should I train on different dataset, Do we need to write load data script?

I repeat my understanding about your codes, can you tell me is it exactly what you mean?
In this example , I omit the procedure of random sample, just look it.
suppose we have sequence of a,b,c,d,e,f,g.
the LSTM's 1st time input : a,b,c and output: o_a, o_b, o_c. you need to concatenate (o_a, o_b, o_c) to calculate loss, all ground truth of (label_a, label_b, label_c) will be used in calculate loss.
the LSTM's 2nd time input: b,c,d and output: o_b,o_c, o_d you need to concatenate (o_b, o_c, o_d) to calculate loss, all ground truth of (label_b, label_c, label_d) will be used in calculate loss.

so my question is:

1. is this example I showed exactly what you do in your experiment?
2. In this example, In order to train the whole dataset, so overlap sequence will repeatly input to LSTM in order to calcuate all position of sequence.
   3..Because there exists a lot of 0-label (image frame with no action on its face) frame, Do you train every sample in train set OR just pick that frame which have AU label != 0. input to LSTM?

Where can we find the source code for any of the other frameworks?

please! Thanks