tudor67 / object-counting Goto Github PK

Wrong implementation of evaluation metrics for model.fit_generator():

• rmse_keras;
• underestimate_keras;
• overestimate_keras;
• difference_keras.
  These functions/methods are from ./utils/evaluation/evaluation.py.

Keras will calculate these metrics per batch and will return their mean.
This is not ok for some evaluation metrics.

I think that one example is enough to get an idea about this:
rmse([pred_batch_1, pred_batch_2, ... pred_batch_k]) != mean (rmse(pred_batch_1), rmse(pred_batch_2), ..., rmse(pred_batch_k))

In other words:
sqrt((a + b + c + d) / 4) != (sqrt((a + b) / 2) + sqrt((c + d) / 2)) / 2.

We need sqrt((a + b + c + d) / 4), but the current implementation returns (sqrt((a + b) / 2) + sqrt((c + d) / 2)) / 2.