asher-stern / crf Goto Github PK

L-BFGS convergence criterion is wrong (requires that the difference between subsequent function values would be small).

This leads to too many LBFGS iterations, and makes run-time significantly too long.

It can happen (it, in fact has happened to me) that values calculated during the forward-backward algorithm become larger than Double.MAX_VALUE, and are represented using POSITIVE_INFINITY.

These are values we can't really compute with, leading to NaN values and wrecking our training procedure.

I would propose to use maximum finite double values in these cases. Add the following lines to CrfUtilities.safeAdd():

if (variable == Double.POSITIVE_INFINITY) variable = Double.MAX_VALUE;
if (variable == Double.NEGATIVE_INFINITY) variable = -Double.MAX_VALUE;

And also provide a function CrfUtilities.safeMultiply():

public static double safeMultiply(double variable, final double val2) {
    final double oldValue = variable;
    variable *= val2;
    if (variable == Double.POSITIVE_INFINITY) variable = Double.MAX_VALUE;
    if (variable == Double.NEGATIVE_INFINITY) variable = -Double.MAX_VALUE;
    if (!Double.isFinite(variable)) {
        //  Note that we have not added the check for ((val2 < 0.0) && (oldValue < variable)) || ((val2 > 0.0) && (oldValue > variable)), because floating point arithmetic is inexact
        throw new CrfException("Error: multiplying value to \"double\" variable yielded unexpected results. "
                + "variable was: " + String.format("%1$.3f", oldValue) + ", value to multiply was: " + String.format("%1$.3f", val2));
    }
    return variable;
}

And use these functions in all places where we do floating point calculations.

I don't know too much about the internals of this project, so I would like your feedback on this.

in LbfgsMinimizer
The code previousItrations.removeLast(); is wrong!
It should be previousItrations.removeFirst();
(That's a terrible bug...)

Hi,
i am a student from China, and i am learning your CRF code.
But i can't get the training data --- Penn Tree-Bank corpus.
I know the corpus is not free, so I just want to get 10 - 20 sentence of the corpus for test.
could you copy some sentence to me for testing ?
Thank you very much !

LiKun,Zhengzhou University,China