PMML conversion : Casting float to decimal causing loss of Precision. about jpmml-converter HOT 7 CLOSED

Thanks @sudarshan1413 for your feedback! I will be releasing an updated sklearn2pmml package later today, so you should be able to export corrected models straight from your Python scripts soon.

As for XGBoost problems, then it must be something else than the data type/precision of split thresholds. XGBoost does all its computations using the float data type exclusively. The most common cause for R/XGBoost vs. (J)PMML mismatches is that the XGBoost feature map has been generated incorrectly. For example, see jpmml/r2pmml#26

from jpmml-converter.

In Python Tree:

The print-out of Python Tree does not use correct numeric type for representing numeric values. For Python Tree, all values are double, but they should be either float or int (and never double).

In above case 526225.988222 getting changed in to 526226 ...

Feature "AA.SUB" is an integer value, so 526226 is a correct representation.

... and 5192.53104377 into 5192.5312

Feature "BB.SUB" is a 32-bit floating-point value, so 5192.5312 is a correct representation.

I have analyzed the source code of jpmml-converter and found that the way converting Float values to Double is wrong.( in ValueUtil.java ).

JPMML-Converter does not perform arbitrary data type conversions. If the underlying ML framework uses 32-bit math, then all values are represented in PMML using Float#toString().

Can you please analyze and see if this is the issue.

This is not an issue. The JPMML software stack is behaving correctly, the generated PMML document is both syntactically and semantically correct, and when evaluated using the JPMML-Evaluator library (don't know about other PMML engines) gives exactly the same predictions as Scikit-Learn.

from jpmml-converter.

@vruusmann you say "JPMML-Converter does not perform arbitrary data type conversions. If the underlying ML framework uses 32-bit math, then all values are represented in PMML using Float#toString()." but jpmml-sklearn casts tree thresholds to single-precision float here.

from jpmml-converter.

but jpmml-sklearn casts tree thresholds to single-precision float here.

@jnothman But isn't it the case that Scikit-Learn stores threshold values as float64, but evaluates them as float32? The X argument of predict(X) and predict_proba(X) methods is explicitly cast into float32 array type, so it would only make sense to compare them against float32 thresholds as well.

The PMML markup holds evaluation-time value representations, so float32 (as implemented by Float#toString()) is correct (I've got integration tests - with Float#toString() everything passes cleanly, but with Double#toString() there is around 1 to 3% bad predictions).

As pointed out in scikit-learn/scikit-learn#9636 (comment), the probable cause for bad predictions in the current case is "pretty formatting of integer-valued floats as integers". Must be the case that 526225.988222 is a special float value, which is declared to be integer-valued by com.google.common.math.DoubleMath#isMathematicalInteger(double), but actually isn't so. The problem is that this method gives correct results for Java's double values, but not for (double)float values (as returned by Float#doubleValue()).

Reopening - the pretty formatting code is a bad/premature optimization, and needs to go.

from jpmml-converter.

from jpmml-converter.

@AkshayTilekar, @sudarshan1413 - Can you check out the HEAD version of the JPMML-SkLearn library, build the executable uber-JAR file, and use it to convert your previously problematic Scikit-Learn pipelines?

Does the commit jpmml/jpmml-sklearn@8c1b208 resolve your problem with mismatching predictions (between Scikit-Learn and (J)PMML) or not?

from jpmml-converter.

Hi vruusmann , thanks a lot for that. It is working fine for us and we were able to match results perfectly with the compiled jar from your HEAD version.

We are facing similar problem with r2pmml also , where we are converting xgboost model to pmml and then trying to match its result with R predictor. Here also we are seeing precision issue similar to that of Python . May be similar issue is happening with r2pmml (jpmml-r) also.... Any thoughts?

from jpmml-converter.