FunctionUtil.evaluate order about jpmml-evaluator HOT 4 CLOSED

The method FunctionUtil#evaluate(...) performs the resolution of a function in the following order:

1. PMML built-in function.
2. Java UDF.
3. PMML UDF as defined by some DefineFunction element.

You cannot override PMML built-in functions (for sanity and security reasons). But you can override DefineFunction elements if you register a Java UDF with the same name during the startup of your application.

For example, every SAS EM generated PMML document that deals with string variables includes a SAS-EM-String-Normalize PMML UDF. The evaluation of PMML UDFs is slow, because it will be necessary to isolate their execution into a separate scope. However, you can override this PMML UDF by registering an instance of org.jpmml.sas.functions.StringNormalize like this:

import org.jpmml.sas.functions.StringNormalize;

FunctionRegistry.putFunction(new StringNormalize());

Please note that you will be registering a singleton instance of your Java UDF. So, it is your responsibility to make sure that its implementation of Function#evaluate(List<FieldValue>) is thread safe.

from jpmml-evaluator.

Thanks for the fast reply. I guess this makes sense if the functions are
relatively static. Will have to check with data science tomorrow if that is
an option for us.

Would be great though to have an option like with the optimizer visitors to
prioritise DefineFunctions over reflection.

Thanks for your great work.
On 3 Mar 2016 9:02 p.m., "Villu Ruusmann" [email protected] wrote:

The method FunctionUtil#evaluate(...) performs the resolution of a
function in the following order:

1. PMML built-in function.
2. Java UDF.
3. PMML UDF as defined by some DefineFunction element.

You cannot override override PMML built-in functions (for sanity and
security reasons). But you can override DefineFunction elements if you
register a Java UDF with the same name during the startup of your
application.

For example, every SAS EM generated PMML document that deals with string
variables includes a SAS-EM-String-Normalize PMML UDF. The evaluation of
PMML UDFs is slow, becaue it will be necessary to isolate their execution
into a separate scope. However, you can override the PMML UDF by
registering an instance of org.jpmml.sas.functions.StringNormalize like
this:

import org.jpmml.sas.functions.StringNormalize;

FunctionRegistry.putFunction(new StringNormalize());

Please note that you will be registering a singleton instance of your Java
UDF. So, it is your responsibility to make sure that its implementation of
Function#evaluate(List) is thread safe.

—
Reply to this email directly or view it on GitHub
#17 (comment)
.

from jpmml-evaluator.

It is not a good idea to change the order of steps 2 and 3, because then you would lose the ability to override PMML UDFs with Java UDFs.

What could be done rather easily, is adding a programmatic configuration option that would prevent FunctionRegistry from performing function look-ups using Java reflection API. I had the impression that JVM is caching the results of ClassLoader.loadClass(String), but your profiling results indicate that this might not be so.

Will have to run actual experiments on this.

from jpmml-evaluator.

I agree on the default order, but I guess we could load the DefineFunction
first and assuming it was replaced with a RichDefineFunction we can check
if the priority is set. If so it takes precedence over Java UDF.
On 3 Mar 2016 9:32 p.m., "Villu Ruusmann" [email protected] wrote:

It is not a good idea to change the order of steps 2 and 3, because then
you would lose the ability to override PMML UDFs with Java UDFs.

What could be done rather easily, is adding a programmatic configuration
option that would prevent FunctionRegistry from performing function
look-ups using Java reflection API. I had the impression that JVM might be
caching the results of ClassLoader.loadClass(String), but your profiling
results indicate that this might not be so.

Will have to run actual experiments on this.

—
Reply to this email directly or view it on GitHub
#17 (comment)
.

from jpmml-evaluator.