Deserialization for PCA model seems to be broken. Failed to find a default value for "k" about mleap HOT 2 OPEN

Can you tell us a bit about your environment. What mleap and spark version do you have?

from mleap.

Well, I can see that we aren't storing k in spark.ml.bundle.ops.feature.PcaOp, so storing then loading back into Spark will crash like you observed. Looking through changelogs on the files I think it has always been this way 😢 . Should be an easy fix here https://github.com/combust/mleap/blob/master/mleap-spark/src/main/scala/org/apache/spark/ml/bundle/ops/feature/PcaOp.scala if you feel up to making a PR?

Stepping back a bit you're doing something a bit unexpected (and which is not tested in mleap) by trying to serialize then deserialize back into spark. Normally for that code flow people just use Spark's built in serialization/deserialization capabilities. MLeap does test parity between spark->mleap via doing:

• transform in spark
• serialize to bundle
• deserialize to mleap
• transform in mleap

Testing to deserialize back to spark is a good idea but I'm not really sure how many other transformers will also have missing values.

from mleap.