Comments (4)
Solution
-
Fix your direct dependencies to be py4j==0.10.7 and pyspark ==2.4.3, to remove this conflict.
I have checked this revision will not affect your downstream projects now. -
Ask your upstream project pyspark to loose the version range of py4j to be >=0.10.7.
Which solution do you prefer, 1 or 2?
Please let me know your choice. Then I can submit a PR to fix it.
from pypmml-spark.
@NeolithEra thanks for your feedback. About solution 1, the dependencies are too strict, and for solution 2, I'm not clear about the fix, could you explain it detailed?
Based on the doc of pyspark, the versions 2.4.x depend on the exactly py4j==0.10.7, maybe have not tested on py4j 0.10.8, not supported, or some reasons.
Actually, it's better that PySpark takes over the dependency of py4j, that means removal py4j from dependencies of PyPMML-Spark itself. Do you have any comments about this solution?
from pypmml-spark.
@scorebot I agree with your solution. Taking over the dependency of py4j is a good choice!
I have submitted a PR following your suggestion.
from pypmml-spark.
I close this issue as the referred PR has been merged.
from pypmml-spark.
Related Issues (8)
- java.io.NotSerializableException: org.pmml4s.transformations.FieldColumnPair #1 HOT 2
- Output unmatch between jpmml and pypmml HOT 11
- NoClassDefFoundError: scala/Product$class HOT 5
- this tool only support pyspark<3.0 ? when i was trying using ScoreModel.fromFile("{"file_path}"), an Error called """TypeError: 'JavaPackage' object is not callable""" ocurred HOT 6
- It couldn't output raw prediction probability. HOT 12
- Importing .pmml file throws error TypeError: 'JavaPackage' object is not callable HOT 2
- TypeError: 'JavaPackage' object is not callable error despite linking jars into spark succesfully HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pypmml-spark.