Comments (3)
jsleight
commented on August 12, 2024
Use the make targets in the repo root to build and run tests. Once you have a build that you think works locally, then can commit and push to origin to create a PR. That will trigger a full build on our CI system.
Specifically regarding scala 2.13 compatibility, please aim for cross compiling to support both scala 2.12 and 2.13. sbt has native support for this and the repo should already be set up correctly to utilize it.
from mleap.
prashil1996
commented on August 12, 2024
@jsleight Thanks for replying, also what are the different requirements for compiling the build locally?
Scala, Java, SBT?
I tried building it locally but I was getting some error. Can you help me with some steps? Any wiki?
The following were encountered after running make on the root repo:
17:38:30.945 [pool-11-thread-20] ERROR ml.dmlc.xgboost4j.java.NativeLibLoader - Failed to load xgboost4j library from jar for platform macos/aarch64
17:38:30.948 [pool-11-thread-20] ERROR ml.dmlc.xgboost4j.java.DMatrix - Failed to load native library
java.io.FileNotFoundException: File /lib/macos/aarch64/libxgboost4j.dylib was not found inside JAR.
at ml.dmlc.xgboost4j.java.NativeLibLoader.createTempFileFromResource(NativeLibLoader.java:285) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.dmlc.xgboost4j.java.NativeLibLoader.loadLibraryFromJar(NativeLibLoader.java:228) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.dmlc.xgboost4j.java.NativeLibLoader.initXGBoost(NativeLibLoader.java:168) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.dmlc.xgboost4j.java.XGBoostJNI.<clinit>(XGBoostJNI.java:34) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32) ~[xgboost4j_2.12-1.7.6.jar:?]
at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21) ~[test-classes/:?]
at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14) ~[test-classes/:?]
at jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) ~[?:?]
at java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128) ~[?:?]
at jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304) ~[?:?]
at java.lang.Class.newInstance(Class.java:725) ~[?:?]
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:454) ~[scalatest-core_2.12-3.2.16.jar:3.2.16]
at sbt.TestRunner.runTest$1(TestFramework.scala:153) ~[?:?]
at sbt.TestRunner.run(TestFramework.scala:168) ~[?:?]
at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:336) ~[?:?]
at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:296) ~[?:?]
at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336) ~[?:?]
at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336) ~[?:?]
at sbt.TestFunction.apply(TestFramework.scala:348) ~[?:?]
at sbt.Tests$.processRunnable$1(Tests.scala:475) ~[?:?]
at sbt.Tests$.$anonfun$makeSerial$1(Tests.scala:481) ~[?:?]
at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:47) ~[?:?]
at sbt.std.Transform$$anon$4.work(Transform.scala:69) ~[?:?]
at sbt.Execute.$anonfun$submit$2(Execute.scala:283) ~[?:?]
at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24) ~[?:?]
at sbt.Execute.work(Execute.scala:292) ~[?:?]
at sbt.Execute.$anonfun$submit$1(Execute.scala:283) ~[?:?]
at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265) ~[?:?]
at sbt.CompletionService$$anon$2.call(CompletionService.scala:65) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1583) ~[?:?]
[info] XGBoostPredictorClassificationModelParitySpec:
[info] ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec *** ABORTED ***
[info] java.lang.ExceptionInInitializerError:
[info] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[info] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[info] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[info] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[info] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[info] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[info] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
[info] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304)
[info] at java.base/java.lang.Class.newInstance(Class.java:725)
[info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:454)
[info] ...
[info] Cause: java.lang.RuntimeException: java.io.FileNotFoundException: File /lib/macos/aarch64/libxgboost4j.dylib was not found inside JAR.
[info] at ml.dmlc.xgboost4j.java.XGBoostJNI.<clinit>(XGBoostJNI.java:37)
[info] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[info] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[info] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[info] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[info] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[info] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[info] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
[info] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304)
[info] at java.base/java.lang.Class.newInstance(Class.java:725)
[info] ...
[info] Cause: java.io.FileNotFoundException: File /lib/macos/aarch64/libxgboost4j.dylib was not found inside JAR.
[info] at ml.dmlc.xgboost4j.java.NativeLibLoader.createTempFileFromResource(NativeLibLoader.java:285)
[info] at ml.dmlc.xgboost4j.java.NativeLibLoader.loadLibraryFromJar(NativeLibLoader.java:228)
[info] at ml.dmlc.xgboost4j.java.NativeLibLoader.initXGBoost(NativeLibLoader.java:168)
[info] at ml.dmlc.xgboost4j.java.XGBoostJNI.<clinit>(XGBoostJNI.java:34)
[info] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[info] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[info] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[info] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[info] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[info] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[info] ...
[error] java.lang.ExceptionInInitializerError
[error] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[error] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[error] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[error] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[error] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[error] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[error] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
[error] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304)
[error] at java.base/java.lang.Class.newInstance(Class.java:725)
[error] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:454)
[error] at sbt.TestRunner.runTest$1(TestFramework.scala:153)
[error] at sbt.TestRunner.run(TestFramework.scala:168)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:336)
[error] at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:296)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFunction.apply(TestFramework.scala:348)
[error] at sbt.Tests$.processRunnable$1(Tests.scala:475)
[error] at sbt.Tests$.$anonfun$makeSerial$1(Tests.scala:481)
[error] at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:47)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error] at sbt.Execute.work(Execute.scala:292)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: File /lib/macos/aarch64/libxgboost4j.dylib was not found inside JAR.
[error] at ml.dmlc.xgboost4j.java.XGBoostJNI.<clinit>(XGBoostJNI.java:37)
[error] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[error] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[error] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[error] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[error] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[error] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[error] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
[error] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304)
[error] at java.base/java.lang.Class.newInstance(Class.java:725)
[error] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:454)
[error] at sbt.TestRunner.runTest$1(TestFramework.scala:153)
[error] at sbt.TestRunner.run(TestFramework.scala:168)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:336)
[error] at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:296)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFunction.apply(TestFramework.scala:348)
[error] at sbt.Tests$.processRunnable$1(Tests.scala:475)
[error] at sbt.Tests$.$anonfun$makeSerial$1(Tests.scala:481)
[error] at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:47)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error] at sbt.Execute.work(Execute.scala:292)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] Caused by: java.io.FileNotFoundException: File /lib/macos/aarch64/libxgboost4j.dylib was not found inside JAR.
[error] at ml.dmlc.xgboost4j.java.NativeLibLoader.createTempFileFromResource(NativeLibLoader.java:285)
[error] at ml.dmlc.xgboost4j.java.NativeLibLoader.loadLibraryFromJar(NativeLibLoader.java:228)
[error] at ml.dmlc.xgboost4j.java.NativeLibLoader.initXGBoost(NativeLibLoader.java:168)
[error] at ml.dmlc.xgboost4j.java.XGBoostJNI.<clinit>(XGBoostJNI.java:34)
[error] at ml.dmlc.xgboost4j.java.DMatrix.<init>(DMatrix.java:69)
[error] at ml.dmlc.xgboost4j.scala.DMatrix.<init>(DMatrix.scala:32)
[error] at ml.combust.mleap.xgboost.runtime.testing.CachedDatasetUtils.$init$(CachedDatasetUtils.scala:21)
[error] at ml.combust.mleap.xgboost.runtime.XGBoostPredictorClassificationModelParitySpec.<init>(XGBoostPredictorClassificationModelParitySpec.scala:14)
[error] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[error] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[error] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128)
[error] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:304)
[error] at java.base/java.lang.Class.newInstance(Class.java:725)
[error] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:454)
[error] at sbt.TestRunner.runTest$1(TestFramework.scala:153)
[error] at sbt.TestRunner.run(TestFramework.scala:168)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.$anonfun$apply$1(TestFramework.scala:336)
[error] at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:296)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFramework$$anon$3$$anonfun$$lessinit$greater$1.apply(TestFramework.scala:336)
[error] at sbt.TestFunction.apply(TestFramework.scala:348)
[error] at sbt.Tests$.processRunnable$1(Tests.scala:475)
[error] at sbt.Tests$.$anonfun$makeSerial$1(Tests.scala:481)
[error] at sbt.std.Transform$$anon$3.$anonfun$apply$2(Transform.scala:47)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error] at sbt.Execute.work(Execute.scala:292)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] (mleap-xgboost-runtime / Test / executeTests) java.lang.ExceptionInInitializerError
[error] Total time: 302 s (05:02), completed 14-Oct-2023, 5:38:31 pm
make: *** [test_xgboost_runtime] Error 1
Any inputs on how I can resolve this?
from mleap.
jsleight
commented on August 12, 2024
Ah
- sbt v1.9.4
- java 11
- spark 3.4
(fyi: the/projectdirectory of sbt projects usually has all these details.)
The libxgboost4j being missing is a known thing when trying to build on mac. XGBoost's java libraries don't support mac. iirc tensorflow libraries might have this problem too.
from mleap.
Related Issues (20)
- MLeap Transformer issue. HOT 1
- MathBinary input validation
- Mleap and python 3.8 HOT 27
- Exception in thread "Thread-4" java.lang.NoClassDefFoundError: ml/combust/bundle/HasBundleRegistry HOT 3
- Need help installing MLeap and XGBoost on databricks HOT 6
- Using XGBoost with the newest mleap=0.22.0 in Python 3.8 HOT 10
- Error while testing root project HOT 2
- org.apache.spark.sql.mleap.TypeConverters can not convert 2D tensor to Matrix
- Tensor to Proto Bug with SparseTensor: " java.lang.IllegalArgumentException: size of dimensions must equals size of values"
- org.apache.spark.ml.parity.SparkParityBase.spark should be a method
- Please release version 0.22.0 and 0.23.0 of mleap-spring-boot on dockerhub HOT 5
- Error building mleap-spring-boot HOT 7
- Mleap Spring Boot Swagger Documentation Seems Incorrect HOT 4
- Update springboot to 2.7.10 and snakeyaml to 2.0
- Reporting a security issue in MLeap HOT 1
- Question about Mleap Spring Boot API HOT 2
- How to use XGBoost PySpark API with MLeap? HOT 3
- Pyspark DecisionTreeRegressionModel bundle does not include all attributes HOT 7
- Getting Key Not Found Exception while Serializing to a mleap bundle HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mleap.