Giter VIP home page Giter VIP logo

jruby-gradle-storm-plugin's People

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

jruby-gradle-storm-plugin's Issues

jrubyStormLocal does not set classpath properly unless created by parent

these lines cause the classpath for the JRubyStormLocal task type to only have their classpath set properly if a parentTask has been provided.

In the case of a stand-alone task definition (i.e. not auto-created by jrubyStorm) this will result in an missing classpath and:

:runMyFancyTopology
Error: Could not find or load main class redstorm.TopologyLauncher
:runMyFancyTopology FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':runMyFancyTopology'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

I should be able to disable default repositories

Perhaps we should just inherit the jruby.defaultRepositories setting from the base plugin.

Either way, forcing me to use jcenter and clojars causes my builds to fail if any of those third party providers has issues.

Cannot use 0.1.6 with the newer releases of the base/jar plugin

* Where:
Build file '/home/tyler/source/lookout/REDACTED/build.gradle' line: 21

* What went wrong:
A problem occurred evaluating root project 'REDACTED'.
> Failed to apply plugin [id 'com.github.jruby-gradle.storm']
   > Could not find method jruby() for arguments [com.github.jrubygradle.storm.JRubyStormPlugin$_apply_closure2_closure8@616a00ab] on task ':jrubyStorm'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 4.408 secs

JRUbyStormLocal tasks use the jrubyExec configuration by default

The configuration management code in JRubyStormLocal needs to be retooled a bit and better tested to make sure that:

  • if we have a parent task, we use their configuration + "Local"
  • if we have no parent task we use jrubyStormLocal by default (which should extend jrubyStorm)

prepareAssembleJRubyStorm is not caching results

In my local testing, when I'm assembling the topology over and over it seems to be installing the gems over and over again.

It's not clear whether this is a storm plugin bug or a bug with the JRubyPrepare task

Gradle 2.2 cannot call super.getJrubyVersion() properly

Working with the pre-release version of 0.4.0, and this line doesn't seem to want to cooperate

@ysb33r this ring any bells? :/

Caused by: groovy.lang.MissingMethodException: No signature of method: com.github.jrubygradle.storm.internal.JRubyStormJar_Decorated.getJrubyVersion() is applicable for argument types: () values: []
Possible solutions: getJrubyVersion(), getJrubyVersion(), getJrubyVersion(), jrubyVersion(java.lang.String), getVersion(), getJrubyMainsVersion()
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:130)
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148)
        at com.github.jrubygradle.storm.internal.JRubyStormJar.getJrubyVersion(JRubyStormJar.groovy:24)
        at com.github.jrubygradle.storm.internal.JRubyStormJar_Decorated.getJrubyVersion(Unknown Source)
        at com.github.jrubygradle.jar.JRubyJar.addJRubyDependency(JRubyJar.groovy:209)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

New artifact layout has issues with classloading from embedded jars on supervisors

In an older Storm cluster, where storm-kafka isn't already baked in, I'm having trouble getting a topology which includes the storm-kafka jar to start properly on supervisor nodes.

The classpath and everything is correct enough to submit and load successfully (on-submit does resolve and make use of storm-kafka).

java.lang.RuntimeException: java.lang.ClassNotFoundException: storm.kafka.KafkaSpout
        at backtype.storm.utils.Utils.deserialize(Utils.java:95) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.utils.Utils.getSetComponentObject(Utils.java:235) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.task$get_task_object.invoke(task.clj:73) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.task$mk_task_data$fn__3061.invoke(task.clj:180) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.util$assoc_apply_self.invoke(util.clj:816) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.task$mk_task_data.invoke(task.clj:173) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.task$mk_task.invoke(task.clj:184) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.executor$mk_executor$fn__5510.invoke(executor.clj:321) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.core$map$fn__4207.invoke(core.clj:2485) ~[clojure-1.5.1.jar:na]
        at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
        at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
        at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
        at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
        at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.5.1.jar:na]
        at clojure.core.protocols$fn__6026.invoke(protocols.clj:54) ~[clojure-1.5.1.jar:na]
        at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13) ~[clojure-1.5.1.jar:na]
        at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na]
        at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na]
        at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:321) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at backtype.storm.daemon.worker$fn__5940$exec_fn__1396__auto____5941$iter__5946__5950$fn__5951.invoke(worker.clj:375) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
        at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
        at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.5.1.jar:na]
        at clojure.lang.RT.next(RT.java:598) ~[clojure-1.5.1.jar:na]
        at clojure.core$next.invoke(core.clj:64) ~[clojure-1.5.1.jar:na]
        at clojure.core$dorun.invoke(core.clj:2781) ~[clojure-1.5.1.jar:na]
        at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
        at backtype.storm.daemon.worker$fn__5940$exec_fn__1396__auto____5941.invoke(worker.clj:375) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.lang.AFn.applyToHelper(AFn.java:185) [clojure-1.5.1.jar:na]
        at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
        at clojure.core$apply.invoke(core.clj:617) ~[clojure-1.5.1.jar:na]
        at backtype.storm.daemon.worker$fn__5940$mk_worker__5996.doInvoke(worker.clj:347) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.5.1.jar:na]
        at backtype.storm.daemon.worker$_main.invoke(worker.clj:454) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        at clojure.lang.AFn.applyToHelper(AFn.java:172) [clojure-1.5.1.jar:na]
        at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
        at backtype.storm.daemon.worker.main(Unknown Source) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
Caused by: java.lang.ClassNotFoundException: storm.kafka.KafkaSpout
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_75]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_75]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_75]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_75]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_75]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) ~[na:1.7.0_75]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_75]
        at java.lang.Class.forName0(Native Method) ~[na:1.7.0_75]
        at java.lang.Class.forName(Class.java:274) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:625) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) ~[na:1.7.0_75]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) ~[na:1.7.0_75]
        at backtype.storm.utils.Utils.deserialize(Utils.java:89) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
        ... 36 common frames omitted

NPE when running a custom JRubyStormLocal task

task runLocalTopology(type: JRubyStormLocal) {
    description 'Run the Topology in local-topology mode'
    topology 'topologies/deferred_url_topology'
    standardInput System.in
}

Results in:

:runLocalTopology FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':runLocalTopology'.
> java.lang.NullPointerException (no error message)

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':runLocalTopology'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:306)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
        at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
        at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
        at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:169)
        at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
        at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
        at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
        at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
        at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
        at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
        at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
        at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
        at org.gradle.launcher.Main.doAction(Main.java:33)
        at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
        at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
        at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
        at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
        at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:56)
Caused by: java.lang.NullPointerException
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.createGroovyObjectGetPropertySite(AbstractCallSite.java:254)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.acceptGroovyObjectGetProperty(AbstractCallSite.java:239)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:231)
        at com.github.jrubygradle.storm.JRubyStormLocal.exec(JRubyStormLocal.groovy:39)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
        ... 55 more

pack jar from jar-dependencies as packed jars

it kind of difficult to load some jars via jar-dependencies and some other jars should not be loaded.

jruby comes with couple of jars with its default gems: jline-2.11.jar, those bouncy-castle jars. jruby-9k will come with snakeyaml-1.14.jar and maybe one or more other jars soon.

AND there is no way to find out which jars jruby bundles with those default gems (any ideas on how to expose those extra dependencies formally are welcome).

now jar-dependencies loads those jars. it will look on "classpath" or "classloader" first.

with maven I just use http://maven.apache.org/plugins/maven-dependency-plugin/copy-dependencies-mojo.html, set useRepositoryLayout=true and outputDirectory=target/classes and the artifact will have those jars where jar-dependencies is going to look for them.

Setting jrubyVersion for embedding in the topology jar gets overwritten

adding the dependency jruby-complete 1.7.22 to jar
unzipping /home/tyler/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.22/42dc9be2f706774c24ef61ec2981d05e2c79a9e2/jruby-complete-1.7.22.jar in the jar
unzipping /home/tyler/.gradle/caches/modules-2/files-2.1/org.jruby.mains/jruby-mains/0.4.0/c767732c802c52592a949b57e68023e66b8b860e/jruby-mains-0.4.0.jar in the jar
task ':jrubyJar' including files in /home/tyler/source/lookout/REDACTED/build/dirinfo/jrubyJar
adding the dependency jruby-complete 1.7.21 to jar
unzipping /home/tyler/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/1.7.21/e061b9f399a5e8d5cfcca84d4a6baf879111e83c/jruby-complete-1.7.21.jar in the jar
unzipping /home/tyler/.gradle/caches/modules-2/files-2.1/org.jruby.mains/jruby-mains/0.4.0/c767732c802c52592a949b57e68023e66b8b860e/jruby-mains-0.4.0.jar in the jar
task ':assembleJRubyStorm' including files in /home/tyler/source/lookout/REDACTED/build/dirinfo/jrubyJar

Looks like my attempt to set a default version to 1.7.21 as a baseline didn't work :(

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.