storm-deploy for "new generation" ec2 hardware about storm-deploy HOT 8 OPEN

This is almost certainly because the version of jclouds storm-deploy is using does not know about those. Unfortunately, I think at this point storm-deploy can't come up to the latest version of jclouds without first upgrading to pallet 0.8.0.

from storm-deploy.

Sorry to be newb about this - but what would the upgrade of pallet to 0.8.0 entail?

from storm-deploy.

A lot. Many things changed between 0.7.x and 0.8.x, so it's by no means straightforward.

from storm-deploy.

The pallet-jclouds library that wraps jclouds is compatible with both 0.7 and 0.8 of pallet, so an upgrade to pallet 0.8 is not required, although largely overdue.

Having said this, pallet-jclouds might need an update if the APIs have changed from jclouds 1.5.5.  One quick way to test is to directly upgrade the jclouds dependencies 1.6 or whichever is the jclouds version that has support for the new instances (see below)

We’ll be looking into putting out a newer version of pallet-jclouds soon.

Toni.

The libraries to add into project.clj in storm-deploy in order to force an update to jclouds are:

[org.jclouds/jclouds-all ~jclouds-ver]
[org.jclouds.driver/jclouds-sshj ~jclouds-ver]
[org.jclouds.driver/jclouds-slf4j ~jclouds-ver]
[org.jclouds/jclouds-compute ~jclouds-ver]
[org.jclouds/jclouds-blobstore ~jclouds-ver]

where ~jclouds-ver is the version of jclouds that you want.

Also, you might want to update the pallet-jclouds dependency to 1.5.3 (currently 1.5.1)

tbatchelli
Sent with Airmail

On February 13, 2014 at 9:17:33 AM, G Gordon Worley III ([email protected]) wrote:

A lot. Many things changed between 0.7.x and 0.8.x, so it's by
no means straightforward.

---

Reply to this email directly or view it on GitHub:
#57 (comment)

from storm-deploy.

Got me really excited --- however, still issues:

--- project.clj ---
(defproject storm-deploy "0.0.6-SNAPSHOT"
:source-paths ["src/clj"]
:test-paths ["test/clj"]
:profiles {:dev {:resource-paths ["conf"]}}
:aliases {"deploy-storm" ["run" "-m" "backtype.storm.provision"]}

:repositories {
"sonatype" "https://oss.sonatype.org/content/repositories/releases"
"jclouds-snapshot" "https://oss.sonatype.org/content/repositories/snapshots"
}

:dependencies [
[storm "0.5.4"]
[commons-codec "1.4"]
[org.cloudhoist/pallet "0.7.3"]
[org.cloudhoist/pallet-jclouds "1.5.3-SNAPSHOT"]
[org.cloudhoist/java "0.5.0"]
[org.cloudhoist/git "0.5.0"]
[org.cloudhoist/ssh-key "0.5.0"]
[org.cloudhoist/automated-admin-user "0.5.0"]
[org.cloudhoist/iptables "0.5.0"]
[org.cloudhoist/maven "0.5.0"]
[org.cloudhoist/zookeeper "0.5.1"]
[org.cloudhoist/nagios-config "0.5.0"]
[org.cloudhoist/crontab "0.5.0"]

             [com.jcraft/jsch.agentproxy.usocket-jna "0.0.5"]
             [com.jcraft/jsch.agentproxy.usocket-nc "0.0.5"]
             [com.jcraft/jsch.agentproxy.sshagent "0.0.5"]
             [com.jcraft/jsch.agentproxy.pageant "0.0.5"]
             [com.jcraft/jsch.agentproxy.core "0.0.5"]
             [com.jcraft/jsch.agentproxy.jsch "0.0.5"]
             [com.jcraft/jsch "0.1.49"]
             [org.jclouds.driver/jclouds-sshj "1.5.2"]
     [org.jclouds/jclouds-all "1.6.0"]
     [org.jclouds.driver/jclouds-sshj "1.6.0"]
     [org.jclouds.driver/jclouds-slf4j "1.6.0"]
     [org.jclouds/jclouds-compute "1.6.0"]
     [org.jclouds/jclouds-blobstore "1.6.0"]
             [org.jclouds.provider/aws-ec2 "1.5.2"]
             [org.jclouds.provider/aws-s3 "1.5.2"]

             [log4j/log4j "1.2.14"]
             [jvyaml "1.0.0"]]

:dev-dependencies [[swank-clojure "1.2.1"]
[org.cloudhoist/pallet-lein "0.5.2"]]
:min-lein-version "2.0.0")

Exception in thread "main" java.io.FileNotFoundException: Could not locate clojure/reflect__init.class or clojure/reflect.clj on classpath: (jclouds.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:5440)
at clojure.lang.Compiler.eval(Compiler.java:5415)
at clojure.lang.Compiler.load(Compiler.java:5857)
at clojure.lang.RT.loadResourceScript(RT.java:340)
at clojure.lang.RT.loadResourceScript(RT.java:331)
at clojure.lang.RT.load(RT.java:409)
at clojure.lang.RT.load(RT.java:381)
at clojure.core$load$fn__4511.invoke(core.clj:4905)
at clojure.core$load.doInvoke(core.clj:4904)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at clojure.core$load_one.invoke(core.clj:4729)
at clojure.core$load_lib.doInvoke(core.clj:4766)
at clojure.lang.RestFn.applyTo(RestFn.java:143)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$load_libs.doInvoke(core.clj:4800)
at clojure.lang.RestFn.applyTo(RestFn.java:138)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$require.doInvoke(core.clj:4869)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at backtype.storm.provision$eval20$loading__4410__auto____21.invoke(provision.clj:1)
at backtype.storm.provision$eval20.invoke(provision.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:5424)
at clojure.lang.Compiler.eval(Compiler.java:5415)
at clojure.lang.Compiler.load(Compiler.java:5857)
at clojure.lang.RT.loadResourceScript(RT.java:340)
at clojure.lang.RT.loadResourceScript(RT.java:331)
at clojure.lang.RT.load(RT.java:409)
at clojure.lang.RT.load(RT.java:381)
at clojure.core$load$fn__4511.invoke(core.clj:4905)
at clojure.core$load.doInvoke(core.clj:4904)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at clojure.core$load_one.invoke(core.clj:4729)
at clojure.core$load_lib.doInvoke(core.clj:4766)
at clojure.lang.RestFn.applyTo(RestFn.java:143)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$load_libs.doInvoke(core.clj:4800)
at clojure.lang.RestFn.applyTo(RestFn.java:138)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$require.doInvoke(core.clj:4869)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at user$eval5$fn__7.invoke(form-init257033639388524929.clj:1)
at user$eval5.invoke(form-init257033639388524929.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:5424)
at clojure.lang.Compiler.eval(Compiler.java:5415)
at clojure.lang.Compiler.load(Compiler.java:5857)
at clojure.lang.Compiler.loadFile(Compiler.java:5820)
at clojure.main$load_script.invoke(main.clj:221)
at clojure.main$init_opt.invoke(main.clj:226)
at clojure.main$initialize.invoke(main.clj:254)
at clojure.main$null_opt.invoke(main.clj:279)
at clojure.main$main.doInvoke(main.clj:354)
at clojure.lang.RestFn.invoke(RestFn.java:422)
at clojure.lang.Var.invoke(Var.java:369)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.Var.applyTo(Var.java:482)
at clojure.main.main(main.java:37)
Caused by: java.io.FileNotFoundException: Could not locate clojure/reflect__init.class or clojure/reflect.clj on classpath:
at clojure.lang.RT.load(RT.java:412)
at clojure.lang.RT.load(RT.java:381)
at clojure.core$load$fn__4511.invoke(core.clj:4905)
at clojure.core$load.doInvoke(core.clj:4904)
at clojure.lang.RestFn.invoke(RestFn.java:409)
at clojure.core$load_one.invoke(core.clj:4729)
at clojure.core$load_lib.doInvoke(core.clj:4766)
at clojure.lang.RestFn.applyTo(RestFn.java:143)
at clojure.core$apply.invoke(core.clj:542)
at clojure.core$load_libs.doInvoke(core.clj:4800)
at clojure.lang.RestFn.applyTo(RestFn.java:138)
at clojure.core$apply.invoke(core.clj:544)
at clojure.core$use.doInvoke(core.clj:4880)
at clojure.lang.RestFn.invoke(RestFn.java:458)
at pallet.compute.jclouds$eval5815$loading__4410__auto____5816.invoke(jclouds.clj:1)
at pallet.compute.jclouds$eval5815.invoke(jclouds.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:5424)
... 55 more

from storm-deploy.

I was only able to go as high as pallet-jclouds 1.5.1 and jclouds 1.5.2 when I was actively working on my storm-deploy fork. Haven't revisited since then.

from storm-deploy.

So even with pallet-jclouds 1.5.1 and jclouds 1.5.2 it doesn't work for me...

from storm-deploy.

I have pushed a branch 'update-clojure-jclouds` that uses the latest jclouds version (1.7.1), and this should allow using all the newer hardware in AWS. Give it a try and report back if you find any issues so I can proceed to merge it into master.

from storm-deploy.