typesafehub / sbt-conductr Goto Github PK

The following output displays for a bundle with the id of 116aa4f196f3ef2b8970b4a4166d0108-a802635856ee251147550871a5f88b46:

ID/BUNDLE/CONF             NAME                          WHERE                   #REP   #STR   #RUN     #CPU     MEM   FSIZE ROLES            
116aa4f/116aa4f/           akkaclusterback-1.0.0         127.0.0.1:9004             1      1      0      1.0   67 MB    5 MB                  

i.e. the 116aa4f is displayed twice.

It's a bit of a let down that loadBundle does autocomplete and startBundle doesn't (understandably, start needs to call conductr via network), but it still would be very nice to have completion there - better than copy pasting hashes.

s/conductr/conduct

Bundle info should provide a #Start column in addition to #Run that distinguishes the number of bundles starting from the number that are running. At present #Run reports a bundle as running even when it is starting.

Did not investigate much but got:

> conductr:info
[trace] Stack trace suppressed: run last conductr:info for the full output.
[error] (conductr:info) java.util.concurrent.TimeoutException: Futures timed out after [10 seconds]
[error] Total time: 10 s, completed 27-Mar-2015 15:27:56

> conductr:set  192.168.59.103
[success] Total time: 0 s, completed 27-Mar-2015 15:28:13


> conductr:info
[trace] Stack trace suppressed: run last conductr:info for the full output.
[error] (conductr:info) akka.actor.InvalidActorNameException: actor name [screen] is not unique!
[error] Total time: 0 s, completed 27-Mar-2015 15:28:14

The command conduct info interferes with the next terminal line:

> conduct info
[success] Total time: 0 s, completed Sep 11, 2015 9:22:21 AMREP   #STR   #RUN
> 5172d     visualizer                 1      0      1
1fecf0e     conductr-elasticsearch     1      0      1
6a043df     conductr-kibana            1      0      1

Instead, first the entire content of conduct info should be displayed as output and afterwards the pointer for the next terminal commands should be displayed.

For example: It should be easy for a developer who issues startBundle to see it running without exiting sbt. This would be nicest to provide as bridge tasks like conduct:info which call through to the conduct CLI I think.

// Take this as feedback from "first time using this stuff developer", probably not high priority but nevertheless would make the offering feel complete / polished.

Incorrect reports how to start a bundle after having loaded a bundle:

[info] Upload completed. Use 'startBundle {"requestId":"f9493fe0-efd2-427d-9e6c-3cbca6969d1c","bundleId":"0a1a79a41c82c83c9c8edc67d18a7527"}' to start.

For consistency with the CLI, and to avoid namespace clash issues with other sbt plugins, we need a single conduct command along with load, run etc as sub commands. One problem with the existing conductr configuration approach is that we cannot have a run command in place of start. run is already defined by sbt and has a different meaning. By moving to sub commands then we may have conduct run and avoid these style of issues.

Let's also have an ip and port sub command in place of controlServer (we should similarly have an ip sub command for the python CLI that sets up the env var accordingly).

If we can move to a situation where sbt-conductr may kick off a built-in, single node version of ConductR then it'd be great to be able to provide a task that allows the developer to verify their services. Here's a draft requirement for this:

As a Developer I wish to be able to verify the endpoints that I have configured so that I can be assured they are valid before handing over to ops.

I'm imagining an sbt sub-task as an extension of our existing conduct task such as per the following usage:

> conduct verify http://:8080
200 OK
> conduct verify http://:9999/customers
200 OK
> conduct verify http://conductr.typesafe.com
200 OK
> conduct verify tcp://:2552
Port 2552 is listening

The verify command would use the sbt-core-next project to manage a Docker based instance of ConductR (single-node flavour only) configured with HAProxy and possibly even our bundling of Elastic Search, rsyslog etc. On the latter point you may then even use the events and logs commands that we're going to implement to diagnose startup issues etc.

I upgraded from 0.25.0, cleaned etc.

this is running on a vagrant conductr cluster that is known to work with sbt-typesafe-conductr 0.25.0
(ie, the control server is accessible etc.)

with the latest (0.27.0) I run:

conductr:controlServer 192.168.77.20:9005
[success] Total time: 0 s, completed 8-Apr-2015 9:58:41 AM

bundle:dist
...
[info] Packaging /Users/dave/projects/akka/conductrR-examples/singlemicro/target/scala-2.11/singlemicro_2.11-1.0.0.jar ...
[info] Done packaging.
[success] Total time: 9 s, completed 8-Apr-2015 10:01:29 AM

conductr:load file:/Users/dave/projects/akka/conductrR-examples/singlemicro/target/typesafe-conductr/singlemicro-1.0.0-aceb2bbfa176b1b1b00794d03ebe483f49adb744e073d1668d2561ae467319f4.zip
[info] Loading bundle to ConductR ...
[trace] Stack trace suppressed: run last singlemicro/conductr:load for the full output.
error Problem loading the bundle: empty stream
[error] Total time: 1 s, completed 8-Apr-2015 10:01:45 AM

last singlemicro/conductr:load
java.lang.RuntimeException: Problem loading the bundle: empty stream
at scala.sys.package$.error(package.scala:27)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22$$anonfun$com$typesafe$conductr$sbt$SbtTypesafeConductR$$anonfun$$anonfun$$anonfun$$loadBundle$1$1.apply(SbtTypesafeConductR.scala:170)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22$$anonfun$com$typesafe$conductr$sbt$SbtTypesafeConductR$$anonfun$$anonfun$$anonfun$$loadBundle$1$1.apply(SbtTypesafeConductR.scala:146)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$.com$typesafe$conductr$sbt$SbtTypesafeConductR$$withConductRController(SbtTypesafeConductR.scala:309)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22.com$typesafe$conductr$sbt$SbtTypesafeConductR$$anonfun$$anonfun$$anonfun$$loadBundle$1(SbtTypesafeConductR.scala:146)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22$$anonfun$apply$25.apply(SbtTypesafeConductR.scala:174)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22$$anonfun$apply$25.apply(SbtTypesafeConductR.scala:174)
at scala.Function3$$anonfun$curried$1$$anonfun$apply$1$$anonfun$apply$2.apply(Function3.scala:26)
at org.scalactic.Accumulation$class.withGoodCurried(Accumulation.scala:411)
at org.scalactic.Accumulation$class.withGood(Accumulation.scala:398)
at org.scalactic.Accumulation$.withGood(Accumulation.scala:1628)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22.apply(SbtTypesafeConductR.scala:174)
at com.typesafe.conductr.sbt.SbtTypesafeConductR$$anonfun$loadBundleTask$1$$anonfun$apply$21$$anonfun$apply$22.apply(SbtTypesafeConductR.scala:140)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
error Problem loading the bundle: empty stream

controlServer is too unspecific.

When entering the conduct command the output is:

> conduct
Usage: conduct <subtask>
[success] Total time: 0 s, completed Oct 20, 2015 11:13:19 AM

This is output is not really helpful for the user. Instead we should list down all available sub commands (similar to conduct help).

this project seems to server a Conductor server and the a sbt plugin.then the project which using the plugin could deploy their reactive application to the Conductor via upload with a bundle id?

and then them can fetch and run the bundle when them want to deploy the bundle to the production via bundle id?

I just git clone and play with it,but could not found how to run a Conductor

It appears that the http "ambiguous" status code is not handled properly:

[error] (*:conduct) com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Specified': was expecting ('true', 'false' or 'null')
[error]  at [Source: java.io.StringReader@2ec32cae; line: 1, column: 10]

This should be explained or the DSL improved.

The 0 there is a magic number there. It's not explained in the quickstart why we're using a weird port number (it is explained in one sentence once you dig through https://github.com/sbt/sbt-bundle). When I see something like "port -> port" instincts from other tools kick in and I say "oh yeah 9000 -> 9000", which is wrong.

It's typed DSL so it could be made something meaningful instead of 0. With conversions from int to this meaningful type. As in Endpoint(DynamicallyAllocated, ...)

For all of our public sbt plugins we are using com.typesafe.sbt.

Update doco and sbt-bundle dependency once sbt/sbt-bundle#20 is merged

The conduct command with that all the options (load, unload, run, stop, info, logs, events) should work globally from the root level across all sub projects.

The error message for the following conduct commands need to be improved:

• logs
• events
• services
• info

So far these commands are timing out after 5 seconds if ConductR does not respond, e.g. if ConductR has not been started. Especially as a first user experience this is particularly bad. Instead the failure case should return with a concrete error message.

This issue should be fixed while implementing the new HTTP library for sbt-conductr.

If there's a project/typesafe.properties file present and image is conductr/conductr-dev then set the imageVersion to the latest version available (1.0.11).

The following accpetance tests need to be added:

• conduct logs
• conduct events
• conduct load
  • without an additional configuration zip file
  • with an additional configuration zip file

The bundle name and other scheduling params for the multipart form data should be derived from the bundle.conf + optional bundle.conf values as per the CLI. Right now, these params are derived from sbt settings. This is fine until you get multiple configurations, each overriding the bundle's name as per reactive-maps (there are three types of bundle derived from the one project, hence three names get overridden).

The link to Reactive Runtime in README.md is a 404

"Upload completed. Use 'startBundle 858d8e0c3215ece38318355d17f25766' to start."

should be:

"Upload completed. Use 'conduct start 858d8e0c3215ece38318355d17f25766' to start."

See https://github.com/typesafehub/conductr/issues/590.

Discussed impl is - Console.bundleInfo:

While this may seem weird, but I think it would be nicer to "smartly block", as in - perform the request in the background, but not return from the method while we're waiting for a response. This would allow (if we want to) for "oh well, nevermind", and aborting waiting for the response, and droping back to the sbt shell (we'd pipe the response to nothingness then (or not)).

Reason I'm pointing this out is, that it looks weird to get the result in the middle of a new shell line:

> conductr:info
[success] Total time: 0 s, completed 27-Mar-2015 15:49:41WHERE                   #REP   #STR   #RUN     #CPU     MEM   FSIZE ROLES                                                                                                                                                                                                                                           > 98535/4998535/           reactive-maps-1.0-SNAPSHOT    172.17.0.8:9004            3      0      1      1.0   67 MB    5 MB                                                                                                                                                                                                                                                                                                          172.17.0.7:9004                                                                                                                                                                                                                                                                                                                                                              172.17.0.6:9004

Looks weird,
I'd rather have:

> conductr:info
WHERE                   #REP   #STR   #RUN     #CPU     MEM   FSIZE ROLES                                                                                                                                                                                                                                            98535/4998535/           reactive-maps-1.0-SNAPSHOT    172.17.0.8:9004            3      0      1      1.0   67 MB    5 MB                                                                                                                                                                                                                                                                                                          172.17.0.7:9004                                                                                                                                                                                                                                                                                                                                                              172.17.0.6:9004

[success] Total time: 0 s, completed 27-Mar-2015 15:49:41
>

even though it's "blocking"

• when there is no conductr
• when conductr stops while the bundleInfo is running

Hi,

I just added the 1.0.1 version of the plugin to my project and this is the result.

[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.typesafe.akka#akka-contrib-extra_2.10;1.18.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Unresolved dependencies path:
[warn]      com.typesafe.akka:akka-contrib-extra_2.10:1.18.0
[warn]        +- com.typesafe.conductr:sbt-conductr:1.0.1 (sbtVersion=0.13, scalaVersion=2.10) (/Users/bjorn/Source/typesafe/cinnamon/master-2/project/plugins.sbt#L9-10)
[warn]        +- default:master-2-build:0.1-SNAPSHOT (sbtVersion=0.13, scalaVersion=2.10)

I needed to add the akka-contrib-extra resolver, but it wasn't mentioned anywhere.

As described in http://eed3si9n.com/testing-sbt-plugins , which seem to still be up to date for 0.13.x

1. Add the scripted plugin
2. Write a test that validates > controlServer actually changes State

Unless I'm mistaken, I think this requires the dev being able to publish locally. I wasn't success in my initial attempts at this, will update this issue with more info

For consistency we should consider consolidating command into a single conduct command with 'info', 'load' etc as params - just like the Python CLI.

It's currently rather confusing how to "load bundle with my config", the nearest thing to it docs wise is the section on shazar here: http://typesafehub.github.io/typesafe-conductr/intro/Quickstart.html

One use case is: Apps sometimes have modes, or demo-apps we prepare as activator templates often have frontend and backend roles (including reactive maps, akkamazing etc). So still while developing the app I want to:

• start bundleX with backend configuration on backend nodes
• start bundleX with frontend configuration on frontend nodes.

This is different than ops overlaying their configuration (with for example prod keys / urls etc). I have the full config in my development sources, and it's committed. I'd advocate for supporting "environment configuration", like in chef: https://docs.chef.io/environments.html so we'd have src/resources/conductr/env.conf and src/resources/conductr/end_frontend.conf etc.

• I want to be able to pick which env config to shazar and include when loading. It would be nice to have conventions in place, so not every team has it completely differently. Thus the above proposal with conductr/env[name].conf (or it could be .sh). I'd personally like it to be usable like so: load -E frontend mybundle.
• I want to have this name exposed as env variable, so my app can act upon it. We do have activator templates that notice "oh, I'm a frontend node now, okey!".

Add scripted tests for all subtasks:
load,run,stop,unload.

This could be enabled with a (task? test plugin?) in the sbt-test build.sbt that mocks a conductr server.

in test:

controlServer 127.0.0.1:9005

startConductRMock 127.0.0.1:9005

conduct load file:///......

checkLoad

...

I'd like to be able to say: conductr 192.168.59.103,
conductr already knows it's http by default, and knows it's default port (9005).

Since it's a special task, and not set conductrUrl = new URL(...) we should be able to ease the users live with this.

Steps to reproduce:

cd into sbt-conductr-tester sub project

changes build.properties to:

sbt.version = 0.13.7

try to load a bundle to a conductr server:

java.lang.RuntimeException: Settings logger used after project was loaded.
at scala.sys.package$.error(package.scala:27)
at sbt.LogManager$$anon$1$$anonfun$slog$1.apply(LogManager.scala:110)
at sbt.LogManager$$anon$1$$anonfun$slog$1.apply(LogManager.scala:110)
at scala.Option.getOrElse(Option.scala:120)
at sbt.LogManager$$anon$1.slog(LogManager.scala:110)
at sbt.LogManager$$anon$1.log(LogManager.scala:115)
at sbt.Logger$class.info(Logger.scala:117)
at sbt.LogManager$$anon$1.info(LogManager.scala:108)
at com.typesafe.conductr.sbt.ConductR$$anonfun$com$typesafe$conductr$sbt$ConductR$$doLoadBundle$1$1.apply(ConductR.scala:39)
at com.typesafe.conductr.sbt.ConductR$$anonfun$com$typesafe$conductr$sbt$ConductR$$doLoadBundle$1$1.apply(ConductR.scala:38)

This is due to using sLog as the logger. It looks like Josh fixed this in 0.13.8

However: wouldn't it be better to just use

state.log.info("yadda")

instead of passing this logger through?

I've tested this and it works. I can create a PR tomorrow 9AM (-4 UTC)

Continuation of #92

The CLI accepts a --scale option to run.
sbt-conductr should also provide the scale option for consistency with the CLI.

The Conduct CLI uses CONDUCTR_IP:CONDUCTR_PORT as default values instead of assuming localhost.
To be consistent among the projects the sbt plugin should do so as well.

Here's the spot: 62723f2#commitcomment-10347447

Given the branch from edwardcallahan/conductr-service-lookup#4, I get the following error when using conduct info from the root project.

[error] (charonShopApp/*:conduct) akka.actor.InvalidActorNameException: actor name [screen] is not unique!