riotgamesminions / motherbrain Goto Github PK

View Code? Open in Web Editor NEW

119.0 119.0 18.0 8.71 MB

An orchestrator for Chef

License: Other

Shell 0.04% Ruby 98.09% Gherkin 1.87%

motherbrain's People

Contributors

Stargazers

Watchers

Forkers

bscott justincampbell awesome telekomlabs rallysoftware-cookbooks velankanisys reset dldinternet dodwyer winmillwill viglesiasce peoplemerge nciaccio mbrukman quadespresso samuelchen ravenb garycarneiro

motherbrain's Issues

Bootstrap happening sequentially

Sometime in the past 2 days or so, a change has caused my bootstrap of 2 nodes to happen sequentially instead of concurrently

Jobs should persist through server restart

The JobManager stores a registry of all active Jobs and a JobRecord for each.

In case of a crash; all active Jobs should update their JobRecord
All JobRecords should save to a persistant store after each update

Simplicity is key with this feature. Let's not use MySQL or PostgreSQL if we don't need to. Remember that MotherBrain will be distributed and there will only be one JobManager. This means a SQLite database to disk may be sufficient.

Keep JRuby support in mind.

provision failing with Provisioner::Manager crashed

I ran the following command to attempt and spin up a couple nodes with some recipes for bacon. This env does not represent a full bacon environment, but I just wanted to see if it would spin nodes up and apply the given recipes.

I ran the following and the command just hung:
bundle exec bin/mb bacon provision BA1 test-env.json -d -v -L ./mb.log

My bacon plugin looks like: https://gh.riotgames.com/gist/614
My test-env.json manifest is: https://gh.riotgames.com/gist/619

The output to mb.log is:

[2013-01-04T21:12:49Z] PID[49401] TID[zvthto] INFO: MotherBrain starting...
[2013-01-04T21:12:49Z] PID[49401] TID[zvu2aw] INFO: Config Manager starting...
[2013-01-04T21:12:49Z] PID[49401] TID[znul2o] INFO: Plugin Manager starting...
[2013-01-04T21:12:49Z] PID[49401] TID[zvnctk] INFO: Provision Manager starting...
[2013-01-04T21:12:49Z] PID[49401] TID[zvp2lg] INFO: Bootstrap Manager starting...
[2013-01-04T21:12:49Z] PID[49401] TID[zo62bc] INFO: Node Querier starting...
[2013-01-04T21:12:49Z] PID[49401] TID[ztdzvw] INFO: Lock Manager starting...
[2013-01-04T21:12:50Z] PID[49401] TID[zomyk0] INFO: Ridley starting...
[2013-01-04T21:12:50Z] PID[49401] TID[zvnctk] ERROR: MotherBrain::Provisioner::Manager crashed!
Celluloid::FiberStackError: stack level too deep (please see https://github.com/celluloid/celluloid/wiki/Fiber-stack-errors)
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/tasks/task_fiber.rb:49:in `rescue in resume'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/tasks/task_fiber.rb:46:in `resume'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/responses.rb:11:in `dispatch'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/actor.rb:329:in `handle_message'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/actor.rb:196:in `run'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/actor.rb:184:in `block in initialize'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/thread_handle.rb:17:in `block in initialize'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/internal_pool.rb:48:in `call'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/internal_pool.rb:48:in `block in create'
[2013-01-04T21:12:50Z] PID[49401] TID[zl7ux4] DEBUG: Terminating 15 actors...
[2013-01-04T21:12:50Z] PID[49401] TID[zvnctk] INFO: Provision Manager stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[zvu2aw] INFO: Config Manager stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[znul2o] INFO: Plugin Manager stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[zvthto] ERROR: MotherBrain::Application crashed!
RuntimeError: a group member went missing. This shouldn't be!
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/supervision_group.rb:99:in `restart_actor'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/actor.rb:355:in `handle_exit_event'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/actor.rb:340:in `block in handle_system_event'
/Users/deilering/.rvm/gems/ruby-1.9.3-p286/gems/celluloid-0.12.4/lib/celluloid/tasks/task_fiber.rb:24:in `block in initialize'
[2013-01-04T21:12:50Z] PID[49401] TID[zvp2lg] INFO: Bootstrap Manager stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[zo62bc] INFO: Node Querier stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[zvthto] INFO: MotherBrain stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[ztdzvw] INFO: Lock Manager stopping...
[2013-01-04T21:12:50Z] PID[49401] TID[zomyk0] INFO: Ridley stopping...
[2013-01-04T21:12:56Z] PID[49401] TID[zl7ux4] INFO: Shutting down...

I had to ctrl-c the command line command.

After chef run hook

We need a way to specify an after chef run hook that will allow us to set attributes, etc.

bin/mb -v / -d should output to STDOUT

running bin/mb with the -d or -v flags should redirect log output to STDOUT instead of the default logfile.

MotherBrain needs to work on MRI and JRuby

We should by default run our tests using MRI so we have guard, spork, and the other tools in the usual testing environment for our gems.

To achieve this we should add an instance function to the Plugin class that returns what "platforms" it can be run on. This function will currently only check if a JMX action is being taken place. If this is the case then the supported platforms would only include JRuby and not MRI.

We should then raise early if something is invoked within an unsupported environment. For example: trying to invoke a command on a plugin which leverages JMX but running on MRI would cause a failure.

The specs for the JMX action should only run when the current platform is JRuby.

We should ensure that we test MotherBrain on Travis or another CI with multiple paltforms including MRI and JRuby.

Motherbrain should have environment-wide locks

A lock should be required for altering state
Locks should be stored in databags
A lock should be freed when the operation is complete or fails
A client should error out if a requested lock is unavailable
The error should contain the timestamp, user, and action of the lock
A blocked client should be able to forcibly override a lock

JMX Gear

We need a JMX gear to be able to make JMX calls.

Bootstrap should accept and set environment attributes

During the bootstrap process we should be able to provide a hash of environment attributes to be set on the environment being bootstrapped

Windows support in NodeQuerier and Bootstrap

The NodeQuerier and any Bootstrap specific commands from Ridley assume the target node is running Linux. We need to add Windows support to these portions of MotherBrain.

This should be accomplished by extending the NodeQuerier and Bootstrap steps to identify if the target host is a Windows machine and using WinRM / Batch instead of SSH / Bash.

⚠️ the transport portion (WinRM) may be invalidated by Opscode's "Pushy" transport protocol ⚠️

Components and commands are listed sorted together

I would prefer to have all components listed together.

Currently:

$ mb first_app
Tasks:
  mb first_app a_server [COMMAND]  # a server
  mb first_app help [COMMAND]      # Describe subcommands or one specific subcommand
  mb first_app nodes ENVIRONMENT   # List all nodes grouped by Component and Group
  mb first_app version             # Display plugin version
  mb first_app z_server [COMMAND]  # z server

Preferred:

$ mb first_app
Tasks:
  mb first_app help [COMMAND]      # Describe subcommands or one specific subcommand
  mb first_app nodes ENVIRONMENT   # List all nodes grouped by Component and Group
  mb first_app version             # Display plugin version
Components:
  mb first_app a_server [COMMAND]  # a server
  mb first_app z_server [COMMAND]  # z server

Ridley errors out when the locks data bag doesn't exist

[2013-01-16T20:47:17Z] PID[55307] TID[ouzl2xzpk] INFO: Locking chef_environment:vagrant
[2013-01-16T20:47:18Z] PID[55307] TID[ouzl2nttw] ERROR: Ridley::Connection crashed!
Ridley::Errors::HTTPNotFound: errors: 'Cannot load data bag item chef_environment-vagrant for data bag _motherbrain_locks_'
/Users/Justin/.gem/ruby/1.9.3/bundler/gems/ridley-c1d5564a090a/lib/ridley/middleware/chef_response.rb:23:in `on_complete'
/Users/Justin/.gem/ruby/1.9.3/gems/faraday-0.8.4/lib/faraday/response.rb:9:in `block in call'
/Users/Justin/.gem/ruby/1.9.3/gems/faraday-0.8.4/lib/faraday/response.rb:63:in `on_complete'
/Users/Justin/.gem/ruby/1.9.3/gems/faraday-0.8.4/lib/faraday/response.rb:8:in `call'
/Users/Justin/.gem/ruby/1.9.3/bundler/gems/ridley-c1d5564a090a/lib/ridley/middleware/chef_auth.rb:34:in `call'
/Users/Justin/.gem/ruby/1.9.3/gems/faraday-0.8.4/lib/faraday/connection.rb:226:in `run_request'
/Users/Justin/.gem/ruby/1.9.3/gems/faraday-0.8.4/lib/faraday/connection.rb:87:in `get'
/Users/Justin/.gem/ruby/1.9.3/gems/celluloid-0.12.4/lib/celluloid/calls.rb:23:in `public_send'
/Users/Justin/.gem/ruby/1.9.3/gems/celluloid-0.12.4/lib/celluloid/calls.rb:23:in `dispatch'
/Users/Justin/.gem/ruby/1.9.3/gems/celluloid-0.12.4/lib/celluloid/actor.rb:327:in `block in handle_message'
/Users/Justin/.gem/ruby/1.9.3/gems/celluloid-0.12.4/lib/celluloid/tasks/task_fiber.rb:24:in `block in initialize'

need to put encrypted data bag on the box even if partial bootstrap

Depend on other Plugin

We need a way to depend on another plugin. Dependencies should automatically be "required" in.

name "pvpnet"

require_plugin "activemq", "= 2.0.0"

Logging to file not working on JRuby

As discovered by @bluepojo, this is because Logger in JRuby doesn't take Pathname objects.

d243b37

plugin generator

shorthand syntax for commands

In the 80% case, it seems like most commands will be running actions (with the same name as the command, most likely) on a set of services, or invoking the same named command on several components.

I'd like to propose an alternate shorthand syntax for those execute blocks:

command "start"
service "database", :start
component "widgets", :start
component "flanges", :start
end

OR

command "start", :services => "database", :components => "widgets,flanges"

Fix broken Plugin Invoker specs

plugin invoker specs are failing after merge of a6be4dc

Require other plugin's bootstrap routine into another

cluster_bootstrap do
  # including another one
  routine("activemq")

  # our standard bootstrap routine
  bootstrap("pvpnet::default")
end

Should catch evaluation errors in plugin files

I had a mistaken require in a plugin file in ~/.mb/plugins. Mb should catch the error and print something friendly rather than a cannot load such file stack trace.

Bootstrap should accept and create or update data bags

The cluster bootstrap process should accept and create or update data bags to be generated during the bootstrap process

REST API Authentication

A token based authentication system will need to be created to protect the MotherBrain REST API. Let's start simple by only having one 'super user' and a token for it. A more complete users system can be added at a later date.

if i ctrl+c a motherbrain command it should release the lock for that environment

bootstrapper assumes that Chef/Ruby have already been installed via Omnibus

When I tried to bootstrap a node that hasn't checked into the chef server yet it failed because it doesn't have embedded ruby on it.

https://gh.riotgames.com/gist/647

Short term fix

Require Chef/Ruby to be installed by omnibus on all nodes that bootstrap is attempted to be run on. This does not include nodes which we provisioned with motherbrain since environment factory installs Chef/Ruby with omnibus for us before handing the node back.

Long term fix

Update the bootstrapper to properly inspect the state of the node. If Chef/Ruby are not installed in the locations we expect then we need to perform an omnibus installation.

a combination provision / bootstrap creates an extra (unused) Job

a new Job should not be created if a provision is executed with :skip_bootstrap set to false (the default).

https://github.com/RiotGames/motherbrain/blob/master/lib/mb/provisioners/environment_factory.rb#L120-L130

A new orphan Job is created for the Boostrap process but never used or updated.

if a bootstrap step fails don't continue the bootstrap

Chef run failure
Cannot connect to a node

create MotherBrain Chef Handler for marshaling Logs to Log Server

At the end of a Chef Client run the node should send it's results to a Log Server for archival purposes.

A Job ID will be necessary to associate the generated log with a Job. This could possibly be achieved by passing setting a value for an attribute containing the ID of the currently executing Job with chef-client's -j flag.

bash# chef-client -j '{ "motherbrain": { "current_job": "SOME_ID" } }'

The handler should clear the currently temporary attribute after a successful report if possible.

dependent on #150

Distribute Mother Brain

create build job
upload gem into internal gem server

Failing specs

Git bisect says introduced with 4cdfb5a

Will try to get to it today

ssh_keys in config should throw file-not-found

If I have:

{
  ...
  "ssh_keys": ["/path/to/my.key.private"]
}

And I've deleted that file, I get:

[2012-12-07T20:12:58Z] PID[21697] TID[ouwkfndik] ERROR: Net::SSH::AuthenticationFailed: root

Should probably get a FileNotFound error.

Config should default to knife config where possible

chef.api_url = :chef_server_url
chef.apl_client = :node_name
chef.api_key = :client_key
chef.validator_client = :validation_client_name
chef.validator_path = :validation_key

And maybe:

log.level = :log_level
log.locaiton = :log_location

If you don't provide a version in your mother brain plugin, you get a stack trace.

You should get a "version is required" message instead.

create a Log Server

A general purpose actor (manager/server) for receiving and storing logs. These logs should be able to be downloaded at a later date.

This actor can used for:

a place to archive the output of Chef Runs
a central place for motherbrain application logging when actors are distributed

Jobs will need to have a log, or potentially many logs, associated with them for retrieval by the end user

resolves:

serial execution

We need the ability to execute actions in a serial fashion, one server at a time, for rolling restarts, etc.

e.g.

on("some_group", :serially) do
  jmx "do some jmx"
  jmx "do some more jmx"
  command("restart").invoke
end

bootstrap does not pass ssh options to ridley

improve error output if given an invalid provision manifest

I attempted to provision with the following manifest:

{
  "chat::databases": 1,
  "chat::services": 2
}

The provision failed but it did not say why.

Logs:

# Logfile created on 2013-01-04 14:52:33 -0800 by logger.rb/31641
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fssis0] INFO: MotherBrain starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fsuq80] INFO: Config Manager starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fxcty4] INFO: Plugin Manager starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2frbwuc] INFO: Provision Manager starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fzuf70] INFO: Bootstrap Manager starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fqjd18] INFO: Node Querier starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2g0sol8] INFO: Lock Manager starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fslmxs] INFO: Ridley starting...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fskz18] DEBUG: job (577cb547-9d92-4954-8bec-000000000000) transitioning to 'failure'
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fpt9zo] DEBUG: Terminating 16 actors...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fssis0] INFO: MotherBrain stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fsuq80] INFO: Config Manager stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fxcty4] INFO: Plugin Manager stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2frbwuc] INFO: Provision Manager stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fqjd18] INFO: Node Querier stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fzuf70] INFO: Bootstrap Manager stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2g0sol8] INFO: Lock Manager stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fslmxs] INFO: Ridley stopping...
[2013-01-04T22:52:33Z] PID[89039] TID[ow2fpt9zo] DEBUG: Shutdown completed cleanly

Deep nested commands (3+ deep) fail

/Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/thread.rb:71:in `sleep': deadlock detected (fatal)
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/1.9.1/thread.rb:71:in `wait'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/mailbox.rb:66:in `receive'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/actor.rb:73:in `block in call'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/actor.rb:72:in `loop'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/actor.rb:72:in `call'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/celluloid-0.12.4/lib/celluloid/proxies/actor_proxy.rb:102:in `method_missing'
        from /Users/michael.ivey/Code/motherbrain/lib/mb/locks.rb:40:in `chef_synchronize'
        from /Users/michael.ivey/Code/motherbrain/lib/mb/command.rb:46:in `invoke'
        from (eval):2:in `block in define_command'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:109:in `invoke'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:213:in `block in subcommand'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
        from /Users/michael.ivey/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/base.rb:425:in `start'
        from /Users/michael.ivey/Code/motherbrain/lib/mb/invoker.rb:16:in `start'
        from bin/mb:6:in `<main>'

motherbrain Cookbook

Create a cookbook to deploy motherbrain

sensitive information (e.g. ssh keys, encrypted data bag keys, etc.) should not be outputted to logs

mb product component nodes fails

rapier:plugins jkiehl$ mb first_app web_server nodes _default
Nodes for 'web_server' in '_default':
/Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/motherbrain-0.0.1/lib/mb/component.rb:86:in `nodes': wrong number of arguments (1 for 0) (ArgumentError)
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/motherbrain-0.0.1/lib/mb/component_invoker.rb:32:in `block (2 levels) in fabricate'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:109:in `invoke'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:213:in `block in subcommand'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:109:in `invoke'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:213:in `block in subcommand'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/thor-0.16.0/lib/thor/base.rb:425:in `start'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/motherbrain-0.0.1/lib/mb/invoker.rb:15:in `start'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/motherbrain-0.0.1/bin/mb:6:in `<top (required)>'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/bin/mb:23:in `load'
    from /Users/jkiehl/.rbenv/versions/1.9.3-p286/bin/mb:23:in `<main>'

environment_attribute does not work

It looks like it might be a problem with Ridley. Environment.find! is returning an environment with no values.

sleep inside on() block

chat needs to be able to sleep 10 seconds of time in between the start of each chat service

  command "start" do
    description "start ejabberd" 

    execute do
      on("services", max_concurrent: 1) do
        service("ejabberd").run("start")
        # TODO (jhowarth): sleep 10
      end
    end
  end

need to be able to pass arguments from command line to commands

Help output is incorrect for components

$ mb first_app web_server 
Tasks:
  mb web_server help [COMMAND]     # Describe subcommands or one specific subcommand
  mb web_server nodes ENVIRONMENT  # List all nodes grouped by Group

Note the lack of first_app in the command

create a motherbrain plugin for motherbrain

This will be used to deploy motherbrain to a cluster using the motherbrain cookbook from #137 to deploy and command and control a motherbrain cluster

provision says EF failed but EF is actually still provisioning

When I provision some nodes with EF, Motherbrain fails and says no nodes were provisioned, but if I go to EF the provision is still in progress. The provision doesn't actually fail.

# Logfile created on 2013-01-08 15:42:36 -0800 by logger.rb/31641
[2013-01-08T23:42:36Z] PID[86861] TID[ow0ttzokc] INFO: MotherBrain starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0tsr4l0] INFO: Config Manager starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0u2tfdk] INFO: Plugin Manager starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0tz5z50] INFO: Provision Manager starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0tu00ms] INFO: Bootstrap Manager starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0tu1nn8] INFO: Node Querier starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0txk4ps] INFO: Lock Manager starting...
[2013-01-08T23:42:36Z] PID[86861] TID[ow0tu9x80] INFO: Ridley starting...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tu1wxo] FATAL: an error occured: Expected '3' nodes to be provisioned but got: '0' 
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tsqd44] FATAL: Job (45d59f10-8686-4586-8c4f-000000000000) failure: Expected '3' nodes to be provisioned but got: '0' 
[2013-01-08T23:42:37Z] PID[86861] TID[ow0ttzokc] INFO: MotherBrain stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tsr4l0] INFO: Config Manager stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0u2tfdk] INFO: Plugin Manager stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tz5z50] INFO: Provision Manager stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tu00ms] INFO: Bootstrap Manager stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tu1nn8] INFO: Node Querier stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0txk4ps] INFO: Lock Manager stopping...
[2013-01-08T23:42:37Z] PID[86861] TID[ow0tu9x80] INFO: Ridley stopping...

on() needs a way to run service actions asynchronously.

Mb should warn if the name of the plugin file is different than the name of the plugin contained

We should not encourage people to name their plugin files differently than what is contained inside the plugin file.

pvpnet.rb should contain name "pvpnet".

Differentiate between a "self-contained" bootstrap and "production" bootstrap

If I write my MB cluster_bootstrap block to bootstrap certain groups, I can end up with a block that duplicates work in some environments, or doesn't work at all in others.

For example,

cluster_bootstrap do
  bootstrap("activemq")
  # bootstrap other things that require activemq
end

In a production world, maybe I don't need to bootstrap activemq because that node is already in my environment. In a "self-contained" environment, I need to bootstrap that node or else some other bootstrap later may fail.

Is there a current way to achieve this functionality?

Could we implement something like:

cluster_bootstrap do
  development do
    bootstrap("activemq")
  end
  # bootstrap other things that require activemq
end

And call it with a command like:

mb my_plugin bootstrap ENVIRONMENT bootstrap.json --development

The development flag sets the block to be run.

need to use recursive merge on attributes

Clone of #155
When upgrading in the upgrade/worker.rb, we'll need to do a recursive merge on the override or default attributes we want to set. Otherwise the results could be unexpected.

For example, in bacon's orchestrator we do this:
https://gh.riotgames.com/bacon/bacon-orchestrator/blob/master/lib/chef_util.rb#L188