Lock cookbook versions based on Berksfile.lock about motherbrain HOT 17 CLOSED

I like this idea, I'd accept a patch that added this functionality.

Cookbook versions are passed in for bootstrap like this: https://github.com/RiotGames/motherbrain/blob/282f754bfdeecc263b2888e52218912259a139bf/lib/mb/cli/sub_command/plugin.rb#L62-L65

Perhaps you could call berkshelf's apply functionality if given a path to Berksfile.lock:
https://github.com/berkshelf/berkshelf/blob/7f4734b1526eecfefd2982fef48468c1ea2c2a9f/lib/berkshelf/lockfile.rb#L103-L131

from motherbrain.

This implies we would need to add berkshelf as a dependency in motherbrain's gemspec. I guess the other alternative would be to do something like

begin
  require 'berkshelf'
  # apply the lockfile
rescue LoadError
  # report that this option requires the berkshelf gem to be installed.
end

Any preference on your end?

from motherbrain.

I'd prefer a rescue as people who don't need the feature need not be burdened by the massive dependency tree that Berkshelf would introduce.

from motherbrain.

The upgrade command in motherbrain came before berks apply and it's current implementation was just a stop gap until we got the design right for how to promote software through environments.

The Environment Cookbook pattern + berks apply actually should just become a first class citizen in motherbrain. That pattern is the end of the road that the the plugin system + the berkshelf lock takes us to.

I would highly recommend making Berkshelf a hard dependency and searching for a Berksfile.lock in the contents of the cookbook that you're upgrading to. Remove the --cookbooks flag all together and instead rely on the lockfile found in the cookbook matching the version we're upgrading an environment to.

If you're unfamiliar about the Environment Cookbook pattern you can read about it here.

from motherbrain.

I usually err on the side of reducing dependencies (ruby dependency hell is no joke), and feel that we should wait till we have more features relying on Berkshelf's internals prior to making it a hard dependency, but since Berkshelf is already more or less a requirement for using Motherbrain and the way we use Motherbrain is very Berkshelf centric, I'm almost as happy with a hard dependency.

from motherbrain.

So what I am hearing is

• Add berkshelf as a dependency to motherbrain.
• Drop the --cookbooks flag from bootstrap, provision and upgrade commands
• Look in the plugin directory for Berksfile.lock. If found then apply those versions to the environment for the bootstrap, provision and upgrade commands.

It seems that the --cookbooks flag could be useful for people that are not using the environment pattern but I will implement it how you guys think best. I guess one option would be to leave the --cookbooks option but if it is not specified and the Berksfile.lock is found then use it.

from motherbrain.

Leave it in for now. Even though we're not at 1.0.0 yet, I still want to be conscious of breaking backwards compatibility. 1.0.0 will be the next release, at which point we can decide if --cookbooks should be deprecated and removed in a future release.

Given the tool is used in production, it should have been 1.0.0 already.

from motherbrain.

@rteabeault the environment cookbook pattern should be assumed when using motherbrain. The software was purposefully designed to enforce particular patterns.

@bluepojo I think hiding the --cookbooks flag and printing a helpful deprecation warning is the way to go. Then we should remove it all together in the releasing following the next.

from motherbrain.

@reset, @bluepojo
Say I have an environment cookbook and a build pipeline where the first step of the pipeline is thor version:bump patch. The lockfile at this point is "mostly" correct except the version of the environment cookbook is off (since we just bumped it). If you do a berks install it gets corrected with the new version. So my question is would motherbrain ensure that berks install (via the berks API) is called or does it assume that berks install was called prior to calling mb bootstrap/provision/upgrade? I would tend to assume the latter.

from motherbrain.

I'd suppose you would generate the Berksfile.lock after the versioning occurs as part of the same job before uploading the cookbook. This opens the small possibility of a dependency's fuzzy matcher picking up a newly released cookbook in the lag time between commit and the regenerating Berksfile.lock in your task, however, so there is some risk with that approach.

We don't yet use the environment cookbook pattern at Riot. We're still puzzling over whether it's a good fit for our methods of deployment automation. We may opt for that pattern, but there are a few fuzzy cases that the basic pattern does not account for (or I just need to understand the pattern better...).

I may write up a blog post on how we create release artifacts (ie, the data required to create an entire production environment) once we've worked out the kinks in the process completely... the end goal is the same as the environment cookbook pattern, just with a bit more flexibility, I think.

from motherbrain.

At the moment I am a strong 👎 on adding Berkshelf as a dependency to MB:

• I'm not sure of any more obvious use-cases for Berkshelf, but I think its too large of a library for only one use-case
• The Berkshelf::Lockfile is just an object holding some data and having the ability to parse itself from a file. To solve my first point above, I think we should just pull in the class to MB (once the below bullet is satisfied).
• We definitely should not depend on a beta version of BS or a copy of Berkshelf::Lockfile until 3.0.0 is released.

Also, I don't think --cookbooks should be so explicitly deprecated as it can be a useful flag. I'd rather see the code formatted in a way to allow cookbooks or a lockfile and then cover the edge cases. To go a step forward, I'd say --cookbooks (being a command line flag) trumps a lockfile even if one exists.

from motherbrain.

@KAllan357 the --cookbooks flag was just a temporary solution when it was added. Both motherbrain and Berkshelf are opinionated software and flags like this suggest alternative workflows that we may not have intended for and add additional noise to an already complex tool.

When do you think you would use that instead of just deploying from a lockfile?

from motherbrain.

@reset My thinking was that I would use that flag when I don't have a lockfile available. I think in general, my opinion is that its okay to have extra flags available and that they should have the highest precedence.

In the case of --cookbook, I don't think its an unreasonable option, nor does it really suggest an alternative workflow. It is just another for of providing me access to the cookbooks JSON in a Chef environment. It could be useful in many cases:

• I have a cookbook that was uploaded without a lockfile but I still want to use that version of the cookbook with motherbrain.
• I want to override the provided lockfile in cases where my lockfile is incorrect. Or when I want to try something crazy.

from motherbrain.

@KAllan357 this workflow suggests that you would never have a cookbook without a lockfile. We shouldn't shy away from marrying Berkshelf with motherbrain, that was always the intention. Would this break something at Riot?

from motherbrain.

Overall, we definitely are 👍 on the feature itself. I just want to make sure we don't deprecate functionality we use just yet (aka --cookbooks).

from motherbrain.

@KAllan357 ya for sure. Deprecation will just be a warning, not a removal. We should avoid removing critical path things :)

from motherbrain.

Feel free to reopen if there's more discussion needed on this topic.

🐹

from motherbrain.