rcarver / config Goto Github PK

Script should behave like File and show print the script to STDOUT during execution.

The Getting Started Guide is out of date. Maybe other things too.

Config needs to manage long running applications, probably via an init system of some kind such as upstart. When a pattern that a service depends on causes a change to the system, it should restart.

When a node execution fails, what happens to the stored previous execution and more importantly, any deleted patterns.

From #1:

MC: An implementation detail but what happens on a config run failure?
Will it remember that it didn't successfully complete and then correctly
attempt to destroy?)

Bootstrap should check that the data repo exists before running so that bootstrapping doesn't fail at the last step.

With no args, it reads from the wrong object.

Generating rdoc takes way too long and we don't need it. Tell ruby not to do that during bootstrap. Also RI.

You should be able to define configuration at the node level. It'll look just like a cluster file, but stored at nodes/<fqn>.rb. You should be able to create this file before or after the node exists.

I'm not sure why this is commented out but it would be nice to have:
https://github.com/rcarver/config/blob/master/lib/config/patterns/file.rb#L138

How are you supposed to be able to access the cluster from the node?

node.cluster and node.cluster_name both seem to be blank inside of a template.

These links don't exist in GETTING_STARTED.md (not sure if that is intentional):
https://github.com/rcarver/config/blob/master/doc/SSH.md
https://github.com/rcarver/config/blob/master/doc/GIT.md
https://github.com/rcarver/config/blob/master/doc/SECRETS.md
https://github.com/rcarver/config/blob/master/doc/GITHUB.md

When config-run executes, it should first cd to a safe and/or known location. It currently executes from the project directory, which means that any commands (and especially scripts) that don't put their files in the right location will likely end up there. This results in weird state in the project dir.

Instead, I propose that /etc/config/run is created fresh at the start of each run, and that before executing any patterns we cd there.

When an exception occurs in the CLI, improve the output. It should show the message clearly, followed by the backtrace. And colors.

Validate that the necessary host key files exist when creating the bootstrap script. Otherwise you only find out very late in the bootstrap process.

With the latest config, if I run it twice it works the first time and fails the second time and I've noticed that the accumulation.marshall file exists only on the second run.

I have a pattern called Base::App.

/opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/core/accumulation.rb:12:in `restore': undefined class/module Base:: (ArgumentError)
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/core/accumulation.rb:12:in `from_string'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/private_data.rb:42:in `accumulation'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/cli/exec_node.rb:22:in `execute'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/cli.rb:80:in `run'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/cli.rb:11:in `exec'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/lib/config/cli/exec.rb:5:in `<top (required)>'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/bin/config-exec-node:2:in `require'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/config-efdb5bef53a6/bin/config-exec-node:2:in `<top (required)>'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bin/config-exec-node:23:in `load'
        from /opt/ruby-config-1.9.3-p194/lib/ruby/gems/1.9.1/bin/config-exec-node:23:in `<main>'

Config::Patterns::Script should add -e to every script by default. Without it, it's very easy for an error in the script to cause confusing downstream effects.

There could be an option to disable this if you know what you're doing, but it should help avoid simple mistakes for the majority of cases.

So if you use the simple templating part of the file helper (which is the default) and you go to remove the file it doesn't work.

It fails because when you marshall the template_context (which is the pattern in this case), it uses it's to_s function and gives you something like this:

{"path":"/usr/src/nginx-rules.patch","owner":null,"group":null,"mode":null,"touch":false,"content":null,"template_path":"/etc/config/project/patterns/packages/templates/nginx-rules.patch.erb","template_context":"[Packages::Nginx ]","operation":"write"}

Unfortunately you can't call get_bindings on a string from here:
https://github.com/rcarver/config/blob/master/lib/config/patterns/file.rb#L93

As far as I can tell it doesn't properly accumulate the script actions. Looking on the surface it looks like it should work but I have a funny feeling something isn't quite right about adding inside of a call. The other patterns work and don't seem to do that.

config-know-hosts is broken without a .data dir

When config-exec-node runs, it should store the execution at /etc/config/last_run (tbd). When config-exec-node runs and that file exists, it uses that to set the previous_execution on the blueprint.

In contrast to the Getting Started Guide, you can't run bin/config-init-project before setting up a git remote.

1. Even if you do this it shouldn't crash, it just won't auto-detect your remotes.
2. The Guide should fix the order of operations so that it does detect your remotes.

I think the defaults aren't getting read somewhere as:

configure :project_git_repo,
  url: "[email protected]:rcarver/config-example.git"

Generates this in /root/.ssh/config

Host
  Port
  Hostname
  User
  IdentityFile /etc/config/ssh-key-

Also you seem to use host and hostname in your ssh configuration, but the docs only refer to hostname:
https://github.com/rcarver/config/blob/master/lib/config/core/ssh_config.rb#L45

Installing ruby gems is pretty common. The install command should be something like this:

 gem install --conservative --no-rdoc --no-ri <GEM>

add .vagrant to .gitignore

Some configuration items should be kept secret. Things like AWS keys or other login details. Other items are large and would be better stored in a file. Still others are both large and secret (certs).

Config should make it easy to encrypt and securely store these items within your project.

You should be able to remove a node from the database with config-remove-node

Two things here:

1. Don't require a secret. If you never use it, it shouldn't be required.
2. If you do use secret(s), verify that they all exist when creating the bootstrap script, not later in the process when they're actually used.

SSH host signatures are currently stored in the private data dir and passed to new nodes on bootstrap. However, these keys are not secret and so could be stored in the project repo. This simplifies setup a lot, especially in the case of a local vagrant-based workflow where each developer needs to generate those keys in order to boot a vagrant box.

Once stored in the project repo, they could be updated/rotated by simply changing the value in the project. The keys will still be copied out of the project repo to /root/.ssh/known_hosts, it will just happen both at bootstrap and also on config-run.

I don't believe this exposes any security risk. It is already the case that if an attacker gains access to the project repo that they could run any arbitrary command on the system. Does including the ability to alter the ssh known host keys change the attack surface? @mcolyer any thoughts here?

The only other big question on this is where in the project would we store these values? There isn't an obvious place today.