Would it be possible to support the cookbooks collection on Github? You'd likely be accessing via the Github API, e.g checking availability etc.

I've recently (yesterday) made the first run at tracking repo specific cookbooks (some of heavywater's cookbooks). Hopefully over time cookbooks will evolve into a collection that:

• live on when whoever is upstream drops them.
• extend beyond opscode (currently only heavywater and miketheman (ruby_enterprise) are tracked).
• compatible irrespective of who is upstream (the qa branch)
• track a changes in who is considered upstream.

Of course this is a classic chicken-and-the-egg problem: lots of people won't use this collection until it is easy to use, the collection will be easy to use when bugs are ironed out by lots of people using them.

Librraian support for site :cookbooks would at least make it easy for people to consider using.
If this is possble I'd remark that the branch qa should be considered the source branch. The master branch being reserved for upstream.

Thoughts?

I've added it, but I'm unsure whether it's best-practice. Clarification would be helpful in the docs.

The difference is unclear between (a) resolving, including re-resolving & updating, and (b) installing. It is unclear when to use which, and it is un[der]documented.

Option 1: Document it.

Option 2: Change update also to perform reinstall, and provide a way to update without reinstalling.

librarian-chef update [XYZ] # resolves, rewrites lockfile, & installs
librarian-chef update [XYZ] --resolve # resolves & rewrites lockfile, but does not install

Option 3: Get rid of the difference between [re]writing the lockfile & performing the install. Change [re]resolving always also to [re]install.

Validate dependency names.

Rules for validating dependency names will vary per-adapter.

If a dependency is listed in the specfile or lockfile or is encountered during resolution, immediately raise with an informative message.

Question: what is the correct rule for the chef adapter, and where is it documented?

Ref: heavywater/pennyworth#6

Comment over here had a point -- code review is awkward when the code is spread around and not conveniently linked. Never written a greasemonkey script, but it seems something that would be worth doing. I'll see if I can find some time and pass it along when I get it working.

For now, I'll just link to the opscode community site (As there's a link to the code there), but perhaps I could get more complex and use their api later.

https://github.com/carlhuda/bundler/blob/master/lib/bundler/cli.rb#L87

No biggie, just thought I'd log the feature request :)

When resolving, check if dependencies/manifests exist at the chef-site source.

Immediately raise a dependency-not-found error if a dependency is not found, failing fast.

Is there any way to force yes for librarian-chef installs? For example RSA key fingerprint.

Let's do it.

Verbose output:
ip-172-16-5-158:chef-repo apeshimam$ librarian-chef install --verbose
[Librarian] Pre-Cached Sources:
[Librarian] Post-Cached Sources:
[Librarian] [:site, "http://community.opscode.com/api/v1", {}]
[Librarian] [:git, "https://github.com/fnichol/chef-jenkins", {:ref=>"v0.5"}]
[Librarian] [:git, "https://github.com/fnichol/chef-rvm", {:ref=>"v0.8.6"}]
[Librarian] [:git, "[email protected]:CXInc/chef-cxapi.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-cxdb.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-cxror.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-cxvpn.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-haproxy.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-hostname.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-munin.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-nagios.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-percona-install.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-php-fpm.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-rundeck.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-ssh_known_hosts.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-s3cmd_logs.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-stud.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-users.git", {}]
[Librarian] [:git, "[email protected]:CXInc/chef-wordpress.git", {}]
[Librarian] Scheduling apache2 (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling application (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling apt (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling aws (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling build-essential (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling couchdb (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling cron (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling chef-client (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling chef_handler (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling chef-server (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling database (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling elasticsearch (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling erlang (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling git (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling graylog2 (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling java (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling logrotate (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling logstash (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling memcached (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling mysql (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling nginx (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling ntp (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling ohai (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling openssh (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling openssl (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling openvpn (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling perl (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling php (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling postfix (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling python (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling rabbitmq (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling rabbitmq_chef (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling route53 (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling rsync (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling rsyslog (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling runit (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling sensu (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling solr (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling sudo (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling sysctl (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling tmux (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling ucspi-tcp (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling varnish (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling vim (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling xfs (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling xml (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling yum (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling zlib (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling zookeeper (>= 0) http://community.opscode.com/api/v1
[Librarian] Scheduling jenkins (>= 0) https://github.com/fnichol/chef-jenkins#v0.5
[Librarian] Scheduling rvm (>= 0) https://github.com/fnichol/chef-rvm#v0.8.6
[Librarian] Scheduling chef-cxapi (>= 0) <[email protected]:CXInc/chef-cxapi.git#master>
[Librarian] Scheduling cxdb (>= 0) <[email protected]:CXInc/chef-cxdb.git#master>
[Librarian] Scheduling cxror (>= 0) <[email protected]:CXInc/chef-cxror.git#master>
[Librarian] Scheduling cxvpn (>= 0) <[email protected]:CXInc/chef-cxvpn.git#master>
[Librarian] Scheduling haproxy (>= 0) <[email protected]:CXInc/chef-haproxy.git#master>
[Librarian] Scheduling hostname (>= 0) <[email protected]:CXInc/chef-hostname.git#master>
[Librarian] Scheduling munin (>= 0) <[email protected]:CXInc/chef-munin.git#master>
[Librarian] Scheduling nagios (>= 0) <[email protected]:CXInc/chef-nagios.git#master>
[Librarian] Scheduling percona-install (>= 0) <[email protected]:CXInc/chef-percona-install.git#master>
[Librarian] Scheduling php-fpm (>= 0) <[email protected]:CXInc/chef-php-fpm.git#master>
[Librarian] Scheduling rundeck (>= 0) <[email protected]:CXInc/chef-rundeck.git#master>
[Librarian] Scheduling ssh_known_hosts (>= 0) <[email protected]:CXInc/chef-ssh_known_hosts.git#master>
[Librarian] Scheduling s3cmd_logs (>= 0) <[email protected]:CXInc/chef-s3cmd_logs.git#master>
[Librarian] Scheduling stud (>= 0) <[email protected]:CXInc/chef-stud.git#master>
[Librarian] Scheduling users (>= 0) <[email protected]:CXInc/chef-users.git#master>
[Librarian] Scheduling wordpress (>= 0) <[email protected]:CXInc/chef-wordpress.git#master>
[Librarian] Resolving apache2 (>= 0) http://community.opscode.com/api/v1
[Librarian] No known prior constraints
[Librarian] Caching http://community.opscode.com/api/v1/cookbooks/apache2
[Librarian] Checking manifests
[Librarian] Checking apache2/1.1.6 http://community.opscode.com/api/v1
[Librarian] Resolving application (>= 0) http://community.opscode.com/api/v1
[Librarian] No known prior constraints
/Users/apeshimam/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/pathname.rb:776:in read': No such file or directory - /Users/apeshimam/chef-repo/tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/application/metadata.json (Errno::ENOENT) from /Users/apeshimam/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/pathname.rb:776:inread'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/chef/source/site.rb:74:in manifests' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/dependency.rb:64:incache_manifests!'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/dependency.rb:59:in manifests' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:49:inblock in recursive_resolve'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:102:in scope' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:36:inrecursive_resolve'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:70:in block (4 levels) in recursive_resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:102:inscope'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:57:in block (3 levels) in recursive_resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:54:ineach'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:54:in block (2 levels) in recursive_resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:102:inscope'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:53:in block in recursive_resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:102:inscope'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:36:in recursive_resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver/implementation.rb:21:inresolve'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/resolver.rb:18:in resolve' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/action/resolve.rb:25:inrun'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/cli.rb:108:in resolve!' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/cli.rb:67:ininstall'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/thor-0.14.6/lib/thor/task.rb:22:in run' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/thor-0.14.6/lib/thor/invocation.rb:118:ininvoke_task'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/thor-0.14.6/lib/thor.rb:263:in dispatch' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/thor-0.14.6/lib/thor/base.rb:389:instart'
from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/lib/librarian/cli.rb:27:in bin!' from /Users/apeshimam/.rvm/gems/ruby-1.9.3-rc1/gems/librarian-0.0.16/bin/librarian-chef:7:in<top (required)>'
from /Users/apeshimam/.rvm/gems/ruby-1.9.2-p290/bin/librarian-chef:19:in load' from /Users/apeshimam/.rvm/gems/ruby-1.9.2-p290/bin/librarian-chef:19:in

Cheffile:

!/usr/bin/env ruby

^syntax detection

site 'http://community.opscode.com/api/v1'
#100% upstream

cookbook 'apache2'
cookbook 'application'
cookbook 'apt'
cookbook 'aws'
cookbook 'build-essential'
cookbook 'couchdb'
cookbook 'cron'
cookbook 'chef-client'
cookbook 'chef_handler'
cookbook 'chef-server'
cookbook 'database'
cookbook 'elasticsearch'
cookbook 'erlang'
cookbook 'git'
cookbook 'graylog2'
cookbook 'java'
cookbook 'logrotate'
cookbook 'logstash'
cookbook 'memcached'
cookbook 'mysql'
cookbook 'nginx'
cookbook 'ntp'
cookbook 'ohai'
cookbook 'openssh'
cookbook 'openssl'
cookbook 'openvpn'
cookbook 'perl'
cookbook 'php'
cookbook 'postfix'
cookbook 'python'
cookbook 'rabbitmq'
cookbook 'rabbitmq_chef'
cookbook 'route53'
cookbook 'rsync'
cookbook 'rsyslog'
cookbook 'runit'
cookbook 'sensu'
cookbook 'solr'
cookbook 'sudo'
cookbook 'sysctl'
cookbook 'tmux'
cookbook 'ucspi-tcp'
cookbook 'varnish'
cookbook 'vim'
cookbook 'xfs'
cookbook 'xml'
cookbook 'yum'
cookbook 'zlib'
cookbook 'zookeeper'

3rd party

cookbook 'jenkins',
:git => 'https://github.com/fnichol/chef-jenkins',
:ref => 'v0.5'
cookbook 'rvm',
:git => 'https://github.com/fnichol/chef-rvm',
:ref => 'v0.8.6'

CX Maintained

cookbook 'chef-cxapi',
:git => '[email protected]:CXInc/chef-cxapi.git'
cookbook 'cxdb',
:git => '[email protected]:CXInc/chef-cxdb.git'
cookbook 'cxror',
:git => '[email protected]:CXInc/chef-cxror.git'
cookbook 'cxvpn',
:git => '[email protected]:CXInc/chef-cxvpn.git'
cookbook 'haproxy',
:git => '[email protected]:CXInc/chef-haproxy.git'
cookbook 'hostname',
:git => '[email protected]:CXInc/chef-hostname.git'
cookbook 'munin',
:git => '[email protected]:CXInc/chef-munin.git'
cookbook 'nagios',
:git => '[email protected]:CXInc/chef-nagios.git'
cookbook 'percona-install',
:git => '[email protected]:CXInc/chef-percona-install.git'
cookbook 'php-fpm',
:git => '[email protected]:CXInc/chef-php-fpm.git'
cookbook 'rundeck',
:git => '[email protected]:CXInc/chef-rundeck.git'
cookbook 'ssh_known_hosts',
:git => '[email protected]:CXInc/chef-ssh_known_hosts.git'
cookbook 's3cmd_logs',
:git => '[email protected]:CXInc/chef-s3cmd_logs.git'
cookbook 'stud',
:git => '[email protected]:CXInc/chef-stud.git'
cookbook 'users',
:git => '[email protected]:CXInc/chef-users.git'
cookbook 'wordpress',
:git => '[email protected]:CXInc/chef-wordpress.git'

Cheffile.lock:

ip-172-16-5-158:chef-repo apeshimam$ cat Cheffile.lock
SITE
remote: http://community.opscode.com/api/v1
specs:
apache2 (1.1.4)
application (0.99.14)
apache2 (>= 0.0.0)
gunicorn (>= 0.0.0)
passenger_apache2 (>= 0.0.0)
php (>= 0.0.0)
python (>= 1.0.6)
runit (>= 0.0.0)
tomcat (>= 0.0.0)
unicorn (>= 0.0.0)
apt (1.4.0)
aws (0.99.1)
bluepill (1.0.4)
build-essential (1.0.0)
capistrano (0.7.0)
chef-client (1.1.2)
chef-server (0.99.12)
apache2 (>= 0.0.0)
bluepill (>= 0.0.0)
couchdb (>= 0.0.0)
daemontools (>= 0.0.0)
gecode (>= 0.0.0)
java (>= 0.0.0)
nginx (>= 0.0.0)
openssl (>= 0.0.0)
runit (>= 0.0.0)
xml (>= 0.0.0)
zlib (>= 0.0.0)
chef_handler (1.0.6)
couchdb (1.0.0)
erlang (>= 0.0.0)
cron (1.0.0)
daemontools (0.9.0)
build-essential (>= 0)
ucspi-tcp (>= 0)
database (1.1.4)
aws (>= 0.0.0)
mysql (>= 1.2.0)
postgresql (>= 0.0.0)
xfs (>= 0.0.0)
elasticsearch (0.1.0)
bluepill (>= 0)
java (>= 0)
erlang (1.0.0)
yum (>= 0.5.0)
gecode (1.2.0)
apt (>= 0.0.0)
build-essential (>= 0.0.0)
git (0.10.0)
runit (>= 0.0.0)
yum (>= 0.0.0)
graylog2 (0.0.5)
apache2 (>= 0.0.0)
apt (>= 0.0.0)
gunicorn (1.0.0)
python (>= 0)
install_from (3.0.4)
iptables (0.10.0)
java (1.4.2)
logrotate (0.8.2)
logstash (0.0.1)
java (>= 0.0.0)
runit (>= 0.0.0)
memcached (1.0.2)
runit (>= 0.0.0)
metachef (3.0.4)
mysql (1.2.4)
openssl (>= 0.0.0)
nginx (0.100.2)
bluepill (>= 0.0.0)
build-essential (>= 0.0.0)
runit (>= 0.0.0)
ntp (1.1.6)
ohai (1.0.2)
openssh (0.8.0)
openssl (1.0.0)
openvpn (0.99.2)
passenger_apache2 (0.99.4)
apache2 (>= 1.0.4)
build-essential (>= 0.0.0)
perl (0.10.0)
php (1.0.2)
build-essential (>= 0.0.0)
mysql (>= 0.0.0)
xml (>= 0.0.0)
postfix (0.8.4)
postgresql (0.99.4)
openssl (>= 0.0.0)
python (1.0.6)
build-essential (>= 0.0.0)
rabbitmq (1.4.0)
apt (>= 1.1)
erlang (>= 0.9)
yum (>= 0.5.0)
rabbitmq_chef (0.10.4)
erlang (>= 0)
redis (3.0.4)
install_from (>= 0.0.0)
metachef (>= 0.0.0)
runit (>= 0.0.0)
route53 (0.2.0)
rsync (0.7.0)
rsyslog (1.0.0)
cron (>= 0.0.0)
runit (0.15.0)
sensu (0.0.7)
apt (>= 0.0.0)
git (>= 0.0.0)
rabbitmq (>= 0.0.0)
redis (>= 0.0.0)
solr (0.8.1)
capistrano (>= 0)
java (>= 0)
runit (>= 0)
sudo (1.0.2)
sysctl (0.1.0)
tmux (1.0.0)
tomcat (0.10.4)
java (>= 0.0.0)
ucspi-tcp (1.0.0)
build-essential (>= 0)
unicorn (1.0.0)
varnish (0.7.0)
vim (1.0.2)
xfs (1.0.0)
xml (1.0.2)
yum (0.5.2)
zlib (1.0.0)
zookeeper (0.1.0)
java (>= 0)
runit (>= 0)

GIT
remote: [email protected]:CXInc/chef-cxapi.git
ref: master
sha: c7615dc8e5327011d28f6b8564565040719f89f1
specs:
chef-cxapi (0.0.25)
java (>= 1.4.2)
runit (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-cxdb.git
ref: master
sha: c3a290ccfba7d9c10b23dc46868e6c4dc48f57b4
specs:
cxdb (0.0.1)

GIT
remote: [email protected]:CXInc/chef-cxror.git
ref: master
sha: 4b2f0cee04663bdd8aa7fd28e0db13552039a175
specs:
cxror (0.0.1)
git (>= 0.0.0)
nginx (>= 0.0.0)
rvm (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-cxvpn.git
ref: master
sha: 201fe9b2a6c3fd5027151ed68cdc16d799099803
specs:
cxvpn (0.0.1)

GIT
remote: [email protected]:CXInc/chef-haproxy.git
ref: master
sha: 1973557fadf36aa6be5668ba5db81e903a378e2e
specs:
haproxy (1.0.58)
build-essential (>= 0.0.0)
git (>= 0.0.0)
rsyslog (>= 0.0.0)
runit (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-hostname.git
ref: master
sha: 082a2d3f3f190dac4258eee1603eb03eb2a662fd
specs:
hostname (0.0.2)
route53 (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-munin.git
ref: master
sha: 0ca8ed0b94438b28e06c9171a52cec1e4db1da80
specs:
munin (1.0.0)
apache2 (>= 1.0.6)
perl (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-nagios.git
ref: master
sha: ef7cafee264020cdc36d10e20507d552a166ac96
specs:
nagios (1.0.5)
apache2 (>= 0.0.0)
build-essential (>= 0.0.0)
php (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-percona-install.git
ref: master
sha: 7a26df8cb392e76d3c801afac08a2f841edab577
specs:
percona-install (0.0.5)

GIT
remote: [email protected]:CXInc/chef-php-fpm.git
ref: master
sha: 43c96e09db2bf4e3b428523eeaf57fc81693fc05
specs:
php-fpm (0.1.0)
php (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-rundeck.git
ref: master
sha: 7fa361d78e1e32be8cd39061ed5030df770ab03d
specs:
rundeck (0.0.2)
java (>= 0.0.0)
rvm (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-s3cmd_logs.git
ref: master
sha: 2b936024979ad6f22c3c4da4a9c1754fe5c1ccde
specs:
s3cmd_logs (0.0.1)

GIT
remote: [email protected]:CXInc/chef-ssh_known_hosts.git
ref: master
sha: 7ee26008156fc3cb65ce3201730b77936f4d9f8a
specs:
ssh_known_hosts (0.3.1)

GIT
remote: [email protected]:CXInc/chef-stud.git
ref: master
sha: cd45c486ed945ba4d732bcb92392bf52bca35e91
specs:
stud (0.0.3)
build-essential (>= 0.0.0)
git (>= 0.0.0)
runit (>= 0.0.0)

GIT
remote: [email protected]:CXInc/chef-users.git
ref: master
sha: 53cad1f8265b03ba10fc717e340f825490201a0e
specs:
users (1.0.1)

GIT
remote: [email protected]:CXInc/chef-wordpress.git
ref: master
sha: 03f2babb53ea22b9a63239b6692ac487cad904da
specs:
wordpress (0.8.8)
apache2 (>= 0.99.4)
git (>= 0.0.0)
mysql (>= 1.0.5)
nginx (>= 0.99.0)
openssl (>= 0.0.0)
php (>= 0.0.0)
php-fpm (>= 0.1.0)

GIT
remote: https://github.com/fnichol/chef-jenkins
ref: v0.5
sha: 3fc6a56e6d40d8229afe301b4eee460d9537f0aa
specs:
jenkins (0.5.0)
iptables (>= 0.0.0)
java (>= 0.0.0)
runit (>= 0.0.0)

GIT
remote: https://github.com/fnichol/chef-rvm
ref: v0.8.6
sha: f5b89cb4dafa261afb5b18c4d0936aaf302e2a15
specs:
rvm (0.8.6)

DEPENDENCIES
apache2 (>= 0)
application (>= 0)
apt (>= 0)
aws (>= 0)
build-essential (>= 0)
chef-client (>= 0)
chef-cxapi (>= 0)
chef-server (>= 0)
chef_handler (>= 0)
couchdb (>= 0)
cron (>= 0)
cxdb (>= 0)
cxror (>= 0)
cxvpn (>= 0)
database (>= 0)
elasticsearch (>= 0)
erlang (>= 0)
git (>= 0)
graylog2 (>= 0)
haproxy (>= 0)
hostname (>= 0)
java (>= 0)
jenkins (>= 0)
logrotate (>= 0)
logstash (>= 0)
memcached (>= 0)
munin (>= 0)
mysql (>= 0)
nagios (>= 0)
nginx (>= 0)
ntp (>= 0)
ohai (>= 0)
openssh (>= 0)
openssl (>= 0)
openvpn (>= 0)
percona-install (>= 0)
perl (>= 0)
php (>= 0)
php-fpm (>= 0)
postfix (>= 0)
python (>= 0)
rabbitmq (>= 0)
rabbitmq_chef (>= 0)
route53 (>= 0)
rsync (>= 0)
rsyslog (>= 0)
rundeck (>= 0)
runit (>= 0)
rvm (>= 0)
s3cmd_logs (>= 0)
sensu (>= 0)
solr (>= 0)
ssh_known_hosts (>= 0)
stud (>= 0)
sudo (>= 0)
sysctl (>= 0)
tmux (>= 0)
ucspi-tcp (>= 0)
users (>= 0)
varnish (>= 0)
vim (>= 0)
wordpress (>= 0)
xfs (>= 0)
xml (>= 0)
yum (>= 0)
zlib (>= 0)
zookeeper (>= 0)

https://gist.github.com/2354413

tsort.rb:152:in `block in tsort_each': topological sort failed: "windows", "chef_handler", "webpi"

The same code, using a cookbook without an underscore succeeds.

heavywater/pennyworth#14

The above issue made me realize that there might be a deficiency to removing the cookbooks dir from .gitignore, in that if there are submodules in the cookbooks-sources directory, their .git folders are pulled in, and this gets messy. Would some sort of per-cookbook strip option be possible?

cookbook 'mycookbook',
  :path => 'cookbooks-sources',
  :exclude => [ ".git" ]

Thoughts?

I'm having trouble updating cookbooks from remote git repositories with librarian-chef update.

I have a cookbook included in my Cheffile like this:

cookbook 'system', :git => 'git://mygitserver.example.com/system.git', :ref=>'master'

It looks to me that after the initial caching(cloning) of the remote repository in tmp/librarian/cache/source/git/... new commits in the master branch don't get pulled at all.

The solution for me is to issue a librarian-chef clean to flush the cache. After that a librarian-chef update pulls in the latest changes from the upstream repository.

I'm I missing something or is this a bug in librarian?

Do not permit dependencies or manifests with blank names. If such exists or is found, fail immediately.

• In the specfile.
• In the lockfile.
• When resolving.

It would be cool given chef roles if the run-lists could be detected and then used to build the initial cookbook inclusions that are in the Cheffile.

Notes:

• https://github.com/opscode/chef/blob/master/chef/lib/chef/run_list.rb#L136-140
• https://github.com/opscode/chef/blob/master/chef/lib/chef/run_list/run_list_expansion.rb#L150-158

I wish I could give more on this without giving my whole repo, but I think it somehow has something to do with the cookbooks themselves... I've gone back and forth between two chef-repos that I'm using librarian for, and the only diff in the knife.rb is the name of the organization on opscode platform (and the fact that there's a comment on the first line in the working one).

Any thoughts on how to troubleshoot this better? For now, I'll just hardcode cookbooks and remove the require.

ERROR: You have an error in your config file /Users/homebase/new-chef-repo/.chef/knife.rb
NoMethodError: undefined method `to_str' for nil:NilClass
  /Users/homebase/new-chef-repo/.chef/knife.rb:1:in `from_file'

     # /Users/homebase/new-chef-repo/.chef/knife.rb
  1: require 'librarian/chef/integration/knife'
  2: 
  3: current_dir = File.dirname(__FILE__)
  4: 

The install path should be a rememberable configuration option.

It would be really cool to support an init task to generate a boilerplate Cheffile a la bundler and guard. If you're looking for help, let me know.

We could do with a command to find and list outdated dependencies.

Ran into this with trying to bundle vagrant 0.9.3 and librarian 0.0.12.

Seems that chef 0.10.8 release is still dependant on net-ssh 2.1.3, while vagrant is set to 2.2.2. Just a heads up that you might want to include a Gemfile to offer an example resolution until the next version of chef comes out (despite it being a vagrant dilemma :)

gem "vagrant",   "= 0.9.3"
gem "librarian", "= 0.0.12"
# net-ssh dependency bumped to 2.2.2 (to match librarian)
gem "chef",
  :git => "https://github.com/opscode/chef.git",
  :ref => "ba4d58f4223"

Sorry if I'm just kinda making up technical words here, but I don't know the specific term.

Basically, I did this in one metadata.rb:

%{
  apache2
  apt
  gpg
}.each do |cookbook|
  depends cookbook
end

It actually worked for awhile, but threw random errors. Also noticed that it created a folder deep down in /tmp called \n.

In retrospect, I should've been using %w{ }, but it couldn't hurt to deal with it so it doesn't discourage someone else :)

Come to think of it, I think I picked up that format from fujin while I was browsing the pennyworth code, so it might explain my weird sporadic errors over there too

Cheers!

The resolution process in update NAME1 NAME2 ... sometimes excludes manifests that are already in the previous resolution, that are not supposed to be stripped out, and that are required for a consistent resolution.

I'm trying to add the ruby-shadow cookbook to my chef-repo via my Cheffile. I have added the following lines:

cookbook 'ruby-shadow', :git => 'git://github.com/37signals/37s_cookbooks.git'

When I run librarian-chef install --verbose I get the following output / errors:

pangaea ~/code/rm/chef-repo[master*]$ librarian-chef install --verbose
[Librarian] Precaching Sources:
[Librarian]   http://community.opscode.com/api/v1
[Librarian]   git://github.com/fnichol/chef-user.git#v0.2.0
[Librarian] Analyzing spec and lock:
[Librarian]   Removed:
[Librarian]   ExplicitRemoved:
[Librarian]   Added:
[Librarian]     ruby-shadow
[Librarian]   NonMatchingAdded:
[Librarian]     ruby-shadow
[Librarian]   Changed:
[Librarian]   DeepKeep:
[Librarian]     redis2
[Librarian]     sudo
[Librarian]     ufw
[Librarian]     user
[Librarian]   ShallowStrip:
[Librarian] Scheduling redis2 (>= 0) <http://community.opscode.com/api/v1>
[Librarian] Scheduling ufw (>= 0) <http://community.opscode.com/api/v1>
[Librarian] Scheduling sudo (>= 0) <http://community.opscode.com/api/v1>
[Librarian] Scheduling user (>= 0) <git://github.com/fnichol/chef-user.git#v0.2.0>
[Librarian] Scheduling ruby-shadow (>= 0) <git://github.com/37signals/37s_cookbooks.git#master>
[Librarian] Resolving redis2 (>= 0) <http://community.opscode.com/api/v1>
[Librarian]   Accords with all prior constraints
[Librarian] Resolving ufw (>= 0) <http://community.opscode.com/api/v1>
[Librarian]   Accords with all prior constraints
[Librarian] Resolving sudo (>= 0) <http://community.opscode.com/api/v1>
[Librarian]   Accords with all prior constraints
[Librarian] Resolving user (>= 0) <git://github.com/fnichol/chef-user.git#v0.2.0>
[Librarian]   Accords with all prior constraints
[Librarian] Resolving ruby-shadow (>= 0) <git://github.com/37signals/37s_cookbooks.git#master>
[Librarian]   No known prior constraints
[Librarian] Running `git clone git://github.com/37signals/37s_cookbooks.git . --quiet` in tmp/librarian/cache/source/git/441271418b19cd0b9aaf0503eb06472d
[Librarian] Running `git rev-parse HEAD --quiet` in tmp/librarian/cache/source/git/441271418b19cd0b9aaf0503eb06472d
[Librarian]     ->  edefbd17eeb8f25f6d42a01e7f207848fb23e49f
[Librarian] Running `git checkout master --quiet` in tmp/librarian/cache/source/git/441271418b19cd0b9aaf0503eb06472d
[Librarian] Running `git rev-parse HEAD --quiet` in tmp/librarian/cache/source/git/441271418b19cd0b9aaf0503eb06472d
[Librarian]     ->  edefbd17eeb8f25f6d42a01e7f207848fb23e49f
[Librarian]   Checking manifests
/Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/manifest.rb:15:in `manifest_path': undefined method `join' for nil:NilClass (NoMethodError)
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/manifest.rb:15:in `map'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/manifest.rb:15:in `manifest_path'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/source/local.rb:48:in `fetch_manifest!'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/source/local.rb:44:in `manifest'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/chef/source/local.rb:52:in `fetch_version!'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/manifest.rb:34:in `version'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/manifest.rb:30:in `to_s'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:58:in `recursive_resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:110:in `debug'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/helpers/debug.rb:28:in `debug'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/ui.rb:32:in `debug'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/helpers/debug.rb:28:in `debug'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:110:in `debug'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:58:in `recursive_resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:56:in `each'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:56:in `recursive_resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:104:in `scope'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:55:in `recursive_resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:104:in `scope'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:38:in `recursive_resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:23:in `resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/resolver.rb:129:in `resolve'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian.rb:162:in `resolve!'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian.rb:103:in `install!'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/cli.rb:50:in `install'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/thor-0.14.6/lib/thor/task.rb:22:in `send'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/thor-0.14.6/lib/thor/task.rb:22:in `run'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/thor-0.14.6/lib/thor/invocation.rb:118:in `invoke_task'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/thor-0.14.6/lib/thor.rb:263:in `dispatch'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/thor-0.14.6/lib/thor/base.rb:389:in `start'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/lib/librarian/cli.rb:15:in `bin!'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/gems/librarian-0.0.5/bin/librarian-chef:7
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/bin/librarian-chef:19:in `load'
    from /Users/michaelguterl/.rvm/gems/ree-1.8.7-2011.03@rm-chef/bin/librarian-chef:19

The RubyGems page for Librarian points at the 0.0.1 version, instead of the latest 0.0.17.

I like that Librarian just grabs the cookbooks, I have tried submodules and they don't work in this situation - far to much fiddling and untangling the inevitable mess is a real pita.

Nonetheless, it would be nice to allow people to easily contibute changes back to the upstream cookbook. In github this is possible, via the issues API. Essentially Librarian would (non-trivial):

• Build a diff of the current local cookbook from what is pointed to in the Cheffile
• Open an issue and place the diff in the issue. Not ideal I know but at least allows people to get their contributions back in one painless step, i.e. they don't have to have a fork. But they do have to have a Github account.

The command might look like

librarian-chef issue <cookbook-name>

In order to open an issue librarian would need to have access the users Github OAuth2 token, say through ENV['GITHUB_TOKEN']. No token no push of the patch.
Of course this is very Github specific, but other avenues might open later, e.g email the patch.

The idea is to make it dead easy for people to push back what changes they have made that work for them. Cookbook maintaners/authors could pick and chose as they wish.

Thoughts

I just ran into this issue. Here's what I did:

• Edit Cheffile
• Add a cookbook referencing a github repo
• ran librarian-chef install

This resulted in the following:

/Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:102:in `block in run!': StandardError
from /Users/brad/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:208:in `popen_run'
from /Users/brad/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:90:in `popen3'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:101:in `run!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:62:in `block in merge!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `block in within'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `chdir'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `within'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:60:in `merge!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:83:in `block in merge_all_remote_branches!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:81:in `each'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:81:in `merge_all_remote_branches!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git.rb:85:in `cache!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/dependency.rb:31:in `cache_manifests!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/dependency.rb:27:in `manifests'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:49:in `block in recursive_resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:102:in `scope'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:36:in `recursive_resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:21:in `resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver.rb:18:in `resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/action/resolve.rb:25:in `run'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:117:in `resolve!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:67:in `install'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/task.rb:22:in `run'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/invocation.rb:118:in `invoke_task'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor.rb:263:in `dispatch'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/base.rb:389:in `start'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:27:in `bin!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/bin/librarian-chef:7:in `<top (required)>'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/bin/librarian-chef:19:in `load'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/bin/librarian-chef:19:in `<main>'

Immediately re-running librarian-chef install yielded:

/Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:102:in `block in run!': fatal: 'merge' is not possible because you have unmerged files. (StandardError)
Please, fix them up in the work tree, and then use 'git add/rm <file>' as appropriate to mark resolution and make a commit, or use 'git commit -a'.
from /Users/brad/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:208:in `popen_run'
from /Users/brad/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:90:in `popen3'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:101:in `run!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:62:in `block in merge!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `block in within'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `chdir'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `within'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:60:in `merge!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:83:in `block in merge_all_remote_branches!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:81:in `each'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:81:in `merge_all_remote_branches!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/source/git.rb:85:in `cache!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/dependency.rb:31:in `cache_manifests!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/dependency.rb:27:in `manifests'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:49:in `block in recursive_resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:102:in `scope'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:36:in `recursive_resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver/implementation.rb:21:in `resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/resolver.rb:18:in `resolve'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/action/resolve.rb:25:in `run'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:117:in `resolve!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:67:in `install'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/task.rb:22:in `run'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/invocation.rb:118:in `invoke_task'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor.rb:263:in `dispatch'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/thor-0.14.6/lib/thor/base.rb:389:in `start'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/lib/librarian/cli.rb:27:in `bin!'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/gems/librarian-0.0.14/bin/librarian-chef:7:in `<top (required)>'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/bin/librarian-chef:19:in `load'
from /Users/brad/.rvm/gems/ruby-1.9.3-p125@chef-coroutine/bin/librarian-chef:19:in `<main>'

It turns out this repo had multiple remote branches, and deleting everything but origin/master allowed me to successfully run librarian-chef install.

I'm not sure if this is related to Issue #44, but if it is, please feel free to act accordingly.

Thanks!

Maybe I'm not using librarian correctly, but running this never updates my cookbooks:

librarian-chef update cookbook_name

I have to do:

librarian-chef install --clean

Then it updates from my github repository. Am I doing something wrong?

Am I crazy? I've been reading through the code trying to figure it out, but no luck. Seems that in order to this to be a true definition of an infrastructure (that we can rebuild from if ever needed), we need to be able to specify a cookbook version from the :site...

Am I missing something?

Awesome gem though, by the way! Was so pumped when I heard about this...

Each adapter should have its own installation mechanism.

The generic install action should simply verify the preconditions and then delegate to the environment's per-adapter install action.

Idea for a knife plugin:

Commands of the form knife cookbook bundle will essentially be aliases for librarian-chef.

$ knife cookbook bundle resolve #=> librarian-chef resolve
$ knife cookbook bundle update #=> librarian-chef update
$ knife cookbook bundle update apt ssh #=> librarian-chef update apt ssh

Permit:

# Cheffile
cookbook 'my-cookbook',
  :git => 'https://github.com/my-private/cookbook-repo',
  :key => 'deploy-keys/my-private/cookbook-repo.key'

# Gemfile
gem 'my-gem',
  :git => 'https://github.com/my-private/gem-repo',
  :key => 'deploy-keys/my-private/gem-repo.key'

With the directory deploy-keys in this example being a directory relative to the specfile.

Librarian::Source::Git::Repository would construct a git-ssh shell script and set the GIT_SSH environment variable for the git process for these specs.

This is useful if one has a private dependency (gem/cookbook/etc) in a private but separate git repository. One is then able to use the git source and point to the external repository, without being required to vendor the external private dependency into the current project. It is best practice to split distinct software components into distinct repositories, and let Librarian manage resolving all of the dependencies; and this feature would facilitate that.

One problem is: constructing a git-ssh server on the local machine such that deploy keys may be tested.

From Chef:

conflicts
recommends
replaces
suggests

@jtimberman What's the difference between recommends and suggests?

Apologies that this is not a true "issue", but I couldn't think of anywhere else to have this type of discussion.

I'm fairly new to Chef, but I saw librarian mentioned a few times so I set it up for our project. Now I'm at the point where I want to start developing some custom cookbooks. Following the Chef documentation I use knife cookbook create NAME. This works, except for the fact that my work is not being tracked in git because cookbooks directory is being ignored.

Is there a current best practice for handling this? Should I be developing my cookbooks in an outside git repository and then "re-bundling" with librarian each time I want to test my changes? Any advice would be appreciated.

Allow configuration to be remembered in a ./.librarian/ path.

If one command updates a rememberable configuration, the next command should be able to read it back out.

Originally ran into this issue when first setting up, and partially diagnosed it, but then it went away and I couldn't figure out why. This is the error I get:

http://pastebin.com/FkYRCPX5

I got it during the initial librarian-chef install, but then it went away for unknown reasons and now it only appears for librarian-chef update. Seems to involve this entry (but no other comparable entries):

cookbook 'newrelic',
  :path => 'cookbooks-sources'

It seems that newrelic is also a cookbook on the community site (but a different one), so my thought it that this is causing the issue. (The other cookbooks in cookbooks-sources either don't exist on community.opscode.com, or they exist, but my local version is only a slight modification, ie. the deps are likely the same)

Anyhow, I'll just use git repos in the Cheffile for now (as opposed to local refs to git submodules, which is simpler for active dev)

Thanks!

Remove git ls-files from the gemspec.

I think this is one of those religious issues....

I get the value added of 'only release commited files'. But to me having git commands in your gemspec is a "code-smell" (these kids come up with crazy phrases). Essentially I'd argue you should be releasing a pure master or release branch, and that branch should have what is released, no-more-no-less. If you have files, etc that shouldn't be released these should be in the excludefiles for the development branch. Git's branching, squash commits, rebase etc means you can keep release/master neat while all the stuff that shouldn't be released is in the development branch.

See here for some details (no doubt google will throw up more):
http://cogniton-mind.tumblr.com/post/1423976659/howto-gitignore-for-different-branches

This is the error I was getting:

> librarian-chef update
/home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:102:in `block in run!': StandardError
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:208:in `popen_run'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open3.rb:90:in `popen3'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:101:in `run!'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:62:in `block in merge!'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `block in within'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `chdir'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:110:in `within'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb:60:in `merge!'

Had to use --verbose to actually figure out what command it failed on and which directory it was running in...

[Librarian] Running `git checkout master --force --quiet` in ../tmp/librarian/cache/source/git/760ea73281ddef714f6380749d5ef4f0
[Librarian] Running `git merge origin/master --quiet` in ../tmp/librarian/cache/source/git/760ea73281ddef714f6380749d5ef4f0

When I manually inspected that directory, this is what I noticed:

> pushd ../tmp/librarian/cache/source/git/760ea73281ddef714f6380749d5ef4f0
> git st
# On branch master
# Your branch and 'origin/master' have diverged,
# and have 1 and 1 different commit(s) each, respectively.
#
# Unmerged paths:
#   (use "git add/rm <file>..." as appropriate to mark resolution)
#
#   both modified:      libraries/search.rb

IMHO, git merge is too risky to use for automated processes like librarian (and is only meant for interactive use by developers) and git reset --hard should always be used instead. I think that's what capistrano uses too, IIRC...

I fixed it for myself by modifying file ~/.rvm/gems/ruby-1.9.3-p125/gems/librarian-0.0.14/lib/librarian/source/git/repository.rb and changing this:

        def merge!(reference)
          within do
            command = "merge #{reference}"
            run!(command)
          end
        end

into this:

        def merge!(reference)
          within do
            command = "reset --hard #{reference}"
            run!(command)
          end
        end

This might be worth adding since people instinctively seem to think the git protocol is supported:

hedgehog/chef-rbenv@2cdbf04

remove .git folder after a cookbook has been downloaded.
This allows paranoid/fragile production deployments to track their cookbooks without getting tied in any git nested repos morass.

Suited to production deployments where a cookbook change is highly unlikely, and where there is some need to ensure the cookbooks travel with the code base.

I'm trying to use librarian on windows. When it tries to download a cookbook from the community site, it calls "fork" and gives an error:

[Librarian] Installing apache2/1.0.8 <http://community.opscode.com/api/v1>
[Librarian] Caching http://community.opscode.com/api/v1/cookbooks/apache2/versions/1_0_8
c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/chef/source/site.rb:180:in `fork': fork() function is unimplemented on this machine (NotImplementedError)
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/chef/source/site.rb:180:in `cache_version_package!'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/chef/source/site/manifest.rb:73:in `fetch_version_manifest!'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/chef/source/site/manifest.rb:69:in `version_manifest'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/chef/source/site/manifest.rb:80:in `install!'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/action/install.rb:55:in `block in install_manifests'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/action/install.rb:54:in `each'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/action/install.rb:54:in `install_manifests'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/action/install.rb:40:in `perform_installation'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/action/install.rb:12:in `run'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/cli.rb:113:in `install!'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/cli.rb:68:in `install'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/thor-0.14.6/lib/thor/task.rb:22:in `run'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/thor-0.14.6/lib/thor/invocation.rb:118:in `invoke_task'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/thor-0.14.6/lib/thor.rb:263:in`dispatch'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/thor-0.14.6/lib/thor/base.rb:389:in `start'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/lib/librarian/cli.rb:27:in `bin!'
        from c:/Ruby192/lib/ruby/gems/1.9.1/gems/librarian-0.0.12/bin/librarian-chef:7:in `<top (required)>'
        from c:/Ruby192/bin/librarian-chef:19:in `load'
        from c:/Ruby192/bin/librarian-chef:19:in `<main>'

I've tried the win32-process gem with no success.
https://rubygems.org/gems/win32-process

I may be able to avoid this issue by suggesting cygwin as a solution, but still wanted to get this on the radar.

I'm trying to use cookbooks stored in a private repo. Git isn't using my ssh key. Any suggestions for getting around this?

I get this error when I try to run librarian-chef install after adding a new cookbook to the Cheffile in a repo where I have already run this once.

The Cheffile:

#!/usr/bin/env ruby
#^syntax detection

git 'git://github.com/opscode/cookbooks'

%w(
  chef-client build-essential users sudo logrotate rsync zsh git vim tmux
  aws
).each { |cb| cookbook(cb) }
# NOTE: 'aws' was the new piece added before the command failed

cookbook 'mongodb', :git => "git://github.com/edelight/cookbooks"

The output with verbose turned on: https://gist.github.com/97c51a0e98b076b36d50

SSIA.

andrea:chef-repo andreag$ librarian-chef update
tar: Error opening archive: Failed to open '/Users/andreag/Documents/Aptana'
/Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:506:in rename': No such file or directory - /Users/andreag/Documents/Aptana Studio 3 Workspace/chef-repo/tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/mongodb/mongodb or /Users/andreag/Documents/Aptana Studio 3 Workspace/chef-repo/tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/mongodb/ddb10eb171f8093fd208b02b177f6214/package (Errno::ENOENT) from /Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:506:inmove'
from /Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:1402:in fu_each_src_dest' from /Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:1418:infu_each_src_dest0'
from /Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:1400:in fu_each_src_dest' from /Users/andreag/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/fileutils.rb:495:inmove'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/chef/source/site.rb:264:in cache_version_package!' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/chef/source/site.rb:80:infetch_version_manifest!'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/chef/source/site.rb:76:in version_manifest' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/chef/source/site.rb:42:infetch_dependencies!'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/manifest.rb:49:in dependencies' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:62:inrecursive_resolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:104:in scope' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:59:inrecursive_resolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:56:in each' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:56:inrecursive_resolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:104:in scope' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:55:inrecursive_resolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:104:in scope' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:38:inrecursive_resolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:23:in resolve' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/resolver.rb:129:inresolve'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian.rb:158:in resolve!' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/cli.rb:74:inupdate'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/thor-0.14.6/lib/thor/task.rb:22:in send' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/thor-0.14.6/lib/thor/task.rb:22:inrun'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/thor-0.14.6/lib/thor/invocation.rb:118:in invoke_task' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/thor-0.14.6/lib/thor.rb:263:indispatch'
from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/thor-0.14.6/lib/thor/base.rb:389:in start' from /Users/andreag/.rvm/gems/ruby-1.8.7-p352@chef/gems/librarian-0.0.8/lib/librarian/cli.rb:15:inbin!'

Hey,

Just getting started with Librarian, and ran into some weird issues: https://gist.github.com/de43e3ba429f967f31bf

Turns out, my Rubygems was out of date (1.3.6, yeah, I don't get out much). It would be awesome if Librarian could validate the rubygems version and error out with a simple error message for unsupported versions. Or, it could be specified in the gemspec itself.

Thanks!

Think I must have learned it was possible in an IRC or issue convo, but just realized that using :git and :path together for chef-repo-style cookbook repositories isn't documented or obvious. Will try to submit a patch when I think about it.

You're using a Process.waitpid2 + fork combo in site.rb. This isn't really necessary, and it won't work on Windows. Instead, I recommend using the system() method since it works on Windows and it blocks by defualt. Alternatively, you could use open3 if you really need more detailed information on failure. For 1.8 you would need the win32-open3 gem.

I would also suggest replacing a hard coded 'tar' command with an actual tar interface, e.g. zlib, minitar, etc, since you cannot guarantee the presence of tar on Windows (or any *nix platform really).