tknerr / bills-kitchen Goto Github PK

i'm thinking of some basic acceptance tests (e.g. using cucumber) that build the devpack and run the scenarios we describe in the GETTING_STARTED guide.

unfortunately it won't run on Travis because it needs to be built on windows currently. But at least for running them locally it would be great.

Current issues when trying to get tknerr/sample-application-cookbook up and running using bill's kitchen master with omnibus chef and ruby 2.0.

Ruby 2.0 issues:

1. with ruby 2.0 yajl fails (see brianmario/yajl-ruby#116
   • check: updating to bundler > 1.4 might help?
   • re-installing the yajl gem via gem install yajl-ruby --platform=ruby helps
2. similar to the above we now get this ruby 2.0 win32-api error ('require': incompatible library version)
   • re-installing the win32-api gem via gem install win32-api --platform=ruby helps
   • this is one of the reasons why omnibus ships with its own embedded ruby to prevent such issues...
3. with ruby 2.0 64-bit compilation of nokogiri fails, see sparklemotion/nokogiri#864
   • seems to be fixed in master, but not released yet
   • immediate solution: use 32-bit ruby

=> for 1. & 2.: can we somehow force bundler to install with the --platform=ruby flag? AFAIK this is not possible via configuration, but can we maybe fake our platform?

Issues due to omnibus chef:

1. with omnibus our sh "knife ..." shellouts in the Rakefile don't work anymore (can not find gems even thoough they were bundle installed). More general: knife -v (uses omnibus chef) works but bundle exec knife -v (uses gem installed chef) does not.
   • the problem is that two rubies' bindirs are on the path and if it finds the omnibus knife command it will use the omnibus ruby, but the bundle installed gems are in the system ruby!
   • a) use the knife ruby API rather than shelling out
   • b) don't use omnibus - always use chef as a gem

=> using binstubs would probably solve this problem, but it did not work because bundle install --binstubs did not create .bat files?

Specific issues with the Gemfile:

1. faraday 0.9 breaks ridley / berkshelf: berkshelf/ridley#239
   • solution: lock to 0.8.9 in Gemfile

AppVeyor is sweet :-)
https://ci.appveyor.com

Hi, sorry if this is a bit out of the ordinary for an issue, but I'm a touch confused on the install directions, I've installed ruby and ruby dev kit to get the install going, and when I run rake build (after successful returns from gem install bundler and bundle install) it goes for a long bit and seems to 'stall' but I don't know if this is just taking a long time? is there any way to cancel and re-run the command in a more verbose manner?

I'm very new to ruby and chef, thanks for taking the time to read
-Paul

Why? Because it's cleaner and gives us more control over the Ruby and DevKit versions.

Also, we can get rid of the TCS Ruby patch if we use the latest 1.9.3.p385 MRI Ruby. In fact, I noticed that the way I'm patching it removes the DevKit executables like make.exe and sh.exe, so this is broken anyway...

Probably cherry-picking some commits from @dkinzer's hells-kitchen fork

Right now we are checking out some code examples:

...
    %w{ tknerr/bills-kitchen-repo.git       repo/my-chef-repo },
    %w{ tknerr/cookbooks-vagrant-ohai.git     repo/my-cookbooks/vagrant-ohai },
    %w{ tknerr/cookbooks-motd.git         repo/my-cookbooks/motd },
    %w{ tknerr/cookbooks-tdd-example.git    repo/my-cookbooks/tdd-example }
...

Need to check which ones we really need and which ones need to be updated for Vagrant 1.2, Chef 11 and Berkshelf...

I really enjoy using the release and the cmd.exe windows shell in con emu works perfectly, but there are certain times I wish I could use git-bash (and the openssl commands, etc.) and have the environment variables set up in the same manner when I launch it - is there a sort of profile that travels with the portablegit? I was wondering what the development effort would be to construct a shell script or if you have any resources that you could point me in for programming a shell script and I could make apull request.

Thanks again!

Todo's for the 3.0 final release:

• make recreate_dirs work again (57d07c1)
• vagrant docker remote host patch: fix multi-vm portforwarding issue (#110)
• cache baseboxes for faster builds (#93)
• add proxy on/off script (#102)
• conditionally b2d download if iso not present (95e01cf)
• add vagrant-winrm plugin
• update to terraform 0.5

This should come out soon with docker 1.6.0:
boot2docker/windows-installer#80

That way we could finally use kitchen-docker on windows too for faster cookbook testing

As suggested by @dkinzer in #19 I gave ConEmu a try and seems to be much better than Console2 + ansicon

We should not need to download the baseboxes for running the acceptance tests every time.

Let's cache it somewhere, e.g. in target/cache and import them before running the acceptance tests.

=> save some bandwith and lots of time

The .7z package contains the build directory with the bills kitchen contents. It should rather contain the contents directly.

/cc @eric-at-vf

This is really annoying: if you build bill's kitchen using bundle exec rake build it will install the gems to your current Ruby's gem directory rather than to bill's-kitchen's-being-built gem directory.

Seems that even though we use Bundler.with_clean_env the GEM_HOME env var is still present when using bundle exec rake build. It is properly cleared when using rake build.

...not sure if that would work on Windows, but we could give it a try at least:
http://blogs.perl.org/users/smylers/2011/08/ssh-productivity-tips.html

see tknerr/ruby-devpack#2

see tknerr/ruby-devpack#1

• install sublime package manager (Ctrl + Shift + p then start typing pc)
• install plugins:
  • shell turtleman
  • rubydebugger
  • search stackoverflow
  • git
  • git gutter
  • cheat sheets (for sublime shortcuts)
  • sublime repl
  • sublime terminal
• fix doskey alias to prevent Error6 (see here)
  • @doskey vi=START "Sublime Text 2" sublime_text $*
  • in set-env.bat
• fix show-console (will bind to Ctrl + ° on a german keyboard)
  • { "keys": ["ctrl+\\"], "command": "show_panel", "args": {"panel": "console", "toggle": true} }
  • in X:\tools\sublimetext2\Data\Packages\User\Default (Windows).sublime-keymap
• use conemu as terminal via X:\tools\sublimetext2\Data\Packages\User\Terminal.sublime-settings:

{
    "terminal": "ConEmu",
    "parameters": [
        "/Title", "Ruby DevPack", "/cmd", "cmd /K \\set-env.bat"
    ]
}

• disable live mode of git gutter, otherwise it behaves laggy. In X:\tools\sublimetext2\Data\Packages\User\GitGutter.sublime-settings:

{
  "live_mode": false
}

• define custom builds, e.g.
  • for rakefile => rake
  • for Gemfile => bundle install
  • ...
    see http://addyosmani.com/blog/custom-sublime-text-build-systems-for-popular-tools-and-languages/

...instead of using the chef-server-on-ubuntu-12.04-server-amd64-vagrant vagrant basebox, so that

1. we can reuse the existing ubuntu-12.04-server-amd64-vagrant.box basebox and don't need an additional one
2. we make the chef-server installation reproducible and explicit

Create new issue since I made mistakes on the past two (sorry :[ )

1.) Got and installed ruby-devpack release successfully.
2.) mounting ruby dev pack on X:\ and using set-env successfully
3.) rake build in cmd fails (https://gist.github.com/paul42/0684a80da8981fbf4992#file-rake-build-errors) the rake build actually finishes now! but it's got a ton a failures
4.) Attempting build in git bash, then I'll attempt build in powershell to see if there are any noticeable differences

I downloaded the release and did the git clone for tests and these are the errors I'm getting - is there a way I can build native ruby to avoid these native gem builds failing?

Failures:

1. usage scenarios developing top-level cookbooks installs gem dependencies via bundle install
   Failure/Error: cmd_succeeds("cd #{APP_COOKBOOK_DIR} && bundle install")

   expected true
        got false
 # ./spec/helpers.rb:33:in `cmd_succeeds'
 # ./spec/acceptance/usage_scenarios_spec.rb:46:in `block (3 levels) in <top (required)>'

2. usage scenarios developing top-level cookbooks runs the unit tests via rake test
   Failure/Error: cmd_succeeds("cd #{APP_COOKBOOK_DIR} && rake test")

   expected true
        got false
 # ./spec/helpers.rb:33:in `cmd_succeeds'
 # ./spec/acceptance/usage_scenarios_spec.rb:49:in `block (3 levels) in <top (required)>'

3. usage scenarios developing top-level cookbooks runs the integration tests via rake integration
   Failure/Error: cmd_succeeds("cd #{APP_COOKBOOK_DIR} && rake integration")

   expected true
        got false
 # ./spec/helpers.rb:33:in `cmd_succeeds'
 # ./spec/acceptance/usage_scenarios_spec.rb:52:in `block (3 levels) in <top (required)>'

Finished in 13 minutes 52 seconds (files took 0.33793 seconds to load)
15 examples, 3 failures

Failed examples:

rspec ./spec/acceptance/usage_scenarios_spec.rb:45 # usage scenarios developing top-level cookbooks inst alls gem dependencies via bundle install
rspec ./spec/acceptance/usage_scenarios_spec.rb:48 # usage scenarios developing top-level cookbooks runs the unit tests via rake test
rspec ./spec/acceptance/usage_scenarios_spec.rb:51 # usage scenarios developing top-level cookbooks runs the integration tests via rake integration

W:\repo\vagrant-workflow-tests>

also is there a way to do this for the 'rake build' from source? I love the tail on acceptance.log, and was wondering if there is one for the rake build in the source file?

Hey tknerr,

Thank you very much for creating Bills-kitchen, i took me days to try to get vagrant+chef to work on windows until i found this repo. The overall experience is AWESOME. there is only two gotchas for me, not sure if it is because i didnt read the documentation careful enough, but anyway, hope this helps others.

1. you need to clone this repo to some directory in windows but make sure the full PATH does not have space, like you cant put it under c:\vagrant chef test. the reason is because when you "rake build" trying to clone repos it will failed.
2. the default password for chef server webui is chefchef. and not p@ssw0rd1. didnt know it until i realized that the chef server was created using "knife server" and i found this page https://github.com/fnichol/knife-server#--webui-password-password

Once again, thank you
P.S, is your real name bill? ^__^

Yay - first Chef-DK alpha for windows is out :-)
https://twitter.com/julian_dunn/status/479416718712766464

Want to integrate ChefDK in favor of Omnibus Chef, thereby dropping the additional Ruby (no longer needed) and updating to the latest Vagrant version, so we are everything Ruby 2.x

May need to keep the DevKit though but use it with the Chef-DK Ruby.

See also @jtimberman 's blog post on Chef-DK:
http://jtimberman.housepub.org/blog/2014/04/30/chefdk-and-ruby/

Running the example in the Getting Started file and everything works fine until I do the 'rake install'. This step fails with:

rake aborted!
cannot load such file -- ruby-wmi

Google revealed that I need to 'gem install win32-open3, ruby-xml, windows-api, windows-pr --no rdoc --no-ri --verbose'
and this failed with:

ERROR: Error installing win32-open3:
win32-open3 requires Ruby version < 1.9.0

I've tried 'gem install win32-open3-19' as a work-around but still seeing an error message 'cannot load such file -- win32/service' when I attempt the rake install.

After running git-bash I get this error when using bundle:

$ bundle install
bash: /c/Repos/_github/bills-kitchen/target/build/tools/chefdk/embedded/bin/bundle: C:/opscode/chefdk/embedded/bin/ruby.exe: bad interpreter: No such file or directory

I preferred hosting the forked/patched gems on gemfury because bundler has a slightly different way of installing gems via a rubygems-like source vs via :git. One of the side effects is that gem list does not show the :git installed gems which might be confusing if you are not aware of this bundler "feature".

Even though this was my previous rationale, I guess I will change it to :git references because:

• it's more transparent for users and easier to change
• removes the hassle to create gemfury fake accounts anymore to bypass the two gem limit for free users ;-) and uploading the gem there

If someone has good reasons for keeping the gemfury approach pls let me know.

Hi, could you tell me what im doing wrong here ?

==> default:   - Exclude: [".vagrant/", ".git/", "node_modules", "bower_components"]
      1 [main] rsync 8052 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer.  Please report this problem to
the public mailing list [email protected]
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /w/repo/erewhon/
Guest path: /src
Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i 'W:/repo/erewhon/cookbook/.vagrant/machines/default/virtualbox/private_key' --exclude .vagrant/ --exclude .git/ --exclude node_modules --exclude bower_components /w/repo/erewhon/ [email protected]:/src
Error:       1 [main] rsync 8052 find_fast_cwd: WARNING: Couldn't compute FAST_CWD pointer.  Please report this problem to
the public mailing list [email protected]
Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
rsync: change_dir "/w/repo/erewhon" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/main.c(1052) [sender=3.0.9]

this is the line in my vagrant file responsible for rsync

  config.vm.synced_folder "..", "/src/", type: :rsync, rsync__exclude: [".git/", "node_modules", "bower_components"],
                          rsync__auto: true

the git user name is set to dummy. No asking during first run of set-env.bat.

For example:

1. use vagrant-wrapper 2.0.1 in Gemfile => works
2. use vagrant-wrapper from git branch in Gemfile => broken
3. back to using vagrant-wrapper 2.0.1 in Gemfile => now broken too!!

The reason is:

• the git ref bundle install creates the binary in C:\Repos\_github\bills-kitchen\target\build\tools\chefdk\embedded\lib\ruby\gems\2.0.0\bin\vagrant.bat
• the non-git ref gem bundle install creates it in C:\Repos\_github\bills-kitchen\target\build\tools\chefdk\embedded\bin\vagrant

The former one is broken in that it expects a ruby.exe in the same directory. Not sure how to fix this. Seen this in ChefDK too and fixing it here at build time.

Not sure how to fix it yet...

Details

With 2.0.1 in the Gemfile it works:

source 'https://rubygems.org'

gem 'vagrant-wrapper', '2.0.1'

Output:

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle install
Resolving dependencies...
Using vagrant-wrapper 2.0.1
Using bundler 1.7.5
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>vagrant --version
Vagrant 1.7.2

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle exec vagrant --version
Vagrant 1.7.2

With 2.0.2.beta1 from the PR branch in the Gemfile it does not:

source 'https://rubygems.org'

gem 'vagrant-wrapper', '2.0.2.beta1',
    :git => 'https://github.com/criteo/gem-vagrant-wrapper.git',
    :ref => 'fix_exec'

Output:

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle install
Updating https://github.com/criteo/gem-vagrant-wrapper.git
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
Using vagrant-wrapper 2.0.2.beta1 from https://github.com/criteo/gem-vagrant-wrapper.git (at fix_exe)
Using bundler 1.7.5
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>vagrant --version
Vagrant 1.7.2

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle exec vagrant --version
'"C:\Repos\_github\bills-kitchen\target\build\tools\chefdk\embedded\lib\ruby\gems\2.0.0\bin\ruby.exe"' is not recognized as an internal or external command,
operable program or batch file.

Back to 2.0.1 it is now broken too!

Gemfile:

source 'https://rubygems.org'

gem 'vagrant-wrapper', '2.0.1'

Output:

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle install
Resolving dependencies...
Using vagrant-wrapper 2.0.1 (was 2.0.2.beta1)
Using bundler 1.7.5
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>vagrant --version
Vagrant 1.7.2

C:\Repos\_github\bills-kitchen\target\build\repo\vagrant-workflow-tests\playground\tmp>bundle exec vagrant --version
'"C:\Repos\_github\bills-kitchen\target\build\tools\chefdk\embedded\lib\ruby\gems\2.0.0\bin\ruby.exe"' is not recognized as an internal or external command,
operable program or batch file.

The README is long out of date and needs a major overhaul.

Maybe we should also reconsider the GETTING_STARTED docs, not sure if here is the best place for them. I'd rather have some automated tests (see #4) that fail if something breaks instead of docs that become potentially outdated with every new release...

Why?

• not sure if anyone really uses it
• always forget to align it with set-env.bat
• it's not tested at all and I honestly can't support it

If anyone needs or uses it please let me know and I'll keep it in (unsupported though). If not, I'll clean up and remove it.

Opscode released the (DRY-ed up) Veewee scripts for their baseboxes in the bento project:
https://github.com/opscode/bento

Also, they are hosting their baseboxes on S3. We should re-use the bento baseboxes if possible, because they a) are more standard than our custom ones and b) we don't need to host them on dropbox.

Note of caution: they don't have bare-OS baseboxes yet (even though that would be easily possible with bento now)

for that we need:

• examples for cookbook testing with cucumber
• examples how to run foodcritic
• examples for role/node testing with cucumber
• the relevant gems

and potentially:

• examples for cookbook testing with minitest-handler and chefspec?
• get it running on Travis-CI

Hi i just installed bills-kitchen, let me say this first, i am really hopefull that i can finally start using vagrant + chef on my windows machine, this has been a huge problem for a very long time for me.

Now i try to execute vagrant and its complaining that cygpath is not found.

W:\>which cygpath                                                                                                                                         
which: no cygpath in (.;W:\tools\chefdk\bin;W:\home\.chefdk\gem\ruby\2.0.0\bin;W:\tools\chefdk\embedded\bin;W:\tools\consul;W:\tools\packer;W:\tools\terra
form;W:\tools\vagrant\HashiCorp\Vagrant\bin;W:\tools\portablegit\cmd;W:\tools\portablegit;W:\tools\kdiff3;W:\tools\cygwin-rsync;W:\tools\cygwin-ssh;W:\too
ls\vagrant\HashiCorp\Vagrant\embedded\bin;W:\tools\conemu;W:\tools\atom\Atom\resources\cli;W:\tools\atom\Atom\resources\app\apm\bin;W:\tools\putty;C:\Prog
ram Files\Oracle\VirtualBox\;;W:\tools\devkit\bin;W:\tools\devkit\mingw\bin;W:\tools\conemu;W:\tools\conemu\ConEmu;C:\Program Files (x86)\NVIDIA Corporati
on\PhysX\Common;C:\ProgramData\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:
\HashiCorp\Vagrant\bin;C:\opscode\chefdk\bin;C:\Program Files\nodejs\;C:\Users\Nick\Documents\cwRsync_5.4.1_x86_Free;C:\ProgramData\chocolatey\bin;C:\Prog
ramData\chocolatey\lib\rsync\content\cwRsync_5.4.1_x86_Free;C:\Users\Nick\.babun;C:\Users\Nick\AppData\Roaming\npm)                                       

Das Devkit ließ sich von github nicht herunterladen, das redirect ging ins leere. Ich vermute, dass es ein Problem mit den protokollen ist, github macht ja https statt http.

Need to check what the licensing requirements are so that we can host bills kitchen as a binary (e.g. on bintray.com)

Using clink you get command autocompletion, command history and GNU Readline command editing features on windows as well.

It can be easily integrated with ConEmu, see here:
https://code.google.com/p/conemu-maximus5/wiki/ConEmu_Clink

Thanks @eric-at-vf

Are we meant to download it ourselves and place it in tools/conemu?

...so that we can use berks viz

Recently sublime text fails with the message

"This build of Sublime Text 2 has expired, please update to a newer one from http://www.sublimetext.com"

An update to the latest 2.0.1 version should help...

Super annoying: since quite a while (feels like it has ever been that way) vim is not usable in ConEmu via vagrant ssh.

Not sure whether this is something ConEmu specific or plain Vagrant / Windows specific. It might have to do with how we are setting TERM in set-env.bat, but this is just a guess.

we have set TERM=msys in set-env.bat as a fix for this:
http://code.google.com/p/msysgit/issues/detail?id=184

however, this breaks the terminal when using vagrant ssh (e.g. ctrl+r does does not work, no syntax coloring in vi, and other weirdnesses)

On Windows some root certificates are not installed, leading to many SSL errors, e.g. in:

• chef-boneyard/vagrant-omnibus#64
• hashicorp/vagrant#3036
• berkshelf/berkshelf#1030

In order to fix this one needs to download the cacert.pem and set the SSL_CERT_FILE env var accordingly, as described here:
https://gist.github.com/fnichol/867550

We should add it with along with an update-cacert-root-certs.bat script.

just tried the precompiled patched ruby 1.9.3 binaries from thecodeshop, and it improved the load performance by > 50%, awesome stuff. need to get that into bills kitchen.

http://itreallymatters.net/post/12897174267/speedup-ruby-1-9-3-on-windows#.UL0EXIddNyw
https://github.com/thecodeshop/ruby/wiki/Downloads

Currently we are bundling SublimeText (trial version), for which we have written consent that this is okay if the package is used for workshops, trials, etc.

However, it still forces the user to obtain a SublimeText license, which is probably well invested but might also be blocking people from using it.

Atom.io looks quite similar and is a truly open source and free alternative:
https://atom.io/

Right now it can not be made portable for windows, though:
atom/atom#2939

Once this is fixed it should replace SublimeText in Bills Kitchen

After updating the whole toolset the README, GETTING_STARTED and COOKBOOK_DEVELOPMENT guides need an update

On Windows the require's seem to take quite long, which is pretty noticeable when you run knife -v for example. It takes about 6s to respond right now on my Windows 7 machine.

Ohai plugins being loaded might also play a role in this, see here:
http://comments.gmane.org/gmane.comp.sysutils.chef.user/979

First tests with the faster_require gem yielded a 4x performance improvement, e.g. knife -v now takes 2s vs 6s, rspec spec in a chefspec project takes 4s instead of 16s. This feels still slow, but is already much better.

The idea is to install the faster_require gem globally as described in the README, so we don't pollute our ruby code with require 'faster_require' and there's a single switch to turn it off.

...as it has more features that we could need, especially:

• direct references via :path shortening the development cycle
• reading cookbook dependencies from metadata.rb

right now it does not work on windows yet (due to dependency to gecode) but that should be fixed with the 0.4.0 release