knife download fails with "undefined method `child' for nil:NilClass" about chef HOT 30 CLOSED

+1

from chef.

I'm running into this issue too, whilst attempting to pull all information from elderly production Chef 10 servers in preparation for building shiny new Chef 11 (or perhaps 12) servers.

from chef.

Any progress on this issue? It is causing us trouble as we plan on upgrading to V12. Is there a release of the Chef Client V11 we can fall back to that this doesn't happen?

from chef.

Confirmed today that neither chef 11 nor chef 12 work using knife download against my production chef 10 servers, so https://www.chef.io/blog/2013/03/12/5106/ doesn't work. We have now formally raised this issue with our representative at Chef.io

from chef.

It would be helpful if you all could provide us with more information. Which versions of chef client are you seeing this failing and against which chef server versions? Are there combinations that it works? Having this information would be helpful in narrowing things down.

from chef.

This error happens with the latest versions of both Chef 11 and Chef 12 acting as download clients against servers running Chef server 10.18.2.

In all cases, the -VV output ends like the following (a null member 'child' is accessed in file_system.rb when recursively copying a directory structure). --concurrency 1 has no effect

/opt/chef/embedded/apps/chef/lib/chef/chef_fs/file_system.rb:323:in `block in copy_entries': undefined method `child' for nil:NilClass (NoMethodError)
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `call'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `process_input'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:257:in `process_one'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:195:in `each_with_exceptions_unordered'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:74:in `wait'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer.rb:48:in `parallel_do'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer.rb:29:in `parallel_do'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/file_system.rb:425:in `parallel_do'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/file_system.rb:322:in `copy_entries'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/file_system.rb:360:in `block in copy_entries'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `call'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:267:in `process_input'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer/parallel_enumerable.rb:257:in `process_one'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer.rb:93:in `call'
    from /opt/chef/embedded/apps/chef/lib/chef/chef_fs/parallelizer.rb:93:in `worker_loop'

from chef.

@mihalis68 Is this issue only with Chef server 10? There is some question if knife download ever worked with Chef Server 10. I'll have to check into that. Is anyone else seeing this against a server other than 10, or is this all related to 10?

Would it be helpful to get some guidance on how to migrate from 10 to 11 or 12?

from chef.

I've answered my own question that knife download did work with chef 10 once upon a time, as here is the blog post that talks about using it to upgrade from chef 10 to 11. https://www.chef.io/blog/2013/03/12/5106/

We'll work on diving deeper into this.

from chef.

Yes I mentioned that blog posting previously here in this ticket.

from chef.

@mihalis68 I over looked it. Sorry about that.

from chef.

No worries! I'm optimistic the fix may be fairly simple. Let me know if there's anything I can help test.

from chef.

Just to clarify - everyone in this ticket having issues with Chef Server 10 is using the open source version? If you happen to be using the Enterprise version, let us know, as we have different tooling for that version that might help you move forward.

In the mean time we'll focus on solving this issue assuming the open source 10 server.

@mihalis68 We might take you up on that offer, as we don't exactly have many open source 10 servers around and that was before our omnibus days. We'll let you know.

from chef.

Yes, open source chef 10 server. It's version 10.18 IIRC because that was the last version packaged as a .deb file and published on apt.opscode.com - it took a while to get away from that since we had tooling around installing chef from a local mirror of that since our production chef servers can't expect Internet access

from chef.

@mihalis68 Can you post the the entire debug log (with -VV and --concurrency 1) somewhere? If you don't want to post it here, you can email it to me at [email protected].

from chef.

Yes absolutely I'll dig that out now

from chef.

I don't see --concurrency 1 or --concurrency=1 actually change the output at all. Is that correct syntax? In particular the traceback at the end always shows the individual filesystem operations being fed into parallel_do

from chef.

I have sent the transcript to the email suggested above. Could you at least let me know if it got through. Our email system likes to eat emails with attachment for inscrutable reasons.

from chef.

@mihalis68 --concurrency 1 should be right. https://docs.chef.io/knife_download.html, so if the output didn't change it's nothing you've done wrong.

from chef.

@mihalis68 Got your email, checking it out now. Thanks!

from chef.

@mihalis68 Can you check something for me? Do you have the knife-essentials gem installed on your system that you're running knife download from? If so, can you uninstall that gem and try again?

from chef.

No it's not a stray gem, for testing I made two brand-new single-purpose VMs with Ubuntu 12.04 and Chef 11 and Chef 12 respectively. The exception traceback from the error is all from /opt/chef/embedded no other paths.

Please do let me know if you suspect that verson of ubuntu however, testing with 14 would be quite straightforward.

from chef.

@mihalis68 Thanks. I asked about knife-essentials because it showed up on a test system and caused this type of error, so I wanted to double check, even though I suspected it wasn't the case based on the output you sent. I highly doubt this is related to ubuntu version, but we'll let you know if our thinking changes on that.

from chef.

I think I have worked out part of what is happening. I hacked up a bunch of the knife source code with extra trace and after quite a bit of fiddling found that the local path for cookbooks is expanded to nill. This seemed to be straight from the chef config.

It turns out, my knife-chef10.rb file contained the line

cookbook_path nil

as per the blog. This causes it to blow up. Removing this line allowed something like this to work

knife download -c .chef/knife-chef10.rb cookbooks

to succeed. W00t!

However a full download of '/' still fails. A quick bash one-liner, however, seems to get the job done:

for STUFF in clients cookbooks data_bags environments nodes roles users; do echo "Downloading $STUFF"; echo; echo ;knife download -c .chef/knife-chef10.rb $STUFF; done

Ran to completion and initial spot checks on the resulting chef data directories all look good!

from chef.

Actually this one-liner may not even be necessary. Taking out some of the hacks I had in knife to diagnose the issue with cookbook_path I am seeing knife download / work as expected. Will try a clean install to verify.

from chef.

Based on my testing, @mihalis68 is right. The cause of this error is having cookbook_path set to nil in the knife.rb. This bug can be worked around by either ensuring that is set or by using the --chef-repo-path option, which is essentially another way to set the cookbook_path for knife download via the command line.

If anyone else is able to verify these work arounds work, it would be appreciated. In the mean time, we'll try to track down why having the cookbook_path being set to nil doesn't work or else make the error message better so it is clear this is the case.

from chef.

All right, so to close this out. This bug is caused when the cookbook_path is set to nil. Basically, never set cookbook_path to nil. The reason this used to work back before Chef 11.8.0 is that in 11.6.2 we had this code: https://github.com/chef/chef/blob/11.6.2/lib%2Fchef%2Fchef_fs%2Fconfig.rb#L191-L199 that would set the paths for values that were not set, and this would ensure that the nil value got overwritten.

With 11.8.0 this code was heavily refactored so that the setting of those values was done in the base config and this check ended up being removed. Instead of adding this check back, the way we intend to fix this is to add validation code to mixlib config that enables default value checking to occur (that would validate that a value isn't nil, for instance).

Until that is added, the work around is don't set any values to nil. We will also update the upgrading from Chef 10 blog post to no longer recommend setting the cookbook_path to nil, since that is now bad advice.

I'd like to thank @mihalis68 for his persistence in helping us with looking into the problem and it was his work that ultimately narrowed this down to to cookbook_path being set incorrectly.

I'm going to close this bug out now and we'll be adding a task to add the validation checking and will update the blog post, as already noted.

from chef.

The Chef blog post with the instructions for upgrading from Chef server 10 has been updated and has a note added to it to not set the cookbook_path to nil. https://www.chef.io/blog/2013/03/12/5106/

from chef.

Unfortunately in updating the Blog post I think you have disturbed the formatting of the knife configuration files. I am now seeing things like this :

transfer<em>repo = File.expand</em>path('..', File.dirname(<strong>FILE</strong>))
chef<em>server</em>url &quot;https://chef-11.example.com&quot;
node<em>name 'admin'
client</em>key &quot;#{transfer<em>repo}/.chef/admin.pem&quot;
repo</em>mode 'everything'
versioned<em>cookbooks true
chef</em>repo<em>path transfer</em>repo

it appears to have been html-escaped

from chef.

@mihalis68 Thanks for the heads up. I've updated the post to fix that. It turns out a markdown plugin that wasn't used on the blog when the post was originally written was messing up the post when it was updated.

Let me know if something else should be corrected.

A link to the blog, just to make it easy to find: https://www.chef.io/blog/2013/03/12/5106/

from chef.

That now looks perfect!

from chef.