Http/s proxy issues about bosh-windows-stemcell-builder HOT 7 CLOSED

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/162209414

The labels on this github issue will be updated when the story is started.

from bosh-windows-stemcell-builder.

So the trick is that it must be escaped in ruby so that it outputs to escaped json to be read by packer and just have the quotes left.

Here is the updated line of code that works
@proxy_settings = http_proxy ? "\"#{http_proxy}\" \"#{https_proxy}\" \"#{bypass_list}\"" : ''

from bosh-windows-stemcell-builder.

Hey @RedOnePrime,

Thanks for the detailed notes on these issues and sorry for the delay in response.

Regarding Issue 1: This seems to be a straight forward bug and we'll ensure that https_proxy can be empty with values for the other parameters. You can track the bug progress here: https://www.pivotaltracker.com/story/show/162209414

Issue 2: I've got a couple of clarifying questions regarding this one.

• I can't find an Invoke-Webrequest call under BOSH.utils. The only Invoke-Webrequests I found under the bosh-psmodules are in the install windows updates task where we explicitly apply a few updates that don't get picked up by the update installer. We typically expect users to start with a machine that's up to date when building stemcells but I understand you have your stemcell creation process automated using Packer. I'm wondering if you're using the PS functions to install windows updates as well?
• What powershell version are you using for your windows2012 stemcells? We now ensure our public stemcells use PS 5.1 as PS 4 was giving us issues with symlinks.

Thanks

from bosh-windows-stemcell-builder.

@RedOnePrime Looking for your feedback on this issue!

from bosh-windows-stemcell-builder.

Sorry I did not get, or missed, the notification that someone replied.

Invoke-WebRequest always tries to download and run that patch regardless of the stemcell being up to date because the litmus test is if c:\provision\patch.msu exists; which is never the case because New-Provisioner and Clear-Provisioner steps modify that directory.

The powershell version is up to date on the stemcell. When I posted this issue it was prior to the change in this repo to use ERB so while the issue still existed post-change it might be a slightly different fix.

The overall issue is with proxy. As it relates to direct downloads (as noted in my original post) or with windows updates. The kicker on the windows updates side of things, is that it seems to be a 50/50 whether BITS will succeed with the downloads on 2012r2 and rarely (if ever) with >= 1709. The method the ps-modules uses to download these updates looks to be the recommended process (which involves BITS under the hood), however the Microsoft.Update.Session com object does not give feedback if it has successfully downloaded the updates. Here is some output when it silently fails:

1548688350,,ui,message, vsphere-clone: 2019-01-28T10:05:05.6063476-05:00 Checking For Windows Updates
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:05:05.6375963-05:00 Script: C:\Program Files\WindowsPowerShell\Modules\BOSH.WindowsUpdates\BOSH.WindowsUpdates.psm1
1548688350,,ui,message, vsphere-clone: Script User: WIN-server\Provisioner
1548688350,,ui,message, vsphere-clone: Started: 1/28/2019 10:05:05 AM
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2325148-05:00 There are 2 more updates.
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2481295-05:00 Update for Windows Server 2012 R2 (KB3118401)
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2637528-05:00 Install this update to resolve issues in Windows. For a complete listing of the issues that are included in this update%!(PACKER_COMMA) see the associated Microsoft Knowledge Base article for more information. After you install this item%!(PACKER_COMMA) you may have to restart your computer.
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2793756-05:00 False
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2793756-05:00 True
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2950010-05:00 Update for Windows Server 2012 R2 (KB3162835)
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2950010-05:00 Install this update to resolve issues in Windows. For a complete listing of the issues that are included in this update%!(PACKER_COMMA) see the associated Microsoft Knowledge Base article for more information. After you install this item%!(PACKER_COMMA) you may have to restart your computer.
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2950010-05:00 False
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.2950010-05:00 True
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.3731263-05:00 Evaluating Available Updates with limit of 500:
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.4200046-05:00 Adding: Update for Windows Server 2012 R2 (KB3118401)
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.4356266-05:00 Adding: Update for Windows Server 2012 R2 (KB3162835)
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:10:59.4356266-05:00 Downloading Updates...
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:11:24.5294268-05:00 The following updates are downloaded and ready to be installed:
1548688350,,ui,message, vsphere-clone: 2019-01-28T10:11:24.5450535-05:00 No updates available to install...

You can see at the end that it adds 2 updates and says it is downloading them, but none are in the resultant collection. This means the catch block is not catching something, or the download is not throwing in all scenarios when it (arguably) should; I think the latter.

-Red

from bosh-windows-stemcell-builder.

Hi, @RedOnePrime, sorry for the delay here.

We've created a tracker story for your issue https://www.pivotaltracker.com/story/show/162209414 so you can follow its progress.

from bosh-windows-stemcell-builder.

@RedOnePrime We believe we have provided a resolution for this request. Please feel free to re-open the issue if it hasn't resolved your issue.

from bosh-windows-stemcell-builder.