tylert / packer-build Goto Github PK

Bionic Beaver VirtualBox image with Packer 1.2.4 crashes. See:
for crash log. Basic issue seems to be with download of iso.

Reverting to Packer 1.2.3 succeeds but generates following message in same area as crash in 1.2.4:
vbox: Downloading or copying: http://myserver:8080/ubuntu/ubuntu-18.04-server-amd64.iso vbox: Error downloading: Get http://myserver:8080/ubuntu/ubuntu-18.04-server-amd64.iso: dial tcp: lookup myserver on 127.0.0.53:53: server misbehaving vbox: Downloading or copying: http://cdimage.ubuntu.com/releases/bionic/release/ubuntu-18.04-server-amd64.iso

I love what you've got going here, the configs are more minimal and tailored to exactly the required boot arguments on newer releases like Xenial (which still seems to suffer from a short kernel args list) but I wonder if it would be more effective to add some of your stuff into boxcutter/ubuntu as well. You seem to have a good handle on the trickier versions like 15.10 and 16.04.

I was looking for a way to build a minimal Ubuntu desktop and tried this repo. Unfortunately all I get is this:

I am running on a Windows 7 64bit machine with recent versions of Packer, Vagrant und VirtualBox. The command above was run in cmder.

Undoubtedly I'm doing stupid, but my attempt to generate a bionic based VirtualBox is failing with the following message:
`==> vbox (vagrant): Creating Vagrant box for 'virtualbox' provider
vbox (vagrant): Unpacking OVA: build/2018-05-26-18-12-14/base-bionic.ova
vbox (vagrant): Renaming the OVF to box.ovf...
vbox (vagrant): Using custom Vagrantfile: source/ubuntu/bionic/base.vagrant
vbox (vagrant): Compressing: Vagrantfile
==> vbox: Running post-processor: shell-local
==> vbox (shell-local): Post processing with local shell script: /tmp/packer-shell679910228
Build 'vbox' errored: 1 error(s) occurred:

• Post-processor failed: archive/tar: cannot encode header: Format specifies GNU; and GNU cannot encode Uname="[email protected]"`

How is packer-build determining the tar file name? Is there a variable I can set.

Thx,

-steve

I hope this is what you are looking for... add this to the seed file to force the EFI override

d-i partman-efi/non_efi_system boolean true

I may be missing something pretty obvious here, but I can not find qemu.sh and vbox.sh under .scripts/. Any idea?

I appreciate that this is possibly a stupid question, but have you successfully built a Bionic image with Packer? I've been going around in circles trying to get mine to work (including pinching ideas from your configs), with no joy. I did come across a comment on this pull request which worried me:

If what you want is to automate server installs of Ubuntu for non-test purposes, that should be done with MAAS, not with subiquity.

From that, it reads like they've ditched the expert mode installer and they're trying to push MAAS for actual installs instead - if that's the case then it's going to be a massive problem for those of us who want to use preseeds!

Running the xenial build failed for me when the "dd" command eventually errors out due to lack of diskspace. I had to add "|| true" or "||:" to make this work for me.

•    "dd if=/dev/zero of=/ZEROFILL bs=4M",
  

•    "dd if=/dev/zero of=/ZEROFILL bs=4M ||:",
  

Nice new features like hashicorp/packer#2600 will require non-backwards-compatible template changes.

We need to set a minimum version for the templates. This work has already been started but requires merely to finish merging in the template changes that will allow the use of this new feature.

At the moment, the latest xenial builds prompt for postfix configuration during the install. Need to get past this step.

I'm running ./scripts/vbox.sh ubuntu/xenial/base-uefi.json and it freezes on GRUB menu.

I have tried ./scripts/vbox.sh ubuntu/xenial/base.json, all works good.

For Ubuntu systems FS0:\EFI\debian\grubx64.efi should be FS0:\EFI\ubuntu\grubx64.efi.

I have not used packer yet , and am on OSX, not Linux.
Would you be able to comment on the use case I have below, so I can get a high level view of if this code could help me ??

I need to create USB bootable thumb drives of 3 types:

Windows 10, with some programs already installed

Raspberry pi.
Same software on it as the windows 10 box

Ubuntu 16.04 server. With postresql and some golang executables.

It would be super awesome if I could make vagrant boxes / qemu boxes of the same also. Then I can test in a vm, then make the bootable USB images as needed or maybe even as part of CI

Thanks in advance

If folks want to speed up builds, setting up an apt-cacher-ng server is (almost) a single command operation. What is missing is the magic to ensure that this caching server can be properly used while building new VM images. Adding a few notes in the README would be a good first step.

According to a hint found at hashicorp/packer#1047 (comment), packer templates can be made a lot simpler using "node anchors" in YAML to generate the large JSON blobs (DRY = Don't Repeat Yourself).

Ideally, rather than store the large, ugly JSON blobs, it may be preferable to store a smaller YAML blob instead and try to use some templating engine such as Jinja to work around the problem of packer not having a good way to nest user variables.

The new shell-local post-processor is coming in Packer 0.9.0 (#4). When it does, it would be nice to generate the JSON version payload complete with checksums.

This JSON version payload template is outlined in the README.rst file checked into the latest dev branch.

Vagrant only supports up to sha256 so don't bother making sha512 checksums.

Any ideas why I get

2020/12/16 10:24:05 packer-builder-qemu plugin: [INFO] Attempting SSH connection to 127.0.0.1:3514...
2020/12/16 10:24:05 packer-builder-qemu plugin: [DEBUG] reconnecting to TCP connection for SSH
2020/12/16 10:24:05 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2020/12/16 10:24:15 packer-builder-qemu plugin: [DEBUG] SSH handshake err: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

Is there something to change here?

 ssh_password: 1ma63b0rk3d
  ssh_password_crypted: '$6$w5yFawT.$d51yQ513SdzariRCjomBwO9IMtMh6.TjnRwQqTBlOMwGhyyVXlJeYC9kanFp65bpoS1tn9x7r8gLP5Dg4CtEP1'

I saw some posts on communities, but nothing relevant. I experience the same from ubuntu1804 or osx.

Edit:
Ok, that tooks me a while to understand that packer needs time to finish ≈40min
Everything seems to be ok now.

Currently, the image files generated when using the QEMU builder produces 2 files named, for example, base-jessie64 and base-jessie64.raw.gz.

It would be nicer to add a suffix of .img and .img.gz, respectively, to these files instead.

@tylert at first thanks for providing this great collection of build-scripts!
I appreciate your work and try to "fork" my own builds from this repo.

Unfortunately things seem not to work and I wonder if it's my fault or not.

I don't use the tool vagrant itself, so I run packer "manually" within packer-build/source/debian/11_bullseye.

I edited some variables to fit my local environment:

 git diff base.pkr.hcl
diff --git a/source/debian/11_bullseye/base.pkr.hcl b/source/debian/11_bullseye/base.pkr.hcl
index e03063c9..c2df116b 100644
--- a/source/debian/11_bullseye/base.pkr.hcl
+++ b/source/debian/11_bullseye/base.pkr.hcl
@@ -15,7 +15,7 @@ packer {
 
 variable "apt_cache_url" {
   type    = string
-  default = "http://myserver:3142"
+  default = "http://ivy.loc.oops.co.at:3142"
 }
 
 variable "boot_wait" {
@@ -111,7 +111,8 @@ variable "iso_path_external" {
 
 variable "iso_path_internal" {
   type    = string
-  default = "http://myserver:8080/debian"
+  #default = "http://myserver:8080/debian"
+  default = "/mnt/platz/isos/debian"
 }
 
 variable "keep_registered" {
@@ -156,7 +157,8 @@ variable "packer_cache_dir" {
 
 variable "preseed_file" {
   type    = string
-  default = "template/debian/11_bullseye/base.preseed"
+  #default = "template/debian/11_bullseye/base.preseed"
+  default = "base.preseed"
 }
 
 variable "qemu_binary" {
@@ -196,7 +198,7 @@ variable "ssh_file_transfer_method" {
 
 variable "ssh_fullname" {
   type    = string
-  default = "Ghost Writer"
+  default = "vagrant"
 }
 
 variable "ssh_handshake_attempts" {
@@ -211,7 +213,7 @@ variable "ssh_keep_alive_interval" {
 
 variable "ssh_password" {
   type    = string
-  default = "1ma63b0rk3d"
+  default = "vagrant"
 }
 
 variable "ssh_port" {
@@ -231,7 +233,7 @@ variable "ssh_timeout" {
 
 variable "ssh_username" {
   type    = string
-  default = "ghost"
+  default = "vagrant"
 }

I only run the vbox-build now:

PACKER_LOG=1 PACKER_LOG_PATH=packer.log packer build -only=virtualbox-iso.vbox base.pkr.hcl

The VM gets created and started, but always fails at setting the keyboard somehow.
I fiddled with settings in the preseed file already, but set them back again.

Do you have positive results with this build currently? Might there be any changes in packer or debian maybe?

In order to do GPU passthrough I need to run the virt-install as:

sudo virt-install \ --name ubuntu-vm \ --boot uefi \ --machine q35 \ --host-device 4b:00.0 --host-device 4b:00.1 \ ... ...

Does this packer script support this? I am not quite sure where to look to work out what the equivalent virt-install bit is

When I run ./script/generate_templates.sh, packer fix (packer 1.4.2) generates this error:

Error! Fixed template fails to parse: Failed to cast root level comment value to string

This is usually caused by an error in the input template.
Please fix the error and try again.

A general question:

Did you lately test base-uefi.pkr.hcl ? I try to get that working (Debian 11.3) with VirtualBox on my Fedora system and see various issues. The boot_command seems not to work anymore etc.

Just asking if that build works for you, @tylert ... thanks!

packer version 1.8.3 is throwing validation errors.

For example on Ubuntu 22.04 Jammy base.pkr.hcl:

❯ packer validate base.pkr.hcl
Error: Unsupported block type

on base.pkr.hcl line 4, in packer:
4:   required_providers {

Blocks of type "required_providers" are not expected here.

Error: Unsupported block type

on base.pkr.hcl line 4, in packer:
4:   required_providers {

Blocks of type "required_providers" are not expected here.

Format has changed (for virtualbox for example)
from

   required_providers {
      virtualbox = {
         source  = "github.com/hashicorp/packer-plugin-virtualbox"
         version = ">= 1.0.0, < 2.0.0"
      }
   }

to

  required_plugins {
     virtualbox = {
        source  = "github.com/hashicorp/virtualbox"
        version = ">= 1.0.0, < 2.0.0"
     }  
  }

roar-alpha@localhost:~/packer-build/source/ubuntu/22.04_jammy$ packer build base.pkr.hcl 
Error: Error in function call

  on base.pkr.hcl line 305:
  (source code not available)

with var as object with 52 attributes,
     var.user_data_location as "user-data".

Call to function "templatefile" failed: user-data:119,20-29: Unsupported
attribute; This object does not have an attribute named "username"., and 1 other
diagnostic(s).

Error: Error in function call

  on base.pkr.hcl line 364:
  (source code not available)

with var as object with 52 attributes,
     var.user_data_location as "user-data".

Call to function "templatefile" failed: user-data:119,20-29: Unsupported
attribute; This object does not have an attribute named "username"., and 1 other
diagnostic(s).

22.04 packer template fails to build may I know how should I debug this?

Packer version:

$ packer version
Packer v1.6.0

Builder: proxmox

Proxmox version:

pveversion --verbose
proxmox-ve: 6.2-1 (running kernel: 5.4.44-1-pve)
pve-manager: 6.2-6 (running version: 6.2-6/ee1d7754)
pve-kernel-5.4: 6.2-3
pve-kernel-helper: 6.2-3
pve-kernel-5.4.44-1-pve: 5.4.44-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
...

I am trying to run a Packer build using the ubuntu-20.04-live-server-amd64.iso, but when the boot_command runs the build just hangs waiting to get an SSH connection to the launched instance. When the ssh_timeout is reached the build fails and the VM is destroyed.

The Packer log reports that the SSH address cannot be obtained because the QEMU guest agent isn't running. It wont be on a fresh ISO and I also set the communicator to ssh, and disabled the qemu_agent, so I'm not sure why this is happening ?

...
  "builders": [
    {
      ...
      "communicator": "ssh",
...
      "qemu_agent": false,
      "ssh_handshake_attempts": "50",
      "ssh_username": "{{user `ssh_username`}}",
      "ssh_password": "{{user `ssh_password`}}",
      "ssh_pty": true,
      "ssh_timeout": "{{user `ssh_timeout`}}",

packer log:

build times-out:

I have copied the complete Packer build file and the user-data below and there is some further detail on this thread which speaks to similar issues (I think I have explored most suggestions here) : hashicorp/packer#9115

Any guidance on how to ensure that the Packer build uses ssh rather than the qemu agent would be much appreciated (or anything else you think might be the culprit).

Kind Regards

Fraser.

host.json:

{
  "builders": [
    {
      "boot_command": [
        "<enter><wait20><enter><f6><esc><wait>",
        "autoinstall ds=nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/",
        "<enter>"
      ],
      "boot_wait": "{{user `boot_wait`}}",
      "communicator": "ssh",
      "disks": [
        {
          "disk_size": "{{user `home_volume_size`}}",
          "storage_pool": "local-lvm",
          "storage_pool_type": "lvm-thin",
          "type": "scsi",
          "format": "raw"
        }
      ],
      "http_directory": "{{user `http_directory`}}",
      "insecure_skip_tls_verify": true,
      "iso_checksum": "{{user `iso_checksum_type`}}:{{user `iso_checksum`}}",
      "iso_file": "{{user `iso_file`}}",
      "memory": 2048,
      "name": "ubuntu-20-04-base",
      "network_adapters": [
        {
          "bridge": "vmbr0",
          "model": "virtio"
        }
      ],
      "node": "{{user `proxmox_target_node`}}",
      "password": "{{user `proxmox_server_pwd`}}",
      "proxmox_url": "https://{{user `proxmox_server_hostname`}}:{{user `proxmox_server_port`}}/api2/json",
      "qemu_agent": false,
      "ssh_handshake_attempts": "50",
      "ssh_username": "{{user `ssh_username`}}",
      "ssh_password": "{{user `ssh_password`}}",
      "ssh_pty": true,
      "ssh_timeout": "{{user `ssh_timeout`}}",
      "type": "proxmox",
      "unmount_iso": true,
      "username": "{{user `proxmox_server_user`}}"
    }
  ],
  "provisioners": [
    {
      "execute_command": "{{ .Vars }} sudo -E -S sh '{{ .Path }}'",
      "inline": [
        "ls /"
      ],
      "type": "shell"
    }
  ],
  "variables": {
    "boot_wait": "2s",
    "http_directory": "http",
    "iso_checksum": "caf3fd69c77c439f162e2ba6040e9c320c4ff0d69aad1340a514319a9264df9f",
    "iso_checksum_type": "sha256",
    "iso_file": "local:iso/ubuntu-20.04-live-server-amd64.iso",
    "proxmox_server_hostname": "proxmox-002",
    "proxmox_server_port": "8006",
    "proxmox_server_pwd": "xxxxxxxxx",
    "proxmox_server_user": "xxxxxxxx",
    "proxmox_target_node": "home",
    "ssh_handshake_attempts": "20",
    "ssh_password": "ubuntu",
    "ssh_username": "ubuntu",
    "ssh_timeout": "10m"
  }
}

user-data:

#cloud-config
autoinstall:
  identity:
    hostname: ubuntu-20-04-base
    password: '$6$wdAcoXrU039hKYPd$508Qvbe7ObUnxoj15DRCkzC3qO7edjH0VV7BPNRDYK4QR8ofJaEEF2heacn0QgD.f8pO8SNp83XNdWG6tocBM1'
    username: ubuntu
  keyboard:
    layout: en
    #toggle: null
    variant: 'gb'
  late-commands:
    - sed -i 's/^#*\(send dhcp-client-identifier\).*$/\1 = hardware;/' /target/etc/dhcp/dhclient.conf
    - 'sed -i "s/dhcp4: true/&\n      dhcp-identifier: mac/" /target/etc/netplan/00-installer-config.yaml'
  locale: en_GB
  network:
    network:
      version: 2
      ethernets:
        ens33:
          dhcp4: true
          dhcp-identifier: mac
  ssh:
    allow-pw: true
    authorized-keys:
    - "ssh-rsa AAAAB3NzaC1yc2..."
    install-server: true
  version: 1

Hi guys,

I'm trying to build images with LUKS encryption but when the OS reboots it asking for the password to unlock the boot.

I'm using QEMU builder with KVM acceleration.

Is there a way to automate this without been in front of the VNC console?

Thanks :)

How can I populate

      "qemuargs": [
        [
          "-display",
          "none"
        ]]

through variables.json as

make BUILD_OPTS='-var-file=variables.json'

Any examples will be useful

It's a little bit of a different approach, but your examples really helped us get this working.
https://git.sr.ht/~lucidone/baseliner

At the moment, the wily and xenial base templates fail to automate the keypresses to get past the initial boot command part so it never actually starts off the install process.

1. extract the official server seed file from the wily and xenial ISOs to confirm they haven't changed
2. merge any changes with the existing wily and xenial preseed files in use here
3. if this doesn't fix it, keep applying various blunt instruments to ubuntu to make it work again

The debian templates all pull in the latest updates while building. Find out why the ubuntu ones fail to do this too.

Hi guys,

First of all, very nicely done, you saved me lots of time.

I tried building boxes using QEMU (14.04, 16.04 and 18.04) . I tried to build my own boxes since 4 years ago - quite painfull and builds seems to be working fine without any errors.

However, building VirtualBox box using MacOS somehow failed at the end of the process

==> vbox: Running post-processor: shell-local
==> vbox (shell-local): Running local shell script: /var/folders/25/4ygjmyhj6ld066wbxqqd2_f80000gp/T/packer-shell007772938
    vbox (shell-local): sed: 1: "build/2018-10-31-10-29- ...": undefined label 'uild/2018-10-31-10-29-24/trusty64S-virtualbox.yaml'
Build 'vbox' errored: 1 error(s) occurred:

* Post-processor failed: Erroneous exit code 1 while executing script: /var/folders/25/4ygjmyhj6ld066wbxqqd2_f80000gp/T/packer-shell007772938

Please see output above for more information.

==> Some builds didn't complete successfully and had errors:
--> vbox: 1 error(s) occurred:

* Post-processor failed: Erroneous exit code 1 while executing script: /var/folders/25/4ygjmyhj6ld066wbxqqd2_f80000gp/T/packer-shell007772938

Please see output above for more information.

==> Builds finished but no artifacts were created.

MacOS: 10.14
Packer version: 1.3.2
Virtual box: 5.2.20

Please do let me know if you need more. I haven't tried the box yet, but it seems to be created (I can see .box file in build directory.

Hello,

not an issue per say, more like a possible hint in regards to this quote from your README:

Even though Packer supports QEMU as an officially-supported provider, Vagrant, for some reason, does not. The 3rd-party plugin named "vagrant-libvirt" provides the missing QEMU support for Vagrant. We are unable at this time to verify this fact due to the following errors encountered while trying to run "vagrant up":

Error while connecting to libvirt: Error making a connection to libvirt URI qemu:///system?no_verify=1&keyfile=/home/whoa/.ssh/id_rsa:
Call to virConnectOpen failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

I had this issue this morning. It's because the libvirtd daemon is not running at all.
On debian (testing/buster), i missed the right package installed (libvirt-daemon-system).
Upon install, the daemon will be started and this message shall disappear.

My 2 cents.

Cheers,