Giter VIP home page Giter VIP logo

freebsd-cloud-images's Introduction

๐Ÿ—ฒ spawn Cloud instances on libvirt!๐Ÿ—ฒ

Build Status PyPI version Documentation

Logo

You want to spawn local VM quickly.. Like... really quickly. You want them to be as generic as possible. Actually you would like to reuse some existing cloud images!

This is the right tool for you.

Virt-Lightning exposes a CLI inspired by the Cloud and Vagrant. It can also prepare the Ansible inventory file.

This is handy to quickly validate a new Ansible playbook, or a role on a large number of environments.

example: less than 30 seconds to spawn an instance โšก

In a nutshell:

echo "- distro: centos-7" > virt-lightning.yaml
vl up
vl ansible_inventory > inventory
ansible all -m ping -i inventory

example: or 75 seconds for 10 nodes lab โšก

During this recording, we:

  1. use the list of distributions to generate a virt-lightning.yaml file.
  2. we then create an environment based on this file
  3. once the environment is ready, we generate an Ansible inventory file
  4. and we use it to call Ansible's ping module on all the host.

demo

Requirements

  • Python 3.8 or greater
  • The Python3 binding for libvirt, the package is probably called python3-libvirt or 'libvirt-python' according to pip.
  • Libvirt must be running, most of the time you just need to run: sudo systemctl start --now libvirtd
  • Finally, be sure your user can access the system libvirt daemon, e.g with: virsh -c qemu:///system

Optional

  • You make also want to install python3-urwid if you want to get the fancy list of VMs. This dependency is optional.

Installation (Fedora/RHEL)

sudo dnf install libvirt-devel gcc python3-devel pipx
pipx ensurepath
pipx install virt-lightning

Installation (Debian/Ubuntu)

sudo apt install python3-venv pkg-config gcc libvirt-dev python3-dev pipx
pipx ensurepath
pipx install virt-lightning

Post Installation

virt-lightning will be installed in ~/.local/bin/. Add it in your $PATH if it's not already the case. For instance you can use:

echo "export PATH=$PATH:~/.local/bin/" >> ~/.bashrc
source ~/.bashrc

Fetch some images

Before you start your first VM, you need to fetch the images. To do so, you just use the vl fetch command:

$ vl fetch fedora-32

Actions

vl is an alias for virt-lightning, you can use both. In the rest of the document we use the shortest version.

vl distro_list

List the distro images that can be used. Its output is compatible with vl up. You can initialize a new configuration with: vl distro_list > virt-lightning.yaml.

vl up

virt-lightning will read the virt-lightning.yaml file from the current directory and prepare the associated VM.

vl down

Destroy all the VMs managed by Virt-Lightning.

vl start

Start a specific VM, without reading the virt-lightning.yaml file.

vl stop

Stop just one VM.

vl status

List the VMs, their IP and if they are reachable.

vl ansible_inventory

Export an inventory in the Ansible format.

vl ssh

Show up a menu to select a host and open a ssh connection.

vl ssh

vl console

Like vl ssh but with the serial console of the VM.

vl ssh

vl viewer

Like vl console but with the SPICE console of the VM. Requires virt-viewer.

vl fetch

Fetch a VM image. You can find here a list of the available images. You can also update the custom configuration to add a private image hub.

Configuration

Global configuration

If ~/.config/virt-lightning/config.ini exists, Virt-Lightning will read its configuration there.

[main]
network_name = virt-lightning
root_password = root
storage_pool = virt-lightning
network_auto_clean_up = True
ssh_key_file = ~/.ssh/id_rsa.pub

network_name: if you want to use an alternative libvirt network

root_password: the root password

storage_pool: if you want to use an alternative libvirt storage pool

network_auto_clean_up: if you want to automatically remove a network when running virt-lightning down

ssh_key_file: if you want to use an alternative public key

private_hub: if you need to set additional url from where images should be retrieved, update the configuration file ~/.config/virt-lightning/config.ini adding the following

[main]
private_hub=url1,url2

VM configuration keys

A VM can be tuned at two different places with the following keys:

  • distro: the name of the base distro image to use, it's the only mandatory parameter.
  • name: the VM name
  • memory: the amount of memory to dedicate to the VM
  • vcpus: the number of vcpu to dedicate to the VM
  • root_password: the root password in clear text
  • ssh_key_file: the path of the public key for connecting to the VM
  • groups: this list of groups will be used if you generate an Ansible inventory.
  • disks: a list of disks to create and attach to the VM. The first one is used as the root disk. Default to [{"size": 15}]
    • size the size of the disk in GB. Default is 1.
  • networks: a list of network to attach to the VM. The default is: one virtio interface attached to virt-lightning network.
    • network: the name of the libvirt network. Default is the key network_name from the configuration (virt-lightning by default). The key cannot be used with bridge.
    • ipv4: a static IPv4. Default is a dynamic IPv4 address.
    • nic_model: the libvirt driver to use. Default is virtio
    • mac: an optional static MAC address, e.g: '52:54:00:71:b1:b6'
    • bridge: optional, the name of a bridge to connect to. This key replace the network key.
    • virtualport_type: The type of the virtualport, currently, this can be used with bridge.

Example: a virt-lightning.yaml file:

- name: esxi-vcenter
  distro: esxi-6.7
  memory: 12000
  root_disk_size: 30
  vcpus: 2
  root_password: '!234AaAa56'
  groups: ['all_esxi']
- name: esxi1
  distro: esxi-6.7
  memory: 4096
  vcpus: 1
  root_password: '!234AaAa56'
  groups: ['all_esxi', 'esxi_lab']
- name: esxi2
  distro: esxi-6.7
  memory: 4096
  vcpus: 1
  root_password: '!234AaAa56'
  groups: ['all_esxi', 'esxi_lab']
- name: centos-7
  distro: centos-7
  networks:
    - network: default
      ipv4: 192.168.122.50
  bootcmd:
    - yum update -y

Example: connect to an OpenvSwitch bridge

- name: controller
  distro: fedora-35
  - bridge: my-ovs-bridge-name
    virtualport_type: openvswitch

Example: To getting DHCP working with a bridge connection

- name: vlvm-fedora-40
  distro: fedora-40
  networks:
    - network: virt-lightning
      nic_model: virtio
    - bridge: br0
      nic_model: virtio

You can also associate some parameters to the distro image itself

cat /var/lib/virt-lightning/pool/upstream/esxi-6.7.yaml
username: root
python_interpreter: /bin/python
memory: 4096
networks:
  - network: virt-lightning
    nic_model: virtio
  - network: default
    nic_model: e1000

Example: working with snapshots

User can create a snapshot of VM to restore it later. Default configuration of virt-lightning supports snapshots both for running and powering off VMs. qcow2 disk format used allows diskspace-wise incremental snapshotting, keeping only updated storage blocks. virsh tool supports it by CLI, virt-manager provides a neat GUI supporting most of the features

# create first snapshot of running machine
virsh snapshot-create-as --domain vm_name --name snapshot_1
# create second snapshot
virsh snapshot-create-as --domain vm_name --name snapshot_2
# validate that both of them were saved
virsh snapshot-list vm_name
# and revert to the first one
virsh snapshot-revert vm_name --snapshotname snapshot_1

Development

install libvirt-dev package:

Debian/Ubuntu:

apt install python3-venv pkg-config gcc libvirt-dev python3-dev

Fedora/RHEL:

dnf install python3-devel gcc libvirt-devel

You can run a development copy in a virtual env:

python3 -m venv /tmp/vl-dev-venv
. /tmp/vl-dev-venv/bin/activate
pip3 install -r requirements.txt
pip3 install -e /path/to/my-virt-lightning-copy
vl

The changes that are introduce in /path/to/my-virt-lightning-copy should be visible when you run vl from within the virtual env. running test will require:

pip3 install -r test-requirements.txt

freebsd-cloud-images's People

Contributors

goneri avatar jbpratt avatar metalefty avatar timotheosh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

freebsd-cloud-images's Issues

bare minimum cloud init

Hey ๐Ÿ‘‹๐Ÿผ !

Thanks for publishing these images, I am trying to use the 13.0 one and got excited when I saw that you had cloud-init included. :D

Few questions, from scrolling through the repo.

  1. Is there a default user?
  2. Do you know if the ssh key injection via the provider works?
  3. Do you have a bare minimum cloud-config that creates an account and allows login via ssh key?

Here is the cloud-config I used:

#cloud-config
users:
  - name: till
    sudo: true
    lock_passwd: true
    ssh_authorized_keys:
      - my-public-key

It sorta work, but I can't login (it'll ask me for password). I scrolled through the log, I noticed it discovered my dhcp settings and created an account (even though I can't login).

...
/usr/local/bin/cloud-init startingCloud-init v. 21.2 running 'init-local' at Sun, 25 Sep 2022 14:34:15 +0000. Up 3.7391374111175537 seconds.
2022-09-25 14:34:15,895 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceOpenStack.DataSourceOpenStackLocal'> failed
...
/usr/local/bin/cloud-init startingCloud-init v. 21.2 running 'init' at Sun, 25 Sep 2022 14:34:23 +0000. Up 10.971212387084961 seconds.
ci-info: ++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++
ci-info: +--------+------+-----------------+------------+-------+-------------------+
ci-info: | Device |  Up  |     Address     |    Mask    | Scope |     Hw-Address    |
ci-info: +--------+------+-----------------+------------+-------+-------------------+
ci-info: |  lo0   | True |    127.0.0.1    | 0xff000000 |   .   |         .         |
ci-info: |  lo0   | True |     ::1/128     |     .      |   .   |         .         |
ci-info: |  lo0   | True |  fe80::1%lo0/64 |     .      |  0x2  |         .         |
ci-info: | vtnet0 | True | ip   | 0xffffff00 |   .   | fa:16:3e:12:9c:ea |
ci-info: +--------+------+-----------------+------------+-------+-------------------+
ci-info: 
lo0: link state changed to DOWN
lo0: link state changed to UP
2022-09-25 14:34:28,100 - util.py[WARNING]: Running module users-groups (<module 'cloudinit.config.cc_users_groups' from '/usr/local/lib/python3.7/site-packages/cloud_init-21.2-py3.7.egg/cloudinit/config/cc_users_groups.py'>) failed
...
/usr/local/bin/cloud-init startingCloud-init v. 21.2 running 'modules:final' at Sun, 25 Sep 2022 14:34:29 +0000. Up 17.358665704727173 seconds.
ci-info: no authorized SSH keys fingerprints found for user till.
ci-info: no authorized SSH keys fingerprints found for user till.

Trim disabled in FreeBSD 14.0 UFS

Is it possible to enable trim on /dev/da0p4 on building image?

Enabling TRIM/DISCARD command telling to free blocks to underline device. Without supporting this command underling thin storages like QCOW2 or ThinLVM will grow infinetely.

 tunefs -p /dev/da0p4
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            4952
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)                                 FreeBSD

Unfortunately it's not possible to enable it in running VM:

# tunefs -t enable /dev/da0p4
tunefs: issue TRIM to the disk set
tunefs: /dev/da0p4: failed to open disk for writing

Cloud-init can't growpart in FreeBSD 14.0 with ZFS #7

There is a VM with scsi disk driver and 50G disk, but after vm creation / part is not grow:

# df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default    1.5G    1.1G    441M    72%    /
devfs                 1.0K      0B    1.0K     0%    /dev
zroot                 441M     24K    441M     0%    /zroot
/dev/cd0              3.8M    3.8M      0B   100%    /mnt/cd0
#  geom disk list
Geom name: cd0
Providers:
1. Name: cd0
   Mediasize: 3985408 (3.8M)
   Sectorsize: 2048
   Mode: r1w0e0
   descr: QEMU QEMU CD-ROM
   ident: (null)
   rotationrate: unknown
   fwsectors: 0
   fwheads: 0

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 53687091200 (50G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e4
   descr: QEMU QEMU HARDDISK
   ident: (null)
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255
ls /dev/da0
/dev/da0    /dev/da0p1  /dev/da0p2  /dev/da0p3  /dev/da0p4
2024-07-03 09:45:12,709 - modules.py[DEBUG]: Running module growpart (<module 'cloudinit.config.cc_growpart' from '/usr/local/lib/python3.9/site-packages/cloud_init-24.1.5-py3.9.egg/cloudinit/config/cc_growpart.py'>) with frequency always
2024-07-03 09:45:12,709 - handlers.py[DEBUG]: start: init-network/config-growpart: running config-growpart with frequency always
2024-07-03 09:45:12,709 - helpers.py[DEBUG]: Running config-growpart using lock (<cloudinit.helpers.DummyLock object at 0x3cbdf51bcbb0>)
2024-07-03 09:45:12,710 - subp.py[DEBUG]: Running command ['growpart', '--help'] with allowed return codes [0] (shell=False, capture=True)
2024-07-03 09:45:12,711 - subp.py[DEBUG]: Running command ['gpart', 'help'] with allowed return codes [0, 1] (shell=False, capture=True)
2024-07-03 09:45:12,716 - subp.py[DEBUG]: Running command ['mount'] with allowed return codes [0] (shell=False, capture=True)
2024-07-03 09:45:12,719 - util.py[DEBUG]: found line in mount -> devpth: zroot/ROOT/default, mount_point: /, fs_type: zfs, options: 'local,nfsv4acls'
2024-07-03 09:45:12,719 - util.py[DEBUG]: resize_devices took 0.003 seconds
2024-07-03 09:45:12,719 - cc_growpart.py[DEBUG]: '/dev/vtbd0p4' SKIPPED: stat of '/dev/vtbd0p4' failed: [Errno 2] No such file or directory: '/dev/vtbd0p4'
2024-07-03 09:45:12,719 - cc_growpart.py[DEBUG]: '/' SKIPPED: stat of 'zroot/ROOT/default' failed: [Errno 2] No such file or directory: 'zroot/ROOT/default'
2024-07-03 09:45:12,719 - handlers.py[DEBUG]: finish: init-network/config-growpart: SUCCESS: config-growpart ran successfully

Not sure what to do to grow / part.

There is no such issue with UFS.

FreeBSD 13.0 EoL since August 31, 2022

FreeBSD 13.0 is EoL since end of August 2022, current supported release is FreeBSD 13.1. Any plans to update the script and the BSD cloud images?
The freebsd-update command is not very Ansible-friendly when doing release upgrades, only interactive mode is supported.

(edit) Same is true for FreeBSD 12.2 (EoL) => 12.3.

FreeBSD 12 image fails to boot

Screenshot_freebsd-12_2021-05-16_10:44:14

Boot fails when running in libvirtd with a config like this:

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
  <name>freebsd-12</name>
  <uuid>0fb13e5d-c61c-4e6c-bcc9-c6c25150c241</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://nixos.org/nixos/unstable"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">524288</memory>
  <currentMemory unit="KiB">524288</currentMemory>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-5.2">hvm</type>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-model" check="partial"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/run/libvirt/nix-emulators/qemu-system-x86_64</emulator>
    <disk type="block" device="disk">
      <driver name="qemu" type="raw" cache="none" io="native"/>
      <source dev="/dev/zvol/rpool/ts/freebsd-12"/>
      <target dev="vda" bus="virtio"/>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/cadey/.cache/within/mkvm/seed/freebsd-12-0fb13e5d-c61c-4e6c-bcc9-c6c25150c241.iso"/>
      <target dev="sda" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="de:a7:c7:e3:56:f5"/>
      <source network="default"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </rng>
  </devices>
  <qemu:commandline>
    <qemu:arg value="-smbios"/>
    <qemu:arg value="type=1,serial=ds=nocloud;h=freebsd-12"/>
  </qemu:commandline>
</domain>

The FreeBSD 13 image works just fine.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.