mdadm uses the hostname in addition to the raid name during assembling the raid.

I.e. the device path is called /dev/md/dev1:boot on host dev1, whereas disko currently uses /dev/md/boot in its fstab:

[root@dev1:~]# ls -la /dev/md/dev1\:boot 
lrwxrwxrwx 1 root root 8 Sep  5 15:55 /dev/md/dev1:boot -> ../md127

i notice that every hybrid example is not an Hybrid partitioning scheme, only a GPT BIOS compatible. Am I wrong?

I need a real Hybrid where dos table have a first vfat (0x0C) partition and then a full disk gpt (0xEE) type partition:

$ fdisk -t dos -l /dev/sda
(...)
Device                    Boot Start   End Sectors  Size Id Type
/dev/sda1                 *       34 65569   65536   32M  c W95 FAT32 (LBA)
/dev/sda2                          1    33      33 16.5K ee GPT

And gpt table have the real partition info:

$ fdisk -t gpt -l /dev/sda
(...)
Device                    Start      End  Sectors Size Type
/dev/sda1                    34    65569    65536  32M EFI System
/dev/sda2                 65570 21037089 20971520  10G Linux root (ARM-64)

I guess a new interface similar to ones for lvm or mdraid will be needed to implement this.

Are multi-disk btrfs filesystems already supported?

I see where I could add additional options to mkfs, so that can be used to pass the other devices and the raid options. However, I can't see how to create the other backing devices, and how to make the dependency such that the backing devices will all be created before the mkfs.

end = "-8GiB"; will cause parted to try to parse -8GiB as options and bail out. -- should be passed before the partitioning commands, as called out in the parted manual.

{ disks ? [ "/dev/sda" ], ... }: {
  lvm_vg = {
    pool = {
      type = "lvm_vg";
      lvs = {
        root = {
          type = "lv";
          size = "100%FREE";
          content = {
            type = "filesystem";
            format = "ext4";
            mountpoint = "/";
          };
        };
      };
    };
  };
  disk = {
    vdb = {
      device = builtins.elemAt disks 0;
      type = "disk";
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            name = "boot";
            type = "partition";
            start = "0";
            end = "1M";
            part-type = "primary";
            flags = ["bios_grub"];
          }
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "100MiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            name = "root";
            type = "partition";
            start = "100MiB";
            end = "-16G";
            part-type = "primary";
            flags = ["bios_grub"];
            content = {
              type = "luks";
              name = "crypted";
              keyFile = "/tmp/secret.key";
              content = {
                type = "lvm_pv";
                vg = "pool";
              };
            };
          }
          {
            name = "root";
            type = "partition";
            start = "-16G";
            end = "100%";
            part-type = "primary";
            bootable = true;
            content = {
              type = "swap";
              randomEncryption = true;
            };
          }
        ];
      };
    };
  };
}

When I run this command:

disko -m mount ./disk-config.nix

I receive this error:

line 23: grep: command not found

+ shopt -s nullglob
+ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS                                      
loop0         7:0    0 798.9M  0 loop  /nix/.ro-store                                   
nvme1n1     259:0    0 953.9G  0 disk  
├─nvme1n1p1 259:1    0     1M  0 part  
└─nvme1n1p2 259:2    0 953.9G  0 part  
  └─md127     9:127  0   1.9T  0 raid0 
nvme0n1     259:3    0 953.9G  0 disk  
├─nvme0n1p1 259:4    0     1M  0 part  
└─nvme0n1p2 259:5    0 953.9G  0 part  
  └─md127     9:127  0   1.9T  0 raid0 
+ findmnt /mnt
+ for r in /dev/md/* /dev/md[0-9]*
+ mdadm --stop '/dev/md/*'
mdadm: error opening /dev/md/*: No such file or directory                               
+ true
+ for r in /dev/md/* /dev/md[0-9]*
+ mdadm --stop '/dev/md[0-9]*'
mdadm: error opening /dev/md[0-9]*: No such file or directory                           
creating partitions...
+ true
+ echo 'creating partitions...'
+ set -efux
+ parted -s /dev/nvme0n1 -- mklabel gpt
Error: Partition(s) 2 on /dev/nvme0n1 have been written, but we have been               
unable to inform the kernel of the change, probably because it/they are in              
use.  As a result, the old partition(s) will remain in use.  You should                 
reboot now before making further changes.
+ rm -rf /tmp/tmp.oCUx3X4Pb5

In #39 I was doing for p in /dev/nvme[0-9]*n1 /dev/sd[a-z]; do blkdiscard "$p"; done to prevent that but I am not sure if this is an issue if your installer also part of some nvme?
I think running blkdiscard on the disk that is about to be partitioned could help here.

Using sops-nix we could set the password file as a secret, this would help make disko suitable for unattended partitioning of LUKS encrypted drives.

I'm having a common problem with disko. It seems to expect all binaries to be reachable within the script context.

That has its pros and its cons. The pro is that you can use the package version you want, as long as it is exposed in the script's $PATH. The con is that it's not very ergonomic unless you're running in a NixOS machine with these imports present:

My specific use case: using hetzner's rescue system (currently based on Debian 11) to format the drives using disko and install nixos. Thus, I'm not on a nixos machine and disko becomes too unpredictable.

My current solution is to wrap the script within an environment that contains all those dependencies, like this:

let
  pkgs = import <nixpkgs> {};
  tsp-create = import ./disko-create;
  wrapScript = pkgs: script:
    let
      diskoEnv = pkgs.symlinkJoin {
        name = "diskoEnv";
        paths = with pkgs; [
          # Device and partition tools
          cryptsetup
          lvm2.bin
          mdadm
          parted

          # Packages that provide a mkfs.* binary
          btrfs-progs
          dosfstools
          e2fsprogs
          f2fs-tools
          nilfs-utils
          util-linux
          xfsprogs
        ];
      } ;
    in pkgs.writeScript "wrapped" ''
      #!${pkgs.bash}/bin/bash
      export PATH=${diskoEnv}/bin:$PATH
      ${tsp-create}
    '';

As you can see, this is very anti-ergonomic and required a lot of research time to see which packages provide the binaries that disko expects to be present.

Options to fix the problem:

1. Disko can provide a helper function that returns such an environment and the script wrapper. Quite similar to upstreaming that snippet and adding some tests to it.
2. Disko accepts a pkgs input, from where to obtain all packages. Then, replace all calls to parted for ${pkgs.parted}/bin/parted, and the same for all other binaries.

I guess option 1 fits better. What do you think?

It would be great if disko could build disk images for each partition and then build disk images from those partitions. This would be beneficial for special setups which need to use disk images. Mounting of those images should be configurable to either be expected as a "physical" disk or straight as a file.

Hello, is there plan to support the depends attribute that nixos filesystems option support? I think right now the mount order is only determined by parsing the name of the mount point for parent and child relationship but it would be nice to also have the option to explicitly specify the dependency relationship.

I followed the steps outlined in the quickstart guide, but when I execute nixos-install, it results in the following error:

error: infinite recursion encountered

       at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:478:28:

          477|         builtins.addErrorContext (context name)
          478|           (args.${name} or config._module.args.${name})
             |                            ^
          479|       ) (lib.functionArgs f);
(use '--show-trace' to show detailed location information)

Using the --show-trace argument shows that this recursion apparently happened

while evaluating the module argument 'pkgs' in "/mnt/etc/nixos/configuration.nix"

and the error disappears when I remove the lines

(pkgs.callPackage ./disko-config.nix {
  disks = ["/dev/vda"];
})

from the imports section of my configuration.nix (of course, it is then replaced by another error because no root file system is defined). Do you know what could be causing this error?

This project looks useful. Can you give some more information?

• What is the status?
• What is the roadmap to upstream it to nixpkgs?
• Why not configure disk setup in configuration.nix using nix expression?
• How is it related to nixpart?

We need declarative disk layout configuration to make NixOS more user friendly! https://discourse.nixos.org/t/user-friendly-nixos-distro/1348/20?u=davidak

more details on the readme would be great.

• can it calculate deltas between a given partition and a wanted partition? will you see log errors?
• does it delete partitions which are not defined?
• what happens definitions in the json are "wrong" (to much space is consumed, sdz does not exist, ... )
• where can I find the json definition schema ? for example for dm-crypt?

I copied the example config from the README to disk-config.nix:

{
  # checkout the example folder for how to configure different disko layouts
  disko.devices = {
    disk.sda = {
      device = "/dev/sda";
      type = "disk";
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "100MiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            name = "root";
            type = "partition";
            start = "100MiB";
            end = "100%";
            part-type = "primary";
            bootable = true;
            content = {
              type = "filesystem";
              format = "ext4";
              mountpoint = "/";
            };
          }
        ];
      };
    };
  };
}

And than ran the command (which I also found in the README):

nix run github:nix-community/disko -- --dry-run ./disk-config.nix
error: attempt to call something which is not a function but a set

       at /nix/store/qwxwbws2nkixrlyfzsbg100a6ss9b4lg-disko/share/disko/cli.nix:21:7:

           20|     else
           21|       import diskoFile ({ inherit lib; } // args);
             |       ^
           22|
(use '--show-trace' to show detailed location information)

It would be really cool if this project supported btrfs. Has anyone started implementing this already? If not I might come around to doing it eventually.

EDIT: I mean btrfs subvolumes, sorry for not clarifying.

Preface

Whether this is an issue with disko or just more of a zfs+systemd general issue is beyond me. I am too much of a beginner with zfs at this point. Just looking for help mostly!

Problem

With the following partitions.nix, the /home mount fails due to "non-empty directory there" (logs further below):

{ disks ? [ "/dev/disk/by-id/<my-disk>" ]
, ... }: {
  disk = {
    main = {
      type = "disk";
      device = builtins.elemAt disks 0;
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            type = "partition";
            name = "ESP";
            start = "0";
            end = "64MiB";
            fs-type = "fat32";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            type = "partition";
            name = "zfs";
            start = "128MiB";
            end = "100%";
            content = {
              type = "zfs";
              pool = "zroot";
            };
          }
        ];
      };
    };
  };
  zpool = {
    zroot = {
      type = "zpool";
      # mode = "mirror";
      rootFsOptions = {
        compression = "lz4";
        "com.sun:auto-snapshot" = "false";
      };
      # mountpoint = "/";

      datasets = {
        "root" = {
          zfs_type = "filesystem";
          mountpoint = null;
        };
        "root/nixos" = {
          zfs_type = "filesystem";
          mountpoint = "/";
        };
        "root/home" = {
          zfs_type = "filesystem";
          mountpoint = "/home";
        };
      };
    };
  };
}

The relevant error log parts from sudo journalctl --boot is as follows (cleaned some probably irrelevant logs from the middle):

Jan 14 09:37:46 nixos-desktop zfs[1008]: cannot mount 'zroot/root/home': mountpoint or dataset is busy
Jan 14 09:37:46 nixos-desktop systemd[1]: Mounted FUSE Control File System.
Jan 14 09:37:46 nixos-desktop systemd[1]: Mounted Kernel Configuration File System.
Jan 14 09:37:46 nixos-desktop systemd[1]: Finished Apply Kernel Variables.
Jan 14 09:37:46 nixos-desktop systemd[1]: Finished Create Static Device Nodes in /dev.
Jan 14 09:37:46 nixos-desktop systemd[1]: Reached target Preparation for Local File Systems.
Jan 14 09:37:46 nixos-desktop systemd[1]: home.mount: Directory /home to mount over is not empty, mounting anyway.
Jan 14 09:37:46 nixos-desktop systemd[1]: Mounting /home...
Jan 14 09:37:46 nixos-desktop systemd[1]: Starting Rule-based Manager for Device Events and Files...
Jan 14 09:37:46 nixos-desktop systemd[1]: Finished Load/Save Random Seed.
Jan 14 09:37:46 nixos-desktop systemd-udevd[1080]: Using default interface naming scheme 'v252'.
Jan 14 09:37:46 nixos-desktop systemd[1]: zfs-mount.service: Main process exited, code=exited, status=1/FAILURE
Jan 14 09:37:46 nixos-desktop systemd[1]: zfs-mount.service: Failed with result 'exit-code'.
Jan 14 09:37:46 nixos-desktop systemd[1]: Failed to start Mount ZFS filesystems.

Output from zfs list is as follows:

➜ zfs list
NAME               USED  AVAIL     REFER  MOUNTPOINT
zroot             19.9G   211G       24K  /zroot
zroot/root        19.9G   211G       25K  /zroot/root
zroot/root/home   9.23G   211G     9.23G  /zroot/root/home
zroot/root/nixos  10.7G   211G     10.7G  /zroot/root/nixos

Finally, even though the boot goes to emergency mode due to the failing mounting, simply continuing will then successfully mount the /home mount without any further issues in use. So this is a somewhat minor issue.

What I expected

The partitions.nix should be valid as I understand it but might be plain wrong due to lack of zfs knowledge.

Partial solutions

Setting

boot.zfs.forceImportAll = true;

will make the boot not enter emergency mode even with the same error messages found in the boot log.

I also tried mounting the / partition separately and noticed the the /home directory already contains an empty user directory. This seems to be the non-empty issue causing this whole bug. There is nothing else except an empty user directory. Deleting this will not persist as it appears again in the next boot or the boot after that. I suppose finding out what creates this empty user directory is crucial?

Setting users.users.nialov.createHome = lib.mkForce false; did not change anything.

In a recent commit, I added flake support.

The scenario is the following:

• You find yourself on a minimal OS
• Now, you want to disko your disk declaratively from somewhere, where you've got your stuff commited and published.

Now the question is: what is the right and balanced interface going forward? (My implementation was only a quick hack)

Maybe it is:

nix bundle --bundler github:nix-community/disko github:my/stuff#diskoProfiles.abraham-lincoln

bash -x ./result

Or similar functionality in the disko CLI:

nix run github:nix-community/disko github:my/stuff#diskoProfiles.abraham-lincoln

Disko profiles need no system spacing, since they are specific to a particular host anyway, so that info is implicit.

First off, thank you so much for your work! Disko has helped saved a lot of time in our embedded Nixos deployments. However, we recently run into a specific use case that is not currently implemented. We want to mount rootfs as readonly at runtime but want to make it read-writable at installation time, right now, disko-mount just use one set of flag that is specified using mountOptions, if we specify read-only mount flags there then the installation will fail since it needs to write to disk, but if we don't then it won't be mounted correctly at runtime.

It would be nice to have disko-mount accept an argument like --installation-mode or something similar as well as supporting another set of mount options in configurations.

I am willing to implement this feature and make a PR if the disko team thinks it's a desirable feature.

Since lvcreate can use remaining free space as percentage, the attribute set of logical volumes should be ordered.

For example, I might want to create a 8GB swap lv and then use the remaining space for my root partition. But if I name my lvs swap and root then the root partition is created first.

To do this, we could either:

1. Keep using the alphabetical order of attributes name in lvs, just name then differently. Nothing has to be done but it feels weird.
2. Add a order attribute and sort the list before dumping in scripts.
3. Replace the lvs Attribute Set with a List.

What do you think ?

• I think this you mean disko.create here ?
  (pkgs.writeScriptBin "tsp-create" (disko.mount cfg))

• You use https://cgit.lassul.us/disko/ here, don't you mean https://github.com/nix-community/disko/?

• Why use a json file for configuration, wouldn't be it nicer to use nix to define the partition table?

I'm working on my own NixOS based OS called ExpidusOS. I've added disko for disk configuration and I've discovered this error when the manual is trying to generate.

error: cannot coerce a list to a string

       at /nix/store/zx0pr42dhxkvl0mlcq364dhdq0n4276k-source/types.nix:12:29:

           11|       name = "subType";
           12|       description = "one of ${attrNames typeAttr}";
             |                             ^
           13|       check = x: if x ? type then typeAttr.${x.type}.check x else throw "No type option set in:\n${generators.toPretty {} x}";

       … while evaluating the attribute 'description'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/types.nix:182:7:

          181|         typeMerge functor deprecationMessage nestedTypes descriptionClass;
          182|       description = if description == null then name else description;
             |       ^
          183|     };

       … while evaluating 'optionDescriptionPhrase'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/types.nix:205:45:

          204|   # ambiguity.
          205|   optionDescriptionPhrase = unparenthesize: t:
             |                                             ^
          206|     if unparenthesize (t.descriptionClass or null)

       … from call site

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/types.nix:598:32:

          597|       name = "nullOr";
          598|       description = "null or ${optionDescriptionPhrase (class: class == "noun" || class == "conjunction") elemType}";
             |                                ^
          599|       descriptionClass = "conjunction";

       … while evaluating the attribute 'type.description'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/types.nix:182:7:

          181|         typeMerge functor deprecationMessage nestedTypes descriptionClass;
          182|       description = if description == null then name else description;
             |       ^
          183|     };

       … while evaluating the attribute 'type'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/options.nix:235:11:

          234|           readOnly = opt.readOnly or false;
          235|           type = opt.type.description or "unspecified";
             |           ^
          236|         }

       … while evaluating 'isDerivation'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/lib/attrsets.nix:446:18:

          445|   */
          446|   isDerivation = x: x.type or null == "derivation";
             |                  ^
          447|

       … from call site

       at /nix/store/s4mflpjnb18gy1kkhz6hwg5gjwkvyf9p-source/nixos/lib/make-options-doc/default.nix:54:8:

           53|   substSpecial = x:
           54|     if lib.isDerivation x then { _type = "derivation"; name = x.name; }
             |        ^
           55|     else if builtins.isAttrs x then lib.mapAttrs (name: substSpecial) x

       … while evaluating 'substSpecial'

       at /nix/store/s4mflpjnb18gy1kkhz6hwg5gjwkvyf9p-source/nixos/lib/make-options-doc/default.nix:53:18:

           52|   # effect, since _type is already used by the module system.
           53|   substSpecial = x:
             |                  ^
           54|     if lib.isDerivation x then { _type = "derivation"; name = x.name; }

       … from call site

       at /nix/store/s4mflpjnb18gy1kkhz6hwg5gjwkvyf9p-source/nixos/lib/make-options-doc/default.nix:67:48:

           66|     // lib.optionalAttrs (opt ? default) { default = substSpecial opt.default; }
           67|     // lib.optionalAttrs (opt ? type) { type = substSpecial opt.type; }
             |                                                ^
           68|     // lib.optionalAttrs (opt ? relatedPackages && opt.relatedPackages != []) { relatedPackages = genRelatedPackages opt.relatedPackages opt.name; }

       … while evaluating the attribute 'options' of the derivation 'options.json'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

       … while evaluating the attribute 'buildCommand' of the derivation 'options.xml'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

       … while evaluating the attribute 'buildCommand' of the derivation 'options-docbook.xml'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

       … while evaluating the attribute 'buildCommand' of the derivation 'generated-docbook'

       at /nix/store/d2flirhsd337gm8j8rxlqklslryx6g3q-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

I've debugged it and the error is only present when I include the disko NixOS module. It seems the description for one of the options is not set correctly.

I'm currently setting up a little server that has 1GB RAM, and upon installation quickly realised I'd need to add some swap!

While disko allows for specifying a "linux-swap" fs-type, it doesn't appear to check for swap partitions or run mkswap or swapon.

Perhaps we could:

1. Add a check for a linux-swap fs-type partition and, if detected, mkswap it and
2. Add a disko.swapon cfg script for enabling swap for the specified partition?

If interested, I might have a go at this soon.

Right now the system is hardcoded to x86_64-linux in flake.nix; I was hoping to use this to deploy an AArch64 server and I assume there's no obstacle to making it run there. Additionally, I usually develop on a macOS machine; it'd be nice if the command-line tool could be run there (of course in dry-run mode only) to check the configuration before I deploy it. flake-utils may come in handy for this.

I run flake-only setups, so any invocation of the CLI leads to error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I). Ideally it'd pin to the nixpkgs disko was built with, but failing that builtins.getFlake could be used (in impure mode only).

shell is a shitty programming language that makes it easy to shoot yourself in the foot: #83

I'm trying to make a script to automate a NixOS installation from the installation media and I use disko to take care of partitioning. However, it seems like whenever I run disko inside a script and use it to make a LUKS partition, it doesn't prompt for the password. It still shows it runs the cryptSetup commands in the output, though. It also fails to partition the disk and terminates with the message No key available with this passphrase.. Running the command directly in a terminal prompts for the password and formats the disk as expected, though.

The exact command I'm using is:

nix run github:nix-community/disko \
	--extra-experimental-features nix-command \
	--extra-experimental-features flakes \
	-- \
	--mode zap_create_mount /tmp/disko-config.nix \
	--arg disks '[ "/dev/vda" ]' ||
	exit 1

This is the full script so far

This is my disko config

I run the script from a NixOS installation medium using this command:
curl -s https://raw.githubusercontent.com/Anomalocaridid/nixos-install/main/nixos-install.sh | sudo bash

It would be extremely useful if disko supported creating tmpfs partitons.

As an example, I use a tmpfs partition mounted on / to achieve non persistance.

e.g.
mount -t tmpfs none /

I found multiple regressions in it.

I've been struggling to use disko in the way I thought it was intended to be used, since the documentation is a bit "silo'd".

So how I thought it would go (after much trial&horror):

1. on install target (myserver):
   • boot a live-USB image (in my case a headless server)
   • lsblk
2. on dev machine:
   • make disko.nix with partitions in it acc. to lsblk output
   • define minimal configuration.nix acc. to example with create and mount scripts
   • define a flake which realises from above and disko and nixpkgs an output, e.g. myserver-partitioning
   • ssh -A <nixos-live-session>
3. on myserver:
   • nixos-rebuild build --flake "gitblabla:/me/mysysflake#myserver-partitioning"
   • result/sw/bin/disko-create (observe how partitions and file systems are created)
   • do some minimal mounts (/ and /boot (?)) into /mnt
   • generate hardware-configuration.nix
4. on dev machine:
   • copy hardware-configuration.nix from myserver
   • make final system flake, referring to same disko.nix (for fileSystems), with output nixosConfigurations.myserver
5. on myserver:
   • nixos-install --root /mnt --flake "gitblabla:/me/mysysflake#myserver"

Something like this (still ongoing with the last step, and step (3) didn't actually go like that: I had to clone the repo into the live session and mess with it locally.)

❓ Is this at all something like the intended usage scenario? And if not, how to achieve the "fresh install from flake" case then?

It is a bit unfortunate that the example has tsp-create instead of disko-create and it is entirely unclear how you're supposed to get the create script in the easiest way.
And it stops working as soon as you actually do "the right thing" (?) and define the partitions in disko.nix that is imported via the module system (i.e. imports = [ ./disko.nix ]), because then the attribute set for config.disko.devices is assigned during "normal" config merging.
Wat worked for me in this case is write the scripts like so (i.e. in the "minimal" configuration.nix:

{
	environment.systemPackages = with pkgs; [
		(pkgs.writeScriptBin "disko-create" (config.system.build.formatScript))
		(pkgs.writeScriptBin "disko-mount" (config.system.build.mountScript))
	];
}

Also not clear from the example/README: how to refer to exactly what kind of disko-config when wanting to use the cli.

So summarising: it would be a great help to untangle the instructions for cli, flake, ... usage and to focus on the actual use case, which is partitioning before install.

raid, nvme, etc

It can be handy to iterate over the diskoConfiguration and just look at the outcome without having to push the flake code anywhere, in order for this to work the full URI of the flake must be resolved before passing it to nix-build so that it's possible to do something like disko -m create --dry-run -f '.#hostname'

Another usecase might be a flake in a private repository that is copied over manually on the target host, without having to configure nix to fetch it remotely.

In nixos-shell this is achieved by using nix flake metadata (https://github.com/Mic92/nixos-shell/blob/master/bin/nixos-shell#L32)

Originally posted by @zarelit in #58 (comment)

Hi, I had a general question about the usage of disko.

Currently, I am using disko in the inputs of my nixos configuration flake (https://github.com/baitinq/nixos-config) (example config in hosts/luna/disks.nix). It appears to work wonderfully, correctly creating the tsp-create/mount scripts on my system.

My question is the following, how would I use this when bootstrapping my nixos system on a new install? (which would be when it is really needed and useful).

I havent seen any flake related documentation on this, and I realise this might be a more personal nix problem with how my configuration is set up, but is there any way to just build a certain part of my configuration with flakes without having a new output created?

Thank you so much.

Having lost some important files to bit rot, I've switched to using ZFS exclusively. Would you please consider supporting ZFS, both unencrypted and natively-encrypted?

Currently I have a snippet similar like this, which will clear out existing partitions, disassemble raids and destroy zpools
before creating the new partitions and filesystems:

{
  nixos-partitioning = nixpkgs.writeShellScript "nixos-partitioning" ''
    set -eux -o pipefail
    # make partitioning idempotent by dismounting already mounted filesystems
    if findmnt /mnt; then
      umount -Rlv /mnt
    fi

    for ds in $(zpool list -H -o name); do
      zpool destroy "$ds"
    done

    # stop all existing raids
    shopt -s nullglob
    for r in /dev/md/* /dev/md[0-9]*; do
      # might fail if the device was already closed
      mdadm --stop "$r" || true
    done

    # clear out existing partition tables
    for p in /dev/nvme[0-9]*n1 /dev/sd[a-z]; do blkdiscard "$p"; done

    echo "create partitions..."
    ${disko.create partitioningCfg}
    echo "mount partitions..."
    ${disko.mount partitioningCfg}
  '';
}

Noticed during some experimentation that spaces in GPT partition names cause problems. Irritating I've lost the exact error output but in the case of a GPT partition name Basic Data Partition, parted complains that Data isn't a valid fs-type i.e. parted interprets the partition name as Basic and then treats Data and Partition as the subsequent arguments. Even escaping the name with strings.escapeShellArg isn't sufficient.

I note that throughout the code, there appears to be little implicit escaping of strings (via strings.escapeShellArg or similar) which might be an intentional decision to keep things explicit, or more likely just something that's not done yet?

In either case, arguments to parted mkpart are a bit more awkward since even if the user thinks to wrap name in escapeShellArg this isn't sufficient, since bash strips the quotes before the command is interpreted by parted. For example this stackoverflow shows an example of the required quoting;

$ sudo parted /dev/sdb mkpart '"Name of the partition"' 0% 100%

I'm trying to work around this on the config side by wrapping the name arguments with this;

escape_parted_name = inp: l.strings.escapeShellArg "\"${inp}\"";

Which I was tempted to submit as a fix targeted specifically at the partition script generation

But then it occurred to me that would introduce an inconsistency, with this one argument being implicitly escaped but the majority of other arguments to parted, and in other generated scripts (names, options etc), being left raw.

Any thoughts to how you guys would like to see this handled? Any existing plans to clean up some of the code around script generation and argument handling?

Hi there,

Going through the code and the examples, I'm not sure BTRFS in LUKS layout is supported with that, is this intentional? Can I send a PR?

Thanks a lot for the amazing tool!

In the installation instructions, it seems recommended to identify disks by label (/dev/disk/by-label/nixos and friends).

It would be very nice if disko could add labels to disks when creating the filesystems.

I am trying to create a disk with a /boot, an encrypted / and a swap partition. How to do it?
I tried to make using the examples of here.

{ disks ? [ "/dev/sda" ], ... }: {
  disk = {
    vdb = {
      device = builtins.elemAt disks 0;
      type = "disk";
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            name = "boot";
            type = "partition";
            start = "0";
            end = "1M";
            part-type = "primary";
            flags = ["bios_grub"];
          }
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "100MiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            name = "luks";
            type = "partition";
            start = "100MiB";
            end = "-16GiB";
            part-type = "primary";
            bootable = true;
            content = {
              type = "luks";
              name = "root";
              mountpoint = "/";
              format = "ext4";
              keyFile = "/tmp/secret.key";
              extraArgs = [
                "--hash sha512"
                "--iter-time 5000"
              ];
            };
          }
          {
            name = "swap";
            type = "partition";
            start = "-16GiB";
            end = "100%";
            part-type = "primary";
            bootable = true;
            content = {
              type = "filesystem";
              format = "linux-swap";
            };
          }
        ];
      };
    };
  };
}

There is no option for a created, but not OS mounted, filesystem. I am not sure if there is some design limitations to avoid this. If there is not i can go with a PR.

I just set up a new PC with disko. The setup process of creating disko.nix and running sudo nix run github:nix-community/disko -- --mode zap_create_mount /path/to/disko.nix --arg disks '[ "/dev/sda " ]' worked great. However, after I finished setup by including the disko module in my nixos config and running nixos-install then rebooting, the system failed to boot, giving the error: stage 2 init script /nix/store/9c0ri...nixos-system-nixos-23.05... /init not found.

Actually, I don't think this path exists in the nix store of the installed system. However, declaring the disk configuration the traditional way (while still using the disk layout generated by disko) then rerunning nixos-install works fine. Below are the disko configuration and the manually generated hardware configuration which works:

disko.nix:

{ disks ? [ "/dev/vdb" ], ... }: {
  disk = {
    vdb = {
      type = "disk";
      device = builtins.elemAt disks 0;
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "256MiB";
            fs-type = "fat32";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            type = "partition";
            name = "luks";
            start = "256MiB";
            end = "100%";
            content = {
              type = "luks";
              name = "nixos";
              # if keyFile is omitted, user is prompted for passphrase
              # keyFile = "/tmp/secret.key";
              content = {
                type = "btrfs";
                extraArgs = [ "--label nixos" ];
                subvolumes = {
                  "/@" = {
                    mountpoint = "/";
                    mountOptions = [ "compress=zstd" "noatime" ];
                  };
                  "/@swap" = {
                    mountpoint = "/swap";
                  };
                  "/@nix" = {
                    mountpoint = "/nix";
                    mountOptions = [ "compress=zstd" "noatime" ];
                  };
                  "/@home" = {
                    mountpoint = "/home";
                    mountOptions = [ "compress=zstd" "noatime" ];
                  };
                  "/@snapshots" = {
                    mountpoint = "/snapshots";
                    mountOptions = [ "compress=zstd" "noatime" ];
                  };
                };
              };
            };
          }
        ];
      };
    };
  };
}

This was loaded with:

  modules = [  inputs.disko.nixosModules.disko ];
  ... 
  disko.devices = import ./disko.nix {
    lib = pkgs.lib;
    disks = [ "/dev/sda" ];
  };

and this is a hardware config which boots fine:

{ config, lib, ... }:

{

  boot.initrd.luks.devices = {
    nixos = {
      device = "/dev/disk/by-uuid/e43cde64-5632-4c67-a8b9-ce9cfd70b976";
    };
  };

  fileSystems."/" =
    { device = "/dev/mapper/nixos";
      fsType = "btrfs";
      options = [ "subvol=@" "compress=zstd" "noatime" ];
    };

  fileSystems."/home" =
    { device = "/dev/mapper/nixos";
      fsType = "btrfs";
      options = [ "subvol=@home" "compress=zstd" "noatime" ];
    };

  fileSystems."/swap" =
    { device = "/dev/mapper/nixos";
      fsType = "btrfs";
      options = [ "subvol=@swap" ];
    };

  fileSystems."/nix" =
    { device = "/dev/mapper/nixos";
      fsType = "btrfs";
      options = [ "subvol=@nix" "compress=zstd" "noatime" ];
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/0C1E-D5C4";
      fsType = "vfat";
    };
}

Please add a license to the repository.

Otherwise it would be illegal to include your project in other projects, as the default in most jurisdictions is "you're allowed to look at public code, and maybe even execute it, but not build upon it". An Open Source License changes that.

I suggest Apache or MIT for the least hassle. GitHub has provided the page https://choosealicense.com/ if you want more options.

If there is already a license, I couldn't find it. In that case, make it more obvious.

My use case involves a BTRFS partition inside LUKS.
Normally, I could add a label to the filesystem using btrfs filesystem label /dev/mapper/example my_label (#29 (comment) for other filesystems)

Is there a way to assign the label using Disko?
I see that in #30 you found a work around to labels, but I'm not sure how/if to utilize part-type for this

Thanks!

When I have the followign config:

cfg = {
    disk = {
      disk0 = {
        device = "/dev/sda";
        type = "disk";
        content = {
          type = "table";
          format = "gpt";
          partitions = [
            {
              name = "nix";
              type = "partition";
              part-type = "primary";
              start = "0%";
              end = "100%";
              bootable = true;
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/a";
              };
            }
          ];
        };
      };
      disk1 = {
        device = "/dev/sdb";
        type = "disk";
        content = {
          type = "table";
          format = "gpt";
          partitions = [
            {
              name = "root";
              type = "partition";
              part-type = "primary";
              start = "0%";
              end = "100%";
              bootable = true;
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/b";
              };
            }
          ];
        };
      };
    };
  };

disko.create cfg fails with the following error:

error: attribute 'deviceDependencies' missing

     at /home/rishi/disko/types.nix:134:61:

        133|     create = devices: let
        134|       sortedDeviceList = diskoLib.sortDevicesByDependencies (diskoLib.meta devices).deviceDependencies devices;
           |                                                             ^
        135|     in ''

I am with files:

flake.nix

{
  inputs.disko.url = "github:nix-community/disko";
  inputs.disko.inputs.nixpkgs.follows = "nixpkgs";

  outputs = { self, nixpkgs, disko }: rec {
    # change `yourhostname` to your actual hostname
    nixosConfigurations.nixos = nixpkgs.lib.nixosSystem {
      # change to your system:
      system = "x86_64-linux";
      modules = [
        ./configuration.nix
        disko.nixosModules.disko
      ];
    };
    defaultPackage.x86_64-linux = nixosConfigurations.nixos.config.system.build.toplevel;
  };
}

configuration.nix

{
  pkgs,
  lib,
  ...
}:
{
  boot.loader.grub.devices = [ "/dev/sda" ];
  system.stateVersion = "22.05";
  # checkout the example folder for how to configure different diska layouts
  disko.devices = {
    disk.sda = {
      device = "/dev/sda";
      type = "disk";
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "100MiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            name = "root";
            type = "partition";
            start = "100MiB";
            end = "100%";
            part-type = "primary";
            bootable = true;
            content = {
              type = "filesystem";
              format = "ext4";
              mountpoint = "/";
            };
          }
        ];
      };
    };
  };
}

When I try to use disko -f #nixosConfiguration, I got an error:

error: syntax error, unexpected end of file

       at «string»:1:1:

            1|
             | ^

I'm using this config:

{ disks ? [ "/dev/nvme0n1" ], ... }: {
  disk = {
    vdb = {
      type = "disk";
      device = builtins.elemAt disks 0;
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            type = "partition";
            name = "ESP";
            start = "1MiB";
            end = "512MiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
              options = [
                "defaults"
              ];
            };
          }
          {
            type = "partition";
            name = "luks";
            start = "512MiB";
            end = "100%";
            content = {
              type = "luks";
              name = "crypted";
              keyFile = "/tmp/secret.key";
              content = {
                type = "btrfs";
                mountpoint = "/";
                subvolumes = [
                  "/home"
                ];
              };
            };
          }
        ];
      };
    };
  };
}

And running it as follows:

[root@nixos:~]# nix run github:nix-community/disko --no-write-lock-file -- -m create ./luks-btrfs.nix 
warning: not writing modified lock file of flake 'github:nix-community/disko':
• Added input 'nixpkgs':
    'path:/nix/store/dk1bqlz87dzsd8g4fq92pscgd60ji2vr-source?lastModified=0&narHash=sha256-n5UBO6XBV4h3TB7FYu2yAuNQMEYOrQyKeODUwKe06ow=' (1970-01-01)
this derivation will be built:
  /nix/store/32al7lhqnjby2cjymdlr6l1vr9wy3yir-disko-create.drv
building '/nix/store/32al7lhqnjby2cjymdlr6l1vr9wy3yir-disko-create.drv'...
+ parted -s /dev/nvme0n1 -- mklabel gpt
+ parted -s /dev/nvme0n1 -- mkpart ESP 1MiB 512MiB
+ udevadm trigger --subsystem-match=block
+ udevadm settle
+ parted -s /dev/nvme0n1 -- set 1 boot on
+ udevadm trigger --subsystem-match=block
+ udevadm settle
+ mkfs.vfat /dev/nvme0n1p1
mkfs.fat 4.2 (2021-01-31)
+ parted -s /dev/nvme0n1 -- mkpart luks 512MiB 100%
+ udevadm trigger --subsystem-match=block
+ udevadm settle
+ udevadm trigger --subsystem-match=block
+ udevadm settle
+ cryptsetup -q luksFormat /dev/nvme0n1p2 /tmp/secret.key
+ cryptsetup luksOpen /dev/nvme0n1p2 crypted --key-file /tmp/secret.key
+ mkfs.btrfs /dev/mapper/crypted
/nix/store/ji7jbvlz22yycysdg0i75a6xan4l6hlg-disko-create: line 30: mkfs.btrfs: command not found

Any ideas why the script seems to think mkfs.btrfs isn't available?

I've recently tried using disko to manage some VMs. This worked great so now I want to use it to manage my more complex ZFS based setups, too. But before I start I was hoping you can shed some light onto some questions that have popped up as I'm currently not sure whether disko is ready for ZFS "in production". I'm also reluctant to test these things before knowing more.

• Why is there a mountpoint attribute on the zpool? This doesn't make any intuitive sense to me, as pools don't have mountpoints. It seems like the default is null so I figure disko expects me to always set this?
• It seems to me like "disko create" and "disko mount" are supposed to only be used once when setting up a new system. Is this correct?
• Is disko guaranteeing creation order as listed such that I can reliably specify datasets with children? As the datasets are not specified as a list, I am unsure whether there is a ordering guarantee at all. (I'd for example need rpool/home (empty parent) and rpool/home/myuser1, rpool/home/myuser2, so creation order is important)
• Is there a specific reason why _mount is not using altroots (zpool create/import -R /mnt) and zfs mount as recommended by ZFS? It looks like all mounting is done manually for some reason.
• It also occurred to me that encryption keys are not imported or regarded in many commands, so I am wondering whether encryption is supported at all (will prompts be shown?).

I'd of course be willing to contribute any changes I have to make to get it to work, but at this point I cannot judge what is intentional and what isn't. I don't fully understand what "guarantees" disko assumes for its create and mount actions, and I couldn't find any documentation apart from the README. Any help would be appreciated.

I see that disks right now in the config are identified by their given name (e.g. sda), would it be possible to use the disks UUID or would this have to be implemented?

I think this would be much better for reproducibility and security.

I am having difficulty figuring out how to create a btrfs swapfile subvolume.

Currently, my config does it manually, using postCreateHook like so:

"/swap" = {
  mountOptions = [ "noatime" ];
  postCreateHook = ''
    mount -t btrfs /dev/mapper/crypted /mnt
    btrfs filesystem mkswapfile --size ${memory} /mnt/swap/swapfile
    umount /mnt
  '';
};

However, I am not sure if this is the proper way to do so with disko or how to add the created swapfile to swapDevices.

I had a raid crash on me and needed to get up again quickly. I only had one disk to start with, but a second one was ordered already. The plan was to create a ZFS mirror, but that requires me to have two disks in place. Or does it?

With help of a sparse file mounted as a loopback device we can create a degraded ZFS mirror.

# truncate -s <size of other disk> /tmp/disk.raw
# losetup -f --show /tmp/disk.raw
/dev/loop1
# zfs create tank mirror ... /dev/disk/by-label/root-a /dev/loop1
# zfs offline tank /dev/loop1

Now when the second disk arrives I can replace the dummy disk and complete my raid1.

The same could be done with raidz1 (one sparse file max) or raidz2 (two sparse files max).

In disko this would probably best fit into the zpool section.