An archlinux on dell xps 15 9550, with NVMe ssd, system boot with bootctl: 3 partition, /boot, /swap and /.

Steps:

• compiled luksipc on system
• boot with arch installation iso, mount, copied binary out to /root, umount.
• resize fs to free last 100MiB
• ./luksipc -d /dev/nvme0n1p3 (which is ext4 rootfs)

Then SquashfsErrors message appears and everything stopped..

Hi,

I'm doing an in-place conversion that has appeared to hang at 11.8% with an external USB drive containing a Toshiba SATA drive and jMicron USB bridge. Here's the details on my machine:

Sun x2200m2
Dual Quad-Core AMD Opteron(tm) Processor 2376 CPUs
32GB DDR2

Running process:
root@pve:/home/dan# ps auuxx | grep luksipc
root 14568 0.6 0.0 24724 22204 pts/1 S+ Oct06 7:41 luksipc -d /dev/sdc1
root 19560 0.0 0.0 12788 968 pts/0 S+ 08:25 0:00 grep luksipc

Kernel info:
root@pve:/home/dan# uname -a
Linux pve.leeper.us 4.15.18-4-pve #1 SMP PVE 4.15.18-23 (Thu, 30 Aug 2018 13:04:08 +0200) x86_64 GNU/Linux

Output in terminal:
[I]: 3:57: 11.8% 225460 MiB / 1907726 MiB 15.8 MiB/s Left: 1682266 MiB 29:30 h:m
[I]: 3:57: 11.8% 225620 MiB / 1907726 MiB 15.8 MiB/s Left: 1682106 MiB 29:30 h:m
[I]: 3:57: 11.8% 225720 MiB / 1907726 MiB 15.8 MiB/s Left: 1682006 MiB 29:30 h:m

Pressing Enter in the screen session does not respond. Ideas?

I was glad to see some solution to in-place encryption, but I also got confused how it works with the LVM or more specifically which device should be encrypted.

I have the following partitioning scheme:

/dev/sda1 ext2 boot
/dev/sda2 extended partition
/dev/sda3 LVM2 PV

This was a default scheme which installer applied when LVM option was chosen, but not encryption. I thought that it might be possible to encrypt everything on /dev/sda2, basicaly convert it to a LUKS partition, but that cannot be done (0Mib = 0Gib chunks I believe and no fs). Would converting /dev/sda3 be the way to go? Or there is more to it?

What would the expected partitioning scheme be after running ipc with or without LVM?

Searching for in-place encryption of hard disks brings up luksipc and cryptsetup-reencrypt, but I didn't find anything that compares them (or even mentions both). It would be nice if the readme addressed what benefits/drawbacks this tool has compared to cryptsetup-reencrypt.

I thought I read the instructions carefully. I had 10 MiB at the end of the partition, unused by the filesystem there.

This is what I got out of luksipc. What did I do wrong?

Please confirm you have completed the checklist:
    [1] You have resized the contained filesystem(s) appropriately
    [2] You have unmounted any contained filesystem(s)
    [3] You will ensure secure storage of the keyfile that will be generated at /root/initial_keyfile.bin
    [4] Power conditions are satisfied (i.e. your laptop is not running off battery)
    [5] You have a backup of all important data on /dev/sda1

    /dev/sda1: 43097 MiB = 42.1 GiB
    Chunk size: 10485760 bytes = 10.0 MiB
    Keyfile: /root/initial_keyfile.bin
    LUKS format parameters: None given

Are all these conditions satisfied, then answer uppercase yes: YES
[I]: Created raw device alias: /dev/sda1 -> /dev/mapper/alias_luksipc_raw_b214106c
[I]: Size of reading device /dev/sda1 is 45190479872 bytes (43097 MiB + 0 bytes)
[I]: Backing up physical disk /dev/sda1 header to backup file header_backup.img
[I]: Performing luksFormat of /dev/sda1
[I]: Performing luksOpen of /dev/sda1 (opening as mapper name luksipc_a605f673)
[I]: Size of luksOpened writing device is 45173702656 bytes (43081 MiB + 0 bytes)
[I]: Write disk smaller than read disk by 16777216 bytes (16384 kiB + 0 bytes, occupied by LUKS header)
[W]: LUKS header larger than chunk copy size. LUKS format probably has overwritten data that cannot be recovered.
[E]: Implausible values encountered in regards to disk sizes (R = 2240806912l, W = 2224029696l), aborting.
[E]: Exit with code 9 [EC_DEVICE_SIZES_IMPLAUSIBLE]: Device sizes are implausible

Hello, I am the author of fstransform https://github.com/cosmos72/fstransform a tool to change the file system type on a device, in place and without need for a backup. Your program looks very interesting, and had to solve some of yhe same challenges I met in mine.

If I understand correctly the documentation, luksipc cannot resume a conversion after a power failure, because it did not have time to create the"resume.bin" file contaning the pointers and the shadow block.

If you are interested, I have an idea to improve you program, so that it can resume a conversion even after a power failure.

In summary, the idea is:
Write the first block and the pointers into resume.bin before writing the LUKS header,
then encrypt the device backwards, i.e. first reading the (N-1)th block and encrypting it into the N-th block, then reading the (N-2)th block and encrypting it into the (N-1)th block, and so on...
Every time you encrypt one block, update the pointers in resume.bin and fsync() everything.

I did something similar in fstransform (actually a bit more complicated, I did not have the luxury of processing blocks sequentially) and it CAN resume a conversion even after a power failure :)

A Google search returns https://www.johannes-bauer.com/linux/luksipc/ before the GitHub project page, or https://johndoe31415.github.io/luksipc/

Since the project is deprecated, https://www.johannes-bauer.com/linux/luksipc/ would probably save you and your prospective users some trouble.

Hi!

Using -D_LARGEFILE64_SOURCE doesn't imply that sizeof(off_t) == 8. Notably, this is not the case for i386. GLibC manual says:

If you define this macro an additional set of functions is made available which enables 32 bit systems to use files of sizes beyond the usual limit of 2GB. This interface is not available if the system does not support files that large. On systems where the natural file size limit is greater than 2GB (i.e., on 64 bit systems) the new functions are identical to the replaced functions.

The new functionality is made available by a new set of types and functions which replace the existing ones. The names of these new objects contain 64 to indicate the intention, e.g., off_t vs. off64_t and fseeko vs. fseeko64.

It seems that _FILE_OFFSET_BITS=64 is a better match:

This macro determines which file system interface shall be used, one replacing the other. Whereas _LARGEFILE64_SOURCE makes the 64 bit interface available as an additional interface, _FILE_OFFSET_BITS allows the 64 bit interface to replace the old interface.

If _FILE_OFFSET_BITS is undefined, or if it is defined to the value 32, nothing changes. The 32 bit interface is used and types like off_t have a size of 32 bits on 32 bit systems.

If the macro is defined to the value 64, the large file interface replaces the old interface. I.e., the functions are not made available under different names (as they are with _LARGEFILE64_SOURCE). Instead the old function names now reference the new functions, e.g., a call to fseeko now indeed calls fseeko64.

If I'm encountering the following, is the data gone? It seams that the header is much larger as documented (16384 kiB) is that the problem?

# ./luksipc -d /dev/sda1
WARNING! luksipc will perform the following actions:
   => Normal LUKSification of plain device /dev/sda1
   -> luksFormat will be performed on /dev/sda1

Please confirm you have completed the checklist:
    [1] You have resized the contained filesystem(s) appropriately
    [2] You have unmounted any contained filesystem(s)
    [3] You will ensure secure storage of the keyfile that will be generated at /root/initial_keyfile.bin
    [4] Power conditions are satisfied (i.e. your laptop is not running off battery)
    [5] You have a backup of all important data on /dev/sda1

    /dev/sda1: 1907728 MiB = 1863.0 GiB
    Chunk size: 10485760 bytes = 10.0 MiB
    Keyfile: /root/initial_keyfile.bin
    LUKS format parameters: None given

Are all these conditions satisfied, then answer uppercase yes: YES
[I]: Created raw device alias: /dev/sda1 -> /dev/mapper/alias_luksipc_raw_ead9863f
[I]: Size of reading device /dev/sda1 is 2000397795328 bytes (1907728 MiB + 0 bytes)
[I]: Backing up physical disk /dev/sda1 header to backup file header_backup.img
[I]: Performing luksFormat of /dev/sda1
[I]: Performing luksOpen of /dev/sda1 (opening as mapper name luksipc_95ed46b0)
[I]: Size of luksOpened writing device is 2000381018112 bytes (1907712 MiB + 0 bytes)
[I]: Write disk smaller than read disk by 16777216 bytes (16384 kiB + 0 bytes, occupied by LUKS header)
[W]: LUKS header larger than chunk copy size. LUKS format probably has overwritten data that cannot be recovered.
[E]: Implausible values encountered in regards to disk sizes (R = 3238002688l, W = 465l), aborting.
[E]: Exit with code 9 [EC_DEVICE_SIZES_IMPLAUSIBLE]: Device sizes are implausible

First - awesome tool, thank you for creating it. Works well, but the readme needs one clarification:

The file system size reduction should NOT be accompanied by reducing the partition size. Most partitioning tools, like gparted treat the file system and the partition containing it as all the same.
So, if you use gparted to reduce the file system size you will have a partition that is shrunk to the size of the reduced file system as well. This will cause the file system to be labelled as broken and unmountable ("too big for the parent partition") when luksipc is done. It could be fixed later by removing and recreating the parent partition for the new size of the filesystem, but it's better avoided altogether by only using resize2fs.

Don't ask me how I found out about this ;)

I follow the documentation and encrypted root partition (Ubuntu 16.04 with a separate /boot partition unencrypted). After rebooting it is stuck in GRUB terminal. I added cryptdevice directives to GRUB_CMDLINE_LINUX but it did not help.