• Cnchi allow install to BTRFS?


    Hi,

    so I had a bad experience where an update caused my system to be unbootable.

    Despite taking many ZFS snapshots on a rolling basis, rolling back to an earlier snapshot did not fix the issue. The reason?

    /boot partition is not affected by ZFS rollbacks, as /boot is ext4 – and I know no such solution for using ZFS for /boot (I don’t think GRUB can read directly from ZFS – please correct me if I’m wrong!)

    Therefore, when the / partition was rolled-back to a previous kernel, /boot/vmlinuz-linux.img had the newer kernel still, and the kernel mismatch made things even worse than the initial problem that instigated the rollback.

    I am taken by the method that OpenSUSE uses: OpenSUSE has used BTRFS by default for about 5 years now. They implement rolling snapshots by default that are triggered by system updates. These are also available to Arch users using the following AUR package:

    https://bbs.archlinux.org/viewtopic.php?id=210015

    BTRFS is accessible by GRUB, and in fact, OpenSUSE allows boot to BTRFS snapshots, which are automatically taken before and after system changes/updates, etc. making the system practically unbreakable.

    In fact, there IS no /boot partition in OpenSUSE - it is a BTRFS subvolume. Take a look at a default-partitioned OpenSUSE VM:

    ID 257 gen 29 top level 5 path <FS_TREE>/@
    ID 258 gen 3884 top level 257 path <FS_TREE>/@/var
    ID 259 gen 3636 top level 257 path <FS_TREE>/@/usr/local
    ID 260 gen 3876 top level 257 path <FS_TREE>/@/tmp
    ID 261 gen 1955 top level 257 path <FS_TREE>/@/srv
    ID 262 gen 3872 top level 257 path <FS_TREE>/@/opt
    ID 263 gen 3872 top level 257 path <FS_TREE>/@/boot/grub2/x86_64-efi
    ID 264 gen 3872 top level 257 path <FS_TREE>/@/boot/grub2/i386-pc
    ID 265 gen 3873 top level 257 path <FS_TREE>/@/.snapshots
    ID 266 gen 3889 top level 265 path <FS_TREE>/@/.snapshots/1/snapshot
    ID 268 gen 203 top level 258 path <FS_TREE>/@/var/lib/machines
    ID 271 gen 48 top level 265 path <FS_TREE>/@/.snapshots/2/snapshot
    

    You can see GRUB (both x86_64 and i386 versions) are in BTRFS subvolumes

    Here’s another way of illustrating the point: Gparted showing all the mount points on /dev/sda1 - efi, boot, even a snapshots mount for grub-ing a snapshot to boot from:

    0_1517200587792_btrfs_mountpoints.png

    Navigating to /boot/grub2/x86_64-efi one can see that there are also several module files that are retained in case there is a kernel malfunction or missing modules, allowing the system to continue to boot even if that is the case:

    drwxr-xr-x 1 root root    120 Jan 28 17:47 ..
    -rw-r--r-- 1 root root  15240 Jan 25 12:51 acpi.mod
    -rw-r--r-- 1 root root   1896 Jan 25 12:51 adler32.mod
    -rw-r--r-- 1 root root   7880 Jan 25 12:51 affs.mod
    -rw-r--r-- 1 root root   8120 Jan 25 12:51 afs.mod
    -rw-r--r-- 1 root root  22848 Jan 25 12:51 ahci.mod
    -rw-r--r-- 1 root root    680 Jan 25 12:51 all_video.mod
    -rw-r--r-- 1 root root   1480 Jan 25 12:51 aout.mod
    -rw-r--r-- 1 root root   5208 Jan 25 12:51 appleldr.mod
    -rw-r--r-- 1 root root   4488 Jan 25 12:51 archelp.mod
    -rw-r--r-- 1 root root   8888 Jan 25 12:51 ata.mod
    -rw-r--r-- 1 root root   7016 Jan 25 12:51 at_keyboard.mod
    -rw-r--r-- 1 root root   2504 Jan 25 12:51 backtrace.mod
    -rw-r--r-- 1 root root   9336 Jan 25 12:51 bfs.mod
    -rw-r--r-- 1 root root   3096 Jan 25 12:51 bitmap.mod
    -rw-r--r-- 1 root root   5392 Jan 25 12:51 bitmap_scale.mod
    -rw-r--r-- 1 root root   3184 Jan 25 12:51 blocklist.mod
    -rw-r--r-- 1 root root   3248 Jan 25 12:51 boot.mod
    -rw-r--r-- 1 root root  47808 Jan 25 12:51 bsd.mod
    -rw-r--r-- 1 root root   3328 Jan 25 12:51 bswap_test.mod
    -rw-r--r-- 1 root root  34328 Jan 25 12:51 btrfs.mod
    -rw-r--r-- 1 root root   2848 Jan 25 12:51 bufio.mod
    -rw-r--r-- 1 root root   4360 Jan 25 12:51 cat.mod
    -rw-r--r-- 1 root root   5704 Jan 25 12:51 cbfs.mod
    
    --- partially truncated for brevity ---
    
    -rw-r--r-- 1 root root  28136 Jan 25 12:51 video_fb.mod
    -rw-r--r-- 1 root root   5304 Jan 25 12:51 videoinfo.mod
    -rw-r--r-- 1 root root     16 Jan 25 12:51 video.lst
    -rw-r--r-- 1 root root   8856 Jan 25 12:51 video.mod
    -rw-r--r-- 1 root root   3792 Jan 25 12:51 videotest_checksum.mod
    -rw-r--r-- 1 root root   5456 Jan 25 12:51 videotest.mod
    -rw-r--r-- 1 root root  10416 Jan 25 12:51 xfs.mod
    -rw-r--r-- 1 root root  41488 Jan 25 12:51 xnu.mod
    -rw-r--r-- 1 root root   3256 Jan 25 12:51 xnu_uuid.mod
    -rw-r--r-- 1 root root   3296 Jan 25 12:51 xnu_uuid_test.mod
    -rw-r--r-- 1 root root  19944 Jan 25 12:51 xzio.mod
    -rw-r--r-- 1 root root   8424 Jan 25 12:51 zfscrypt.mod
    -rw-r--r-- 1 root root  10648 Jan 25 12:51 zfsinfo.mod
    -rw-r--r-- 1 root root  56968 Jan 25 12:51 zfs.mod
    

    So I noticed there is no option to install BTRFS using Cnchi (unless I am mistaken).

    I am a huge fan of ZFS, and it was my main goal coming to Antergos to try and make an unbreakable system that can boot from snapshots using ZFS. But the real goal is for an unbreakable system, so the FS is not the most important factor, just a means to an end. So I guess my questions are:

    1. How difficult do you think it would be to implement a structure where the /boot partition is under a snapshot-able FS (BTRFS and LVM are both supported by snapper) and automatically snapshotted via default settings? It is apparently possible given OpenSUSE’s example, which it has been doing for several years now… I am definitely NOT opposed to copying their design and modifying it for Arch.

    2. Is there any appetite among the developers for these sorts of things? If someone could add the option for BTRFS in Cnchi I could attempt to implement and test the rest of the features, like snap-pac, kernel mods in efi, etc. and maybe we could make Antergos unbreakable…

    Edit: After writing all this, I saw that implementing ZFS boot “partition” (dataset) is actually fairly easy - would probably be easier than setting up BTRFS as an option in Cnchi:

    https://wiki.archlinux.org/index.php/Installing_Arch_Linux_on_ZFS#Booting_your_kernel_and_initrd_from_ZFS

    But would still like to see BTRFS. :) Sorry for the confusion…

    Thanks!
    -Avery

  • It’s funny that you requested a BTRFS option in Cnchi because I very recently suggested to try to implement it myself. See https://forum.antergos.com/topic/9040/adding-btrfs-install-option-to-cnchi.

    I’ve been testing BTRFS for quite a while now under Antergos to try out different ways of installing using it to verify its stability, since there’s still quite a lot of FUD going around the filesystem. I’ve written a few blog posts about full setups under Antergos. See https://blog.wheredoi.click/installing-antergos-with-a-btrfs-filesytem-and-subvolumes/ for legacy mode and https://blog.wheredoi.click/installing-antergos-linux-with-a-btrfs-filesytem-and-subvolumes-with-uefi/ for UEFI mode.

    Recently I took the step towards a RAID1 setup, which has been running as smoothly as a single setup, but I need more experience before I can talk optimistically about its usefulness and stability. But I have no doubts. In your case, you’d need at least a RAID1 setup to have a solid filesystem that you can repair too, otherwise BTRFS can only tell you which files are corrupted and can’t do anything with it. Your snapshots won’t be much of use either, if the original file is corrupted.

    Regarding your full BTRFS setup including the boot section, I’m not sure how that would be entirely be possible. I haven’t tested it because I dual boot with Windows. As far as I know UEFI always required me to have a FAT32 partition for the EFI boot files. I can’t say for sure whether that’s a 100% requirement or Cnchi just forces me to have that kind of setup because it doesn’t know better in combination with BTRFS (which it doesn’t support except for formatting the drive into it). In any case, right now I can only help you as far as the setup that I’m experienced with, so no full BTRFS setup. Since the FAT32 partition is really small, it’s easy to backup by other means though. And even then, you can easily fix the EFI partition with a live CD in case your system won’t boot up. I’ll be looking into OpenSUSE’s setup though. I have already been using their documentation a lot as a base for my own setups.

    As for snapper, I have been using it exclusively. It’s wonderful and easy. snap-pac helps with automatic snapshots when using pacman. I’d rather recommend you to do manual snapshots with a decent description every time though. snap-pac doesn’t give useful description when you use pamac for installations.

    It’s not really an answer to your question so sorry for that, but just so you know, you’re not the only one who is looking for the ultimate BTRFS setup in Antergos.

    I have also experienced quite a few system freezes where I can’t go into shell mode by quitting X. I had to force reboot then. And recently my PC just restarted by itself and during the next boot gave CPU errors. Not sure if BTRFS is anything related to it, but I experience no corruption after these reboots.

  • @earthmind Hey, no problem. I’m glad you’re working on this

    I know it’s completely possible to do the setup entirely with subvolumes on / using mount points

    It’s important that /boot be snapshotted – I learned this the hard way with my ZFS setup just recently

    About EFI, that’s a good point - I have a 2nd SSD on my laptop with Windows 1709 and the OpenSUSE installer detected it automatically and incorporated it into Grub’s boot screen. I think the installer used the Windows SSD EFI partition for the OpenSUSE EFI, too.

    The installation was completely 100% default. Basically just clicked “yes” to everything.

    I wrote you a reply to your first question about BTRFS from a couple days ago before I saw that you had written me back.

    That’s interesting about the BTRFS self-healing redundancy requirements. Is it something like this? https://serverfault.com/questions/866214/does-btrfs-self-healing-work-only-by-software-raid/866239

  • Ubuntu also has a decent BTRFS setup in their installer. It also creates a few subvolumes by default. Of course OpenSUSE goes much further in that.

    The serverfault link is talking about it yes, with the difference is that it’s in combination with other equipment, which makes it lose most of its features because you’re not using BTRFS RAID. So that would be a bad idea, unless the manufacturer of the equipment takes a different approach so users can take most out of the BTRFS FS, like Synology does. They don’t use a pure BTRFS filesystem but they use a combination of different layers, to avoid the parts where BTRFS is unreliable (RAID5/6), from damaging the whole storage.

    My opinion is that BTRFS is not just software RAID as it goes further than that, also in reliability. Calling it RAID is also not correct, because again it goes further than what the usual RAID usually has to offer. As for performance, it has no trouble competing against hardware solutions with minimal impact on the CPU for RAID calculations and compression, so I don’t see the point in going for hardware RAID or fake RAID as discussed on the webpage. These things also have issues that come with them

  • By the way, I checked the OpenSUSE installation wizard and it also creates a vfat partition for the UEFI files. I think what you meant is the GRUB files that are stored on the BTRFS. Have a look at this: https://image.tf/show/pCkCbJL3Lnvp7rDn

    So you’ll still need a different back-up scheme for that partition. But what I think is that the vfat partition only contains the EFI firmware, which remains unchanged during kernel upgrades. GRUB will have all the changes applied to and those files are on the BTRFS partition. Although I can’t guarantee that’s how it actually works

    Edit: It seems like we were both wrong: https://wiki.ubuntu.com/EFIBootLoaders
    But it’s not impossible: https://www.rodsbooks.com/refind/drivers.html

cnchi154 btrfs13 allow5 Posts 5Views 993
Bloom Email Optin Plugin

Looks like your connection to Antergos Community Forum was lost, please wait while we try to reconnect.