• zfs dkms error during linux kernel (4.11.6-1) update -unsolved


    Here’s the error from terminal output:

    Running pre-transaction hooks...
    Remove DKMS modules
    ==> dkms remove spl/0.7.0rc4 -k 4.11.5-1-ARCH
    ==> dkms remove zfs/0.7.0rc4 -k 4.11.5-1-ARCH
    Upgrading boost-libs (1.64.0-1 -> 1.64.0-2)...
    Upgrading linux (4.11.5-1 -> 4.11.6-1)...
    >>> Updating module dependencies. Please wait ...
    Upgrading linux-headers (4.11.5-1 -> 4.11.6-1)...
    Upgrading perl-http-message (6.11-3 -> 6.13-1)...
    Running post-transaction hooks...
    Install DKMS modules
    ==> dkms install zfs/0.7.0rc4 -k 4.11.6-1-ARCH
    configure: error: 
    	*** Please make sure the kmod spl devel <kernel> package for your
    	*** distribution is installed then try again.  If that fails you
    	*** can specify the location of the spl objects with the
    	*** '--with-spl-obj=PATH' option.  Failed to find spl_config.h in
    	*** any of the following:
    	/usr/src/spl-0.7.0rc4/4.11.6-1-ARCH
    	/usr/src/spl-0.7.0rc4
    

    I restored from last night’s midnight cron backup /boot, /, and /home and am booted up to the Desktop fine.

    Will wait for your review and reply until I try again.
    Dietrich

  • So, I gave this 4 hours to see if anyone would reply. Nope.
    I am going back to ext4 for now. ZFS is not ready for prime time, it would appear, and, after more than a year still issues.

    Closing the issue. --Dietrich

  • Reopening this issue to give an update.
    I did a clean install of Antergos with ZFS again, this time not creating a separate partition for /home and not opting (as previously) to install Linux Kernel-LTS.

    This time around, I have proactively installed zfs-auto-snapshot which is in the Arch repo.
    The backup routines are not controlled by a crond, but enabled, started, stopped using systemctl; ex:
    #systemctl enable zfs-auto-snapshot-hourly.timer
    #systemctl start --now zfs-auto-snapshot-hourly-timer
    "#systemctl list-timers" produces this output confirming the services are running one about to fire in 4 minutes:
    http://i.imgur.com/mTAwtkY.png
    That’s it. If I get another update error of the kind above, I can now restore from a snapshot, presumably.
    Thanks

  • Update: It is 6:30 am now. 1st cup of coffee and checked to see if the auto snapshot is working:

    [[email protected] dts]# zfs list -t snap
    NAME                                             USED  AVAIL  REFER  MOUNTPOINT
    [email protected]                      13.5M      -  6.88G  -
    [email protected]_2017-06-21-0000_hourly       11.0M      -  6.88G  -
    [email protected]_2017-06-21-0100_hourly       19.0M      -  6.91G  -
    [email protected]_2017-06-21-0200_hourly       26.0M      -  6.92G  -
    [email protected]_2017-06-21-0300_hourly       36.6M      -  6.93G  -
    [email protected]_2017-06-21-0400_daily         248K      -  6.94G  -
    [email protected]_2017-06-21-0400_hourly        376K      -  6.94G  -
    [email protected]_2017-06-21-0500_hourly       50.7M      -  6.98G  -
    [email protected]_2017-06-21-0600_hourly       56.7M      -  6.98G  -
    [email protected]_2017-06-21-0700_hourly       72.6M      -  7.02G  -
    [email protected]_2017-06-21-0800_hourly       76.8M      -  7.03G  -
    [email protected]_2017-06-21-0900_hourly       84.6M      -  7.04G  -
    [email protected]_2017-06-21-1000_hourly       92.2M      -  7.05G  -
    antergos_p2v2/[email protected]_2017-06-21-0000_hourly  10.1M      -   412M  -
    antergos_p2v2/[email protected]_2017-06-21-0100_hourly  5.04M      -   457M  -
    antergos_p2v2/[email protected]_2017-06-21-0200_hourly  5.71M      -   460M  -
    antergos_p2v2/[email protected]_2017-06-21-0300_hourly  4.89M      -   461M  -
    antergos_p2v2/[email protected]_2017-06-21-0400_daily      0B      -   463M  -
    antergos_p2v2/[email protected]_2017-06-21-0400_hourly     0B      -   463M  -
    antergos_p2v2/[email protected]_2017-06-21-0500_hourly  4.93M      -   465M  -
    antergos_p2v2/[email protected]_2017-06-21-0600_hourly  5.97M      -   468M  -
    antergos_p2v2/[email protected]_2017-06-21-0700_hourly  5.01M      -   469M  -
    antergos_p2v2/[email protected]_2017-06-21-0800_hourly  5.16M      -   471M  -
    antergos_p2v2/[email protected]_2017-06-21-0900_hourly  5.27M      -   473M  -
    antergos_p2v2/[email protected]_2017-06-21-1000_hourly  5.20M      -   475M  -
    [[email protected] dts]# 
    
    

    So it is. I am pleased. And I must apologize for my remarks (not ready for prime time). I was wrong. It is ready.

    Now, I feel that even if a package update fails such as the above error with dkms, I can roll back instantaneously and wait for help from the Antergos Team.

    I’ll mark this solved and again thank you to the developers for their hard work. – Sincerely Dietrich

  • Well, I just reopened this issue (marked unsolved) b/c this am I found updates ready to apply that included linux:

    [[email protected] ~]$ sudo pacman -Syyu
    [sudo] password for dts: 
    :: Synchronizing package databases...
     antergos                 153.2 KiB  1344K/s 00:00 [######################] 100%
     core                     125.6 KiB  1047K/s 00:00 [######################] 100%
     extra                   1665.3 KiB  2.01M/s 00:01 [######################] 100%
     community                  3.9 MiB  2.05M/s 00:02 [######################] 100%
     multilib                 176.6 KiB  2.08M/s 00:00 [######################] 100%
    :: Starting full system upgrade...
    resolving dependencies...
    looking for conflicting packages...
    
    Packages (5) lib32-icu-59.1-1  lib32-libxml2-2.9.4+16+g07418011-1
                 linux-4.11.6-3  linux-headers-4.11.6-3  lua-5.3.4-2
    
    Total Download Size:    77.49 MiB
    Total Installed Size:  168.56 MiB
    Net Upgrade Size:       47.50 MiB
    
    :: Proceed with installation? [Y/n] y
    

    It seems that every time an Antergos system using ZFS attempts to apply a Linux update, it is scripted to run dkms to remove the old ZFS drivers and then it commences to add the new ones, … and chokes, errors out as below:

    :: Proceed with installation? [Y/n] y
    :: Retrieving packages...
     linux-4.11.6-3-x86_64                                             61.4 MiB  2.02M/s 00:30 [####################################################] 100%
     linux-headers-4.11.6-3-x86_64                                      7.7 MiB  2.05M/s 00:04 [####################################################] 100%
     lua-5.3.4-2-x86_64                                               227.0 KiB  2.07M/s 00:00 [####################################################] 100%
     lib32-icu-59.1-1-x86_64                                            7.6 MiB  2016K/s 00:04 [####################################################] 100%
     lib32-libxml2-2.9.4+16+g07418011-1-x86_64                        556.6 KiB  1899K/s 00:00 [####################################################] 100%
    (5/5) checking keys in keyring                                                             [####################################################] 100%
    (5/5) checking package integrity                                                           [####################################################] 100%
    (5/5) loading package files                                                                [####################################################] 100%
    (5/5) checking for file conflicts                                                          [####################################################] 100%
    (5/5) checking available disk space                                                        [####################################################] 100%
    :: Running pre-transaction hooks...
    (1/1) Remove DKMS modules
    ==> dkms remove zfs/0.7.0rc4 -k 4.11.6-1-ARCH
    ==> dkms remove spl/0.7.0rc4 -k 4.11.6-1-ARCH
    :: Processing package changes...
    (1/5) installing lib32-icu                                                                 [####################################################] 100%
    (2/5) upgrading lib32-libxml2                                                              [####################################################] 100%
    (3/5) upgrading linux                                                                      [####################################################] 100%
    >>> Updating module dependencies. Please wait ...
    (4/5) upgrading linux-headers                                                              [####################################################] 100%
    (5/5) upgrading lua                                                                        [####################################################] 100%
    :: Running post-transaction hooks...
    (1/3) Install DKMS modules
    ==> dkms install spl/0.7.0rc4 -k 4.11.6-3-ARCH
    Job for systemd-modules-load.service failed because the control process exited with error code.
    See "systemctl status systemd-modules-load.service" and "journalctl -xe" for details.
    ==> dkms install zfs/0.7.0rc4 -k 4.11.6-3-ARCH
    Job for systemd-modules-load.service failed because the control process exited with error code.
    See "systemctl status systemd-modules-load.service" and "journalctl -xe" for details.
    (2/3) Updating linux initcpios
    ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
      -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
    ==> Starting build: 4.11.6-3-ARCH
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [autodetect]
      -> Running build hook: [modconf]
      -> Running build hook: [block]
      -> Running build hook: [keyboard]
      -> Running build hook: [keymap]
      -> Running build hook: [resume]
      -> Running build hook: [zfs]
    ==> ERROR: module not found: `zlib_deflate'
      -> Running build hook: [filesystems]
    ==> Generating module dependencies
    ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
    ==> WARNING: errors were encountered during the build. The image may not be complete.
    ==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
      -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
    ==> Starting build: 4.11.6-3-ARCH
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [modconf]
      -> Running build hook: [block]
    ==> WARNING: Possibly missing firmware for module: wd719x
    ==> WARNING: Possibly missing firmware for module: aic94xx
      -> Running build hook: [keyboard]
      -> Running build hook: [keymap]
      -> Running build hook: [resume]
      -> Running build hook: [zfs]
    ==> ERROR: module not found: `zlib_deflate'
      -> Running build hook: [filesystems]
    ==> Generating module dependencies
    ==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img
    ==> WARNING: errors were encountered during the build. The image may not be complete.
    error: command failed to execute correctly
    (3/3) Arming ConditionNeedsUpdate...
    [[email protected] ~]$ 
    

    ================================

    It’s the same situation as happen when I opened this ticket last week but on a different day. ;) Additionally journalctl shows this:

    Jun 24 07:41:50 dts systemd[1]: Starting Load Kernel Modules...
    -- Subject: Unit systemd-modules-load.service has begun start-up
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    -- 
    -- Unit systemd-modules-load.service has begun starting up.
    Jun 24 07:41:50 dts systemd-modules-load[22865]: Failed to lookup alias 'sg': Function not implemented
    Jun 24 07:41:50 dts systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
    Jun 24 07:41:50 dts systemd[1]: Failed to start Load Kernel Modules.
    -- Subject: Unit systemd-modules-load.service has failed
    -- Defined-By: systemd
    -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
    -- 
    -- Unit systemd-modules-load.service has failed.
    -- 
    -- The result is failed.
    Jun 24 07:41:50 dts systemd[1]: systemd-modules-load.service: Unit entered failed state.
    Jun 24 07:41:50 dts systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.
    

    Unless I rollback ZFS, which I can do, I am dead in the water when I reboot.

    Please review and advise. Thanks all – Dietrich

  • So, this is a follow-up with some thoughts.
    The last attempt to recover from this zfs dkms update (paired with a Linux kernel update) my snapshot was not done appropriately, I think.

    My pool name is ‘data’.
    I have since reinstalled Antergos with Gnome-shell and ZFS and have researched the best way to handle this.
    Since it is necessary to have grub2 on /boot with ext4, a zfs snapshot won’t back it up.
    During the update, there may be a variable number of non /boot files added or changed which the snapshot will pick up.

    A snapshot done with the -r (recursive) switch handles doing a snapshot for each dataset. Example:
    #zfs snapshot -r [email protected]<snapshot-name>

    Here’s the result of using it:
    [[email protected] dts]# zfs list -t snap | grep 1756
    [email protected] 4.16M - 6.32G -
    data/[email protected] 4.78M - 914M -
    data/[email protected] 152K - 3.35M -
    [[email protected] dts]#

    My system is also backed up externally at midnight to a docking station SATA drive and picks up all partitions using rsync, so that means I have one backup of /boot from 0:00am that has /boot I can restore.

    This will allow me to troubleshoot this issue (restore recursive snapshot and external /boot backup, as many times as needed to until a resolution is reached.

    In reading other issues, I got the sense in at least one case that the main problem is with the timing of syncing of mirrors to result in a pairing of Linux kernel (version) to a zfs dkms module (version matching the kernel).

    Am I correct in that this is part of the problem?
    If so, I can uncheck the kernel so it doesn’t get updated and wait 24 hours to see if the mirrors are in sync.

    Your thoughts please. – Dietrich
    P.S. Am I the only one who has experienced this issue with ZFS?

  • Fascinating. There is 0 interest in addressing making ZFS work.
    Well, I won’t devote any more time than I have if this is how Antergos Developers do business. Going back to ett4 and closing issue.

  • @dtschmitz said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    So, I gave this 4 hours to see if anyone would reply. Nope.

    No comment.

    and not opting (as previously) to install Linux Kernel-LTS.

    This is a mistake. Always have a LTS kernel installed so if you have any problems with kernel module updates you can always fallback to the LTS kernel.

    In reading other issues, I got the sense in at least one case that the main problem is with the timing of syncing of mirrors to result in a pairing of Linux kernel (version) to a zfs dkms module (version matching the kernel).

    This shouldn’t be a problem. With each kernel update, zfs module and spl module need to be rebuilt. DKMS should take care of this, but of course, if this fails, you won’t be able to boot your system.

    Fascinating. There is 0 interest in addressing making ZFS work.

    I won’t lie, this is the less preassuring issue we have in our hands. We offer ZFS as an “extra”, and with the assumption that only expert people will use it.

    Having said that, I apologise for not having answered this earlier.

    Cheers!

  • @karasu said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    There is 0 interest in addressing making ZFS work.

    I am sorry I couldn’t help you at all. I did see your thread, but I have absolutely no experience with ZFS:disappointed:. Thus, I had no advice to offer. I am assuming that this was the case with most others on this Forum also.

    That said, I was really impressed with how much determination and sheer Linux knowledge you appear to have.

    As for the @developers, they are actually some of the kindest, most helpful people I know on this Forum. This may be hard to believe due to your experience, but they really do help when they can. I really wish you had had a better experience with Antergos, friend.

    See you later! (hopefully)

    Keep trying, never give up. In the end, you will find that it was all worth it

    Community ISO: https://antergoscommunityeditions.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • Folks,
    I have got no quarrels with the Antergos Developer Team. I understand. You have ‘bigger fish to fry’.

    But, wonder where Antergos will go (at last check, #9 on Distrowatch) if ZFS becomes the default file system (assuming it becomes hardened with thorough debugging). Cream always rises to the top. ;)

    I’d go back to ZFS in a ‘New York Minute’ if I could rely upon one or two contact points to make themselves available on relative short notice should a fubar arise. This is important and would place Antergos on a totally different footing and raise its level of respectability through the stratosphere.

    See what I mean? Give it some thought. – Dietrich

  • @dtschmitz said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    if ZFS becomes the default file system

    This is difficult to ever happen. ZFS can’t be integrated into the kernel (copyright issues), so it will always be a module.

    To be honest, I see BTRFS as a better candidate (not stable enough atm, and it has not all the features it should have to compete with ZFS).

    I’d go back to ZFS in a ‘New York Minute’ if I could rely upon one or two contact points to make themselves available on relative short notice should a fubar arise. This is important and would place Antergos on a totally different footing and raise its level of respectability through the stratosphere.

    It was my idea to include ZFS in our installation. I have no real experience with it, so I can only help in Antergos/ZFS related issues (mainly dkms and kernel modules compilation/integration).

    My advice would be to always install LTS kernel and ZFS with it, this way you’ll always have a backup to be able to boot when some update screws your system.

    As I’m now in this thread, feel free to ask your questions here, I’ll try to solve them asap.

    Cheers!

  • @karasu Thank you – I am wondering if LTS is even necessary.
    One ought to be able to revert to a ZFS snapshot prior to the error condition.
    And, boot to the previous kernel in Grub->Advanced, yes?
    4th of July Holiday weekend here in the USA. Will cut myself some slack and follow up in the coming week after 4/7/2017. – Dietrich

  • @dtschmitz said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    4th of July Holiday weekend here in the USA.

    I know! I can’t wait to go see the fireworks:slight_smile:. It’s nice that we have a day set aside just to remember our country’s founders and those who continue to sacrifice so much for it.

    I’ll be thinking of you when the 4th comes around!

    Keep trying, never give up. In the end, you will find that it was all worth it

    Community ISO: https://antergoscommunityeditions.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • Have a safe 4th of July

  • You too, @dtschmitz.

    Keep trying, never give up. In the end, you will find that it was all worth it

    Community ISO: https://antergoscommunityeditions.wordpress.com
    Linux Basics: https://linuxbasicssite.wordpress.com

  • @dtschmitz said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    Thank you – I am wondering if LTS is even necessary.
    One ought to be able to revert to a ZFS snapshot prior to the error condition.

    The main problem is that for any circumstance your zfs module is not built when updating, once you reboot, you won’t be able to load the zfs module. That makes your system unusable. It’s easier to boot using kernel LTS and its modules (that would not have been updated).

    It’s like a failsafe option.

    Cheers!

  • “Once you reboot”

    But if one rolls back to a prior snapshot for the ZFS partitions and ‘then’ reboots into Grub->Advanced previous kernel, one should be able to recover from the error with a fully operating system and report the error to Antergos and wait for assistance. Yes?

  • @dtschmitz said in zfs dkms error during linux kernel (4.11.6-1) update -unsolved:

    But if one rolls back to a prior snapshot for the ZFS partitions and ‘then’ reboots into Grub->Advanced previous kernel, one should be able to recover from the error with a fully operating system and report the error to Antergos and wait for assistance. Yes?

    If you realise of this before rebooting, yes, no problem. In fact, a zfs rollback is not even needed if you downgrade the just updated kernel (linux package) and check with dkms that modules are not erased but installed (dkms install spl and dkms install zfs).

    Once you’ve rebooted, there is no way to access your ZFS volumes without the ZFS kernel module.

    EDIT: This is what I’m talking about: https://forum.antergos.com/topic/7269/msft0101-platform-acpi-version-232-the-zfs-module-are-not-loaded

  • Just me thinking aloud:
    If your scripts which detect and download a given package (in this case a kernel update) can sense a non-zero bash exit result error condition, would a scripted downgrade of the kernel at the point of error be feasible? This would greatly streamline the update process and give the end-user a recovery option by simply answering an on-screen message:

    “WARNING: There was an error in completing your Linux kernel update – your kernel will revert to the version prior to this update. Please report the error found in file: (path/to/file/kernelupdate.log) to Antergos support. PRESS ANY KEY TO CONTINUE”

  • Hi,

    I don’t think it’s possible to downgrade a package that has been updated at the same time… will check it.

    As I want to reinstall my home computer, I’ll do it using ZFS so I can myself “suffer” these situations. It would be easier to detect them and think how to avoid them in the future.

    Cheers!

dkms5 error196 kernel79 during23 Posts 42Views 3169
Log in to reply