• pacman locked up installing virtualbox 5.0 - then kernel panic, then worse...


    I ran a pacman update this morning and saw that pacman hung in the middle of upgrading virtualbox to 5.0 like around 20% mark. There were some 50ish other updates being applied - this was around 48 of 50. No mouse, no keyboard light toggle, nothing. I waited about 5 minutes, nothing changed - no more screen updates. Upon power reset / reboot I got a kernel panic message.
    Over on the Arch Forums I found a thread on kernel panics from today with people starting to get kernel panics but nothing useful has been found yet.

    In the past I’ve tried a mkinitcpio to fix something like this, so I tried it… made it much worse. My steps:

    • Downloaded latest Antergos ISO “antergos-2015.07.01-x86_64.iso” and used “dd” to make bootable USB.
    • Added the kernel parameter “systemd.unit=multi-user.target” and logged in as root.
    • Used the following command to chroot:
    # mkdir /mnt/arch
    # mount /dev/sda3 /mnt/arch
    # mount /dev/sda1 /mnt/arch/boot
    # mount /dev/sda4 /mnt/arch/home
    # cd /mnt/arch
    # systemd-nspawn
    

    Upon trying “mkinitcpio -p linux” I see messaged like "awk: error while loading shared libraries “file too short”. They appear to be zero-byte files now. Some of them include:

    /usr/lib/libmpfr.so.4
    /usr/lib/libudev.so.1

    I have other message about “sort” terminated by signal 13 and modprobe missing parameters.

    Now after rebooting, I can’t even get to the panic screen, I get unable to find root device with a referenced UUID and the keyboard does not work. I can boot the USB key and chroot again no issue… before I make things worse, figured I ask for help / suggestions.

  • the UUID being referenced at boot is infact the root device and UUID referenced is correct. “blkid” shows the device UUID and it matches, so I think the main problem is that the initramfs file on the root device is corrupt. Seems like I should be able to copy the initramfs files from the USB drive… any thoughts?

  • Hi,

    Try using arch-chroot instead of systemd-nspawn:

    su
    mkdir /mnt/arch
    mount /dev/sda3 /mnt/arch
    mount /dev/sda1 /mnt/arch/boot
    mount /dev/sda4 /mnt/arch/home
    arch-chroot /mnt/arch
    pacman -Syyu
    mkinitcpio -p linux
    grub-mkconfig -o /boot/grub/grub.cfg
    exit
    reboot
    
  • I reach the step for running pacman and it just does nothing… no output. Just goes back to the command line with no error on the screen. Even if I makeup crazy bad parameters I get nothing.

    sh-4.3# pacman -Syyu
    sh-4.3# pacman -Syyu --verbose
    sh-4.3# pacman work-damit
    sh-4.3#
    

    Perhaps related to this…

    sh-4.3# yaourt
    /usr/lib/yaourt/pacman.sh: line 48: /usr/bin/pacman: Success
    sh-4.3# yaourt ?
    package-query: error while loading shared libraries: /usr/lib/libalpm.so.9: file too short
    

    The libalpm.so.9 file is a symlink to libalpm.so.9.0.1 and the libalpm.so.9.0.1 file is infact a zero byte file.

  • :scream: That’s not good. Hmmm…Okay, try this:

    su
    mkdir /mnt/arch
    mount /dev/sda3 /mnt/arch
    mount /dev/sda1 /mnt/arch/boot
    mount /dev/sda4 /mnt/arch/home
    pacman -Syy
    pacman -S base base-devel --force --root /mnt/arch
    # Then if that completes successfully you can proceed with...
    arch-chroot /mnt/arch
    mkinitcpio -p linux
    grub-mkconfig -o /boot/grub/grub.cfg
    exit
    reboot
    
  • I ran the pacman command with the force and root… not sure how well it worked.

    • I have lots of messages with “ldconfig: File /usr/lib32/xxxxxx.so.* is empty, not checked”.
    • I have lots of messages with “ldconfig: File /usr/lib/lib* is empty, not checked”.
      When it got to “reinstalling linux” I have errors related to module dependencies:
    depmod: ERROR: failed to load symbols from /lib/modules/4.1.2-2-ARCH/extramodules/vboxnetflt.ko.gz: Invalid argument
    depmod: ERROR: failed to load symbols from /lib/modules/4.1.2-2-ARCH/extramodules/vboxnetadp.ko.gz: Invalid argument
    depmod: ERROR: failed to load symbols from /lib/modules/4.1.2-2-ARCH/extramodules/vboxpci.ko.gz: Invalid argument
    depmod: ERROR: failed to load symbols from /lib/modules/4.1.2-2-ARCH/extramodules/vboxdrv.ko.gz: Invalid argument
    

    Then at Generating initial ramdisk, using mkinitcpio Please wait…
    “ERROR: /proc must be mounted”.
    I assume this can be ignored for now since we are not in the chroot yet.

    I tried the arch-chroot and mkinitcpio and got an error:
    “ERROR: Hook /usr/lib/initcpio/install/udev has no build function”

    Followed by warning messages that the image may not be complete.

  • Solved my issue… hopefully this helps someone.

    To address the error message above, I deleted the zero byte files

    total 4376
    -rw-r--r-- 1 root root 4432368 Jul 10 18:14 nvidia.ko.gz
    -rw-r--r-- 1 root root   37515 Jul 10 18:14 nvidia-uvm.ko.gz
    -rw-r--r-- 1 root root       0 Jul 21 22:42 vboxdrv.ko.gz
    -rw-r--r-- 1 root root       0 Jul 21 22:42 vboxnetadp.ko.gz
    -rw-r--r-- 1 root root       0 Jul 21 22:42 vboxnetflt.ko.gz
    -rw-r--r-- 1 root root       0 Jul 21 22:42 vboxpci.ko.gz
    -rw-r--r-- 1 root root      13 Jul 15 08:32 version
    
    Deleted zero-byte modules
    
    sh-4.3# rm vbox*
    sh-4.3# ls -l
    total 4376
    -rw-r--r-- 1 root root 4432368 Jul 10 18:14 nvidia.ko.gz
    -rw-r--r-- 1 root root   37515 Jul 10 18:14 nvidia-uvm.ko.gz
    -rw-r--r-- 1 root root      13 Jul 15 08:32 version
    

    Tried to reinstall kernel and trigger mkinitcpio.

    # pacman -S linux
    

    No more module errors, but still failed with ERROR: Hook /usr/lib/initcpio/install/udev has no build function

    Tried to reinstall all packages:

    # pacman -S $(pacman -Qeq)
    # pacman -S linux
    

    But still failed with ERROR: Hook /usr/lib/initcpio/install/udev has no build function and lots of “is empty, not checked” messages.

    Forced reinstall of every package with references to file “is empty, not checked” such as (lots more):

    # pacman -S gdk-pixbuf2 --force
    # pacman -S libgdk --force
    # pacman -S libibus --force
    # pacman -S gtk3 --force
    

    Tried to reinstall kernel and trigger mkinitcpio.

    # pacman -S linux
    

    NOW - I was able to get a clean mkinitcpio without errors, but still had many lib32 based “is empty, not checked”.

    # pacman -S lib32-gcc-libs --force
    # pacman -S lib32-systemd --force
    # pacman -S lib32-libtiff --force
    # pacman -S lib32-libldap --force
    

    No longer had errors. Upon reboot I no longer had errors about unable to find root device with a referenced UUID. However GDM locked up.

    Searched for additional libraires that were zero bytes and reinstalled:

    # pacman -S xorg-server --force
    # pacman -S gnome-settings-daemon --force
    # pacman -S gst-plugins-ugly --force
    # pacman -S gstreamer0.10-ugly-plugins --force
    # pacman -S ffmpeg --force
    

    Rebooted, and now GDM worked and reached gnome desktop.

  • BTW - I did remove virtualbox during some of the troubleshooting and have not added it back yet. The upgrade of virtualbox 4.x to 5.x is what started this mess (hung during install). I’ll get a nice backup before trying that again.

Posts 8Views 1507
Log in to reply