• After enabling VT-d in BIOS then booting or grub fails


    Sorry for bother you all with this problem, I’m stuck with it more than a week now. Basically if the VT-d option is disabled then the system is booting properly when it’s enabled it stuck somewhere after the BIOS boot and the system boot. All I see is monitors flashing after a BIOS initial screen and it doesn’t react to anything.

    My setup is MSI Gaming Z270 M7 + Intel i7-7700K + GTX 1080. One monitor is plugged into the GTX via display port and an other is plugged into the Motherboards hdmi, in BIOS the Intel’s GPU is enabled.

    Without VT-d both monitors turning on and after booting on the login screen both of them in use ( I changed the login screen to the lightdm-gtk-greeter ).

    I want to use my GTX card as a PCI Passtrough with KVM virtualization. Sadly I’ve tried to solve it with Google’s search results but I haven’t found the holy grail yet.

    How can I debug what’s happening?
    Has somebody achieved KVM PCI Passtrough on Antergos?
    I didn’t see any usefull information in the logs :confused:

    Using Ubuntu since 2008, Using Arch/Antergos since 2017
    Full-Stack PHP/Polymer Application Developer

  • @adaliszk said in After enabling VT-d in BIOS then booting or grub fails:

    VT-d
    I am interested in trying it too, but not investiagate in this till now…

    https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • @joekamprad Thanks, I’m already used that source, but I set everything like the description require to me, but sadly the whole system is not booting when VT-d is enabled, I don’t know that the Grub or the System boot fails because I don’t have any output on my both monitor and in the logs I didn’t see any information regarding this.

    My CPU is a Intel® Core™ i7-7700K which is supporting VT-d. My Motherboard is a MSI Gaming Z270 M7 which also supporting this technologie and my GPU is a MSI GeForce GTX 1080 Aero 8GB OC which is supporting UEFI so it should be OK, but it doesn’t.

    I will try reinstall the whole system, because I used an other build which had the nvidia driver by default, but since I wont use that card in the host system it wont be necessary and it maybe the build’s fault that it doesn’t do anything.

    I will post updates when I have them :innocent:

    Using Ubuntu since 2008, Using Arch/Antergos since 2017
    Full-Stack PHP/Polymer Application Developer

  • Hi,

    Did you try to check last boot logs with journalctl?

    If not:

    First, set VT-d and boot… wait until you think your system would be ready (or hanged ;).

    Reboot and unset VT-d

    Boot normally, login and check your logs with journalctl:
    journalctl -xb -1

    The -1 means to show the logs not from the very last boot but the one before, so hopefully you will see why your system is not booting with VT-d on.

  • @karasu Nice stuff, it stucked after started the Journaling on the disks, but it doesn’t mention why.

    Eventually after a few boot with switching vt-d on/off removing the GPU it broke somehow :kissing_smiling_eyes:

    Now I got the passtrough working with the following steps following the https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF article:

    1. Remove the GPU from the Motherboard

    2. Disable legacy boot, use only UEFI

    3. Enable VT-d and other Virtualization settings

    4. Switch to BIOS display mode to the Internal GPU
      which should be on auto settings, but make it to force it to the motherboards output

    5. Make an installer from the Antergos site

    1. Install it, make sure you have /boot/efi correctly
      it may not work for the first time, keep try with the same stick with multiple usb port and rewrite the usb if doesn’t work, I had to install Antergos 4 times and rewrite the stick 2 times to get it working

    2. Install linux-vfio from the software manager
      It takes a lot of time, I had it done in 4 hours and on the middle of the process it restart itself for some reason but it wait for user input, so be prepare for that.
      Before you install it add Linus and Greg keys because it seems it doesn’t added by default:
      gpg --recv-keys 79BE3E4300411886
      gpg --recv-keys 38DBBDC86092693E
      Info: https://aur.archlinux.org/packages/linux-vfio/

    3. Install qemu and libvirt packages so you would have virtualization ready. Also after install enable the virtd service with: systemctl enable libvirtd
      Also I suggest to install virt-manager as well so you would have a GUI tool to create virtual machines.

    4. Edit the grub /etc/default/grub and add the intel_iommu=on (or amd_iommu=on) option to the GRUB_CMDLINE_LINUX_DEFAULT variable.

    5. Save the settings into the grub script:
      grub-mkconfig -o /boot/grub/grub.cfg

    6. Edit the /etc/mkinitcpio.conf and add vfio modules to the MODULES variable:
      vfio vfio_iommu_type1 vfio_pci vfio_virqfd

    7. Run mkinitcpio -p linux to regenerate initramfs configuration since we added new modules to it.

    8. Make a script file with the following content:

    #!/bin/bash
    shopt -s nullglob
    for d in /sys/kernel/iommu_groups/*/devices/*; do 
        n=${d#*/iommu_groups/*}; n=${n%%/*}
        printf 'IOMMU Group %s ' "$n"
        lspci -nns "${d##*/}"
    done;
    
    1. Shutdown ; Reinstall the GPU ; Boot the system

    2. Disable the monitor which is plugged in into the GPU and make the Motherboards one as default

    3. Run the previously created script, it should have an output with the addresses, somehow the lspci doesn’t work at this point and it freezes the system.

    4. Edit the /etc/modprobe.d/vfio.conf and add/edit the options line:
      options vfio-pci ids=10de:13c2,10de:0fbb
      where the id’s should be the GPU and the GPU’s sound card, add both of them with coma separated

    5. Reboot

    After all of this the PCI device is grabbed with the vfio module and it is in a IOMMU group and I was able to add the device into a virtual machine using virt-manager

    For now I had only get this passtrough working and I will post the steps of the actual virtual machine creation as well.

    Using Ubuntu since 2008, Using Arch/Antergos since 2017
    Full-Stack PHP/Polymer Application Developer

  • @adaliszk said in After enabling VT-d in BIOS then booting or grub fails:

    I will post the steps of the actual virtual machine creation as well

    You are welcome to put this in our WIKI!
    https://antergos.com/wiki/

    [updates once a week] = [90% less problems]
    http://gofccyourself.com
    my-blog#k
    how to add system logs
    i3 GNOME

  • @joekamprad said in After enabling VT-d in BIOS then booting or grub fails:

    You are welcome to put this in our WIKI!

    Will do, actually I plan to make a blog post on my currently non-exitst blog and a video about this :blush:

    Using Ubuntu since 2008, Using Arch/Antergos since 2017
    Full-Stack PHP/Polymer Application Developer

Posts 7Views 440
Log in to reply