I got my RX 570 Passthrough working on my Ubuntu 18.04 box!

My short journey to get rid of an old windows version in a new way of doing things.

Updates: Got the USB passthrough working as well – post update coming – 02.02.2021

I am not a windows user. Every time i have to switch to windows for any reason i feel in pain. But i like gaming and i have (so far) only one PC for my use and also for my wife’s eventual use to do some work on Windows and the type of tools she needs are not for Linux. At the moment i am writing this i still have Windows 7 installed in my “windows partition”.

All started for 2 reasons: 1. I want to upgrade the windows version to 10 and; 2. get rid of my SATA disks as now i have 1 500GB m.2 NVME and another 1TB SSD drive; and in 2021 flash memory drives are cheap.

For my windows (and probably other systems) new installation i will use KVM as my hypervisor as nowadays for Linux it should be the default. Linux VirtualBox users are just less bad Windows users.

This will not be actually a how-to. I will just share some things i have done to achieve my goal and maybe some caveats that might be interesting for some people.

My PC has the following configuration:

AMD Ryzen 7 2700X
16GB DDR4 2933
GPU Radeon RX 570
3 audio interfaces – you will understand later why this info is important:
– 1 onboad audio HD Intel alike;
– 1 small USB hub that contains also an audio interface;
– 1 UMC202HD;

*** I don’t have a 2nd graphic device nor another Monitor. ***

To make it work i use the pci passthrough using VFIO. PCI-STUB is deprecated and should not be used anymore. You might make it work, but no one in the internet will hardly support this decision.

You only can passthrough PCI devices when they are mapped in IOMMUs. Mine was all ok from the beginning ( so having to group something or updating BIOS, etc. was not necessary for me).

That said, lets go to the transcription of my passthrough configuration, sorry, without much explanation.

Ah! Also i must say that i want to have my windows vm booting from Grub!

So first you should find out your GPU device ids to add to the Linux command line on Grub.

$ sudo lspci -nn |grep Radeon
<strong>26:00.0</strong> VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X] [<strong>1002:67df</strong>] (rev ef)
<strong>26:00.1</strong> Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [<strong>1002:aaf0</strong>]
Code language: HTML, XML (xml)

No the codes above in bold (26:00.0 and 26:00.1 will be used in the QEMU command line parameters) and also (1002:67df and 1002:aaf0 will be used in the kernel boot) .

As my old windows is still there i created a (custom) new one Windows 10 menu entry booting in runlevel 3. On Ubuntu 18.40 i have done by adding it into /etc/grub.d/40_custom:

These are the arguments i did add to the linux command line:

vfio-pci.ids=1002:67df,1002:aaf0 quiet splash amd_iommu=on video=efifb:off vfio_iommu_type1.allow_unsafe_interrupts=1 rd.driver.pre=vfio-pci kvm.ignore_msrs=1 3

It resulted in the follow configuration. I highlighted the parameters.

$ sudo vim /etc/grub.d/40_custom
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Windows 10' {
     recordfail                                                              
     load_video                                                              
     insmod gzio                                                             
     if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi      
     insmod part_msdos                                                       
     insmod ext2                                                             
     if [ x$feature_platform_search_hint = xy ]; then                        
       search --no-floppy --fs-uuid --set=root  3a081659-7e5e-4d0b-9945-0122ef7582c5
     else                                                                    
       search --no-floppy --fs-uuid --set=root 3a081659-7e5e-4d0b-9945-0122ef7582c5
     fi                                                                      

     echo    'Loading Linux 5.4.0-64-generic ...'                            
     linux   /boot/vmlinuz-5.4.0-64-generic root=UUID=3a081659-7e5e-4d0b-9945-0122ef7582c5 ro <strong>vfio-pci.ids=1002:67df,1002:aaf0 quiet splash amd_iommu=on video=efifb:off vfio_iommu_type1.allow_unsafe_interrupts=1 rd.driver.pre=vfio-pci kvm.ignore_msrs=1</strong> 3
     echo    'Loading initial ramdisk ...'                                   
     initrd  /boot/initrd.img-5.4.0-64-generic            
}
Code language: PHP (php)

With this done, you can update your grub configuration with the new entry:

$ sudo update-grub

Virtualization

I followed several tutorials over the internet, but none of them really worked seemingly. Some things i saw over the internet that did not apply to my setup:

– I don’t have a second monitor so i i did to test the QEMU/KVM or Linux kernel paramenters i did through ssh from my work laptop;
– virt-manager was generating 100% broken configuration for this purpose. Maybe in a newer version it would work well;
– All tutorials was saying to use -cpu host,kvm=off – in my case this did not work and i was getting BSOD all the time. I tried to change the configuration in so many different ways, but what worked for me was using -cpu EPYC.
– Yes, you should use -M q35. Also tried lots of others while trying to get rid of BSOD but this is the only one that provides all you need;
– The passthrough graphic device needs to have the x-vga=on. This was one of the main configs i was not able to add through virt-manager.

My QEMU/KVM command line looks now the following:

#!/bin/bash

qemu-system-x86_64 -enable-kvm -M q35 -m 8G -cpu EPYC \
-smp 8,sockets=2,cores=4,threads=1 \
-rtc clock=host,base=localtime \
-boot menu=on \
-vga none \
-nographic \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=26:00.0,bus=root.1,addr=00.0,x-vga=on \ -device vfio-pci,host=26:00.1,bus=root.1 \
-device piix4-ide,bus=pcie.0,id=piix4-ide \ -drive file=/disk/images/win10.qcow2,format=qcow2,if=virtio,id=hd0 \
-drive file=/disk/images/iso/Win10_20H2_v2_EnglishInternational_x64.iso,format=raw,if=none,id=isocd -device ide-cd,bus=piix4-ide.1,drive=isocd \
-drive file=/disk/images/iso/Win10_20H2_v2_EnglishInternational_x64.iso,format=raw,if=none,id=isocd -device ide-cd,bus=piix4-ide.1,drive=isocd \
-object input-linux,id=mouse1,evdev=/dev/input/by-id/usb-SINOWEALTH_Wired_Gaming_Mouse-event-mouse \
-object input-linux,id=kbd1,evdev=/dev/input/by-id/usb-SONiX_USB_Keyboard-event-kbd,grab_all=on,repeat=on \
-soundhw hda
Code language: JavaScript (javascript)

The evdev object lines were added to enable keyboard and mouse to work. You can get this by running ls /dev/input/by-id/ and testing the ones you think is proper by cat /dev/input/by-id/YOUR-DEVICE-NAME and typing (if keyboard) or moving/clicking (if mouse). If some noise appears in the screen, you got the right ones. Once you have the proper devices, just add it to your config as i did above.

You will need to add the virtio drivers for windows 10 in the installation. I did it along with the Windows 10 ISO but i removed the line and i now i don’t know how to do it by heart. O promise updating the post with this info once i test it again with a second CD drive.

Audio saga

Probably the most annoying problem after you have it running. I plan to passthrough the audio as well (my onboard one) but so far i have it working using pulse audio by adding -soundhw hda exporting the device to the VM as a hda-intel device (which in the end is the same as the host).

In my current setup (just finished) my Windows VM runs as root as i need to use qemu commands instead of libvirt’s vm autostart, but pulseaudio is not so fond of running as root, i start pulseaudio as a normal user and after pulseaudio is up i copy the user cookie to the root user so it can send/read data from the socket and also “guess” what is the socket to read/write to. When i had the audio working it was crackling too much then i found out some env variables that could fix MY issue

I am highlighting the variables i did use in case someone need them.

export QEMU_PA_SERVER=${SOCK} # i am getting the socket after pulseaudio is up - check the script
export QEMU_PA_LATENCY_OUT=60 # tried with smaller and bigger values. This one worked fine for me.
export QEMU_PA_SAMPLES=44100 # This is normally the sample rate for HD interfaces. Some good ones (as my UMC202HD) this value could go until 192k.
Code language: PHP (php)

I end up with the following script:

#!/bin/bash -x

# This should boot only in runlevel 3
target=`systemctl get-default`

runlevel=`runlevel |egrep -o [1-5]`
[ ${runlevel} -ne 3 ] && exit 0

# Go ahead and boot vm

# Start pulseaudio daemon (while not using vfio for audio device)
# @todo - change this to a daemon user
RUNUSER=runner
sudo -u$RUNUSER -i pulseaudio -D

sleep 2
cp /home/$RUNUSER/.config/pulse/cookie /root/.config/pulse/

SOCK=$(lsof -p `pidof pulseaudio` |grep native.*STREAM |awk '{ print $9}')

export QEMU_PA_SERVER=${SOCK}
export QEMU_PA_LATENCY_OUT=60
export QEMU_PA_SAMPLES=44100


qemu-system-x86_64 -enable-kvm -M q35 -m 8G -cpu EPYC \
-smp 8,sockets=2,cores=4,threads=1 \
-rtc clock=host,base=localtime \
-boot menu=on \
-vga none \
-nographic \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=26:00.0,bus=root.1,addr=00.0,x-vga=on \
-device vfio-pci,host=26:00.1,bus=root.1 \
-device piix4-ide,bus=pcie.0,id=piix4-ide \
-drive file=/disk/images/win10.qcow2,format=qcow2,if=virtio,id=hd0 \
-drive file=/disk/images/iso/Win10_20H2_v2_EnglishInternational_x64.iso,format=raw,if=none,id=isocd -device ide-cd,bus=piix4-ide.1,drive=isocd \
-object input-linux,id=mouse1,evdev=/dev/input/by-id/usb-SINOWEALTH_Wired_Gaming_Mouse-event-mouse \
-object input-linux,id=kbd1,evdev=/dev/input/by-id/usb-SONiX_USB_Keyboard-event-kbd,grab_all=on,repeat=on  \
-soundhw hda
exit 0

Code language: PHP (php)

But this was not the end. Pulseaudio was for some reason just choosing randomically which audio interface it would use. So i had to stick the audio output i wanted as default at /etc/pulse/default.pa. The line i added:

set-default-sink alsa_output.pci-0000_28_00.3.analog-stereo

Next steps

I am planning now to make the audio passthrough instead of using pulse and also try to make a libvirt xml file from this configs so i can just start libvirtd and using its autostart function.

Hoipe this can help some people. I will add some references i followed. in the bottom of the page, although my ending setup does not look quite like them.

Thanks!

I got my RX 570 Passthrough working on my Ubuntu 18.04 box!

Bytheflockers

My short journey to get rid of an old windows version in a new way of doing things.

Virtualization

Audio saga

Next steps

Useful links:

By theflockers

Related Post

How to deal with frustration?

System tuning suggestions

Os atuais MDAs e as linguagens de filtragem de e-mail (Procmail)

Leave a Reply Cancel reply

You missed

I got my RX 570 Passthrough working on my Ubuntu 18.04 box!

Indiferenças

Desiderata

How to deal with frustration?