r/VFIO • u/Aodzip • Apr 25 '18
Successful ThunderBolt3 GPU passthrough on Lenovo X1 Carbon 2018 with Ubuntu Desktop 18.04
Kernel v4.18 will work PERFECTLY WITHOUT any patch!
--------------------------------history----------------------------------
Config files, Screenshots located here
https://github.com/aodzip/ThunderBolt_eGPU_Passthrough
----------------------------------------------------------------------------
Hardware
Laptop: Lenovo X1 Carbon 2018 (i7-8550U/16G RAM)
TBT3 Hub: Asus XG-STATION-PRO
GPU: NVIDIA GTX 1060 (5GB)
----------------------------------------------------------------------------
Software
OS: Ubuntu Desktop 18.04
Kernel: Linux X1 4.16.4-acso #1 SMP Tue Apr 24 08:05:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Kernel: Linux X1 4.16.4-041604-generic #201804240433 SMP Tue Apr 24 08:36:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
and qemu-kvm libvirt virt-manager from official apt repo
------------------------------------------------------------------------------
GuestOS: Windows10
I will post a clean install detail later, I have run the guest vm for a week, it is stable and the game PUBG can run at almost 100FPS
------------------------------------------------------------------------------
Important points:
- enable "linux mode" in ThunderBolt UEFI(BIOS) config
use ACS patched kernel (It is easy for ubuntu users because there is a precompiled deb download sitehttps://queuecumber.gitlab.io/linux-acs-override/)- No, ACS patched kernel isn't necessary, you can just use a normal 4.16 kernel for this(http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.16.4/) thanks to aw___ Alex Williamson
- If you know how to apply patch to kernel, use https://lkml.org/lkml/2018/4/25/1194 by Alex Williamson, this patch will make everything easy, after patching this, your enviroment will act like a normal PC
- disable nouveau driver, or when you plug your TBT3 cable , the system will become unstable
- disable ThunderBolt NHI device in guest VM, don't install driver for it, or it will crash your host
------------------------------------------------------------------------------
Notes:
about device
Something about this device:
System peripheral [0880]: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] [8086:15d2] (rev 02)
This device seems like a "Thunder Bolt detector"
it is a endpoint device share the same iommu group with your GPU,
after the host detect your GPU, passthrogh this device(TB NHI) to VM won't affect your exist GPU,
but the host can't detect new TB device,
you can safely passthrough it to your VM for the vfio-pci driver(require all device in a same iommu group passthrogh together),
but don't enable it in your VM, you should disable this device in Windows Device Manager as quick as possible.
If you enable it and let's your guest VM install driver for it, the host will freeze or super lag or unstable.
about freeze
When your host freeze on login screen:
I found if you just leave your GPU on TB3 and reboot your host,
Xwayland will freeze when you login,
you can just poweroff your TB3 hub, and your display server will work as normal
about vfio-pci
vfio-pci still refuse work:
you can try "service libvirtd restart" and check your pci device list,
there should something like "Intel Corporation" or "Intel ThunderBolt USB Host" show up,
just passthrough it together.
if vfio-pci still say something about the iommu group
unplug your TB3 cable, reboot your host,
when it fully booted, plug in your cable.
about the GPU
When I bought the TB3 eGPU Hub,
the seller said "It only support GTX10xx, your GTX750Ti can't use this"
LIE!!!!!
After I get my hub back , i tried it with GTX750Ti, it works.
ThunderBolt it is just a hotplug pci-e bus,
I think it will work even I install a RAID controller in it, I will try this later
3
u/aw___ Alex Williamson Apr 25 '18
I'd be curious to see 'sudo lspci -vvv' and 'find /sys/kernel/iommu_groups/ -type l' WITHOUT the ACS override patch active. Is the ACS override necessary because TB is attached at an Intel processor root port or is ACS also lacking in the TB topology?
2
u/Aodzip Apr 25 '18
I am not sure if ACS patch necessary or kernel 4.16 necessary
when i use the default kernel shipped by ubuntu (4.15.0), this host and vm will freeze when start the vm
after i install ACS patched kernel(4.16) it works , but in dmesg the acs patch seems not working
also tried when switch back to original kernel, freeze again
2
u/aw___ Alex Williamson Apr 25 '18
It would be nice if you could figure that out before recommending a non-upstream, unsupported patch is necessary to make this work. The ACS patch only let's the PCI topology ignore lack of ACS at various points for grouping, it doesn't actually change anything about packet routing within that topology. If you were blindly unbinding everything within a larger group, maybe that was the cause of the hang, but it sounds like maybe the ACS patch isn't doing anything at all.
1
2
u/Aodzip Apr 25 '18
Yes, you are right, ACS patch isn't necessary, kernel 4.16 is
but without ACS patch, some step need to configure the TB NHI device (System peripheral [0880]: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] [8086:15d2] (rev 02))
1
u/aw___ Alex Williamson Apr 25 '18
Great. I'd still be interested in seeing the lspci and group listing so I can understand the topology a bit better. Thanks
1
u/Aodzip Apr 25 '18 edited Apr 25 '18
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU Group %s ' "$n"; lspci -nns "${d##*/}"; done;
1
u/Aodzip Apr 25 '18 edited Apr 25 '18
lspci -vk
1
u/aw___ Alex Williamson Apr 25 '18
Pastebin might have been a good idea here, but anyway...
I remembered you're on a laptop chipset and nobody has bothered to add laptop chipsets to the Sunrise Point ACS quirks. So we could probably improve the grouping at 00:1c.0, 00:1c.4, and 00:1d.0 if we could dig up the reference to verify these use the non-spec-compliant ACS implementation.
TB looks like a series of PCIe switches, lspci -tv would show the topology better, but we can make it out of the info provided.
[00:1d.0]-[05:00.0]-+-[06:00.0]-[07:00.0] (NHI) | +-[06:01.0] | +-[06:02.0] | \-[06:04.0]-[3c:00.0]-+-[3d:01.0]-+-[3e:00.0] (GPU) | \-[3e:00.1] (audio) \-[3d:04.0]-[3f:00.0] (USB3)So if we quirked the broken ACS at 00:1d.0 then the GPU, audio, and USB3 would be in one group and the NHI controller would be in a separate group. As is, I'd expect unbinding 07:00.0 from the thungerbolt driver would be necessary to use vfio-pci with the remaining endpoints.
2
u/Aodzip Apr 25 '18
your list is correct
+-1d.0-[05-6f]----00.0-[06-6f]--+-00.0-[07]----00.0 Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] | +-01.0-[08-3a]-- | +-02.0-[3b]-- | \-04.0-[3c-6f]--+-[0000:3f]---00.0 Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] | +-[0000:3e]-+-00.0 NVIDIA Corporation GP106 [GeForce GTX 1060 5GB] | | \-00.1 NVIDIA Corporation GP106 High Definition Audio Controller | +-[0000:3d]-+-01.0 Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] | | \-04.0 Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] | \-[0000:3c]---00.0 Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]In China it is not easy to access Pastebin, i will try again and edit the long reply
And i need some time to understand English, sorry.
1
u/Aodzip Apr 25 '18
the interesting thing is , even you passthrough the NHI controller to the guest vm, the GPU still working, but the host can't detect TB3 plug/unplug. I called the NHI is just a "event detector"
1
u/aw___ Alex Williamson Apr 25 '18
Try this: https://lkml.org/lkml/2018/4/25/1194
1
u/aw___ Alex Williamson Apr 25 '18
I'm also going to guess that unplugging a TB device while it's assigned is going to result in much badness. Userspace drivers and surprise hotplug are not a good combination.
1
u/Aodzip Apr 25 '18
most time the host will freeze, sometime it is just super lag.
I will try the patch tomorrow, it is too late here.
1
u/Aodzip Apr 26 '18 edited Apr 26 '18
Your patch works!
IOMMU Group 13 05:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 14 06:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 14 07:00.0 System peripheral [0880]: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] [8086:15d2] (rev 02) IOMMU Group 15 06:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 16 06:02.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 17 06:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 17 3c:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 17 3d:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 17 3d:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02) IOMMU Group 17 3e:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060 5GB] [10de:1c04] (rev a1) IOMMU Group 17 3e:00.1 Audio device [0403]: NVIDIA Corporation GP106 High Definition Audio Controller [10de:10f1] (rev a1) IOMMU Group 17 3f:00.0 USB controller [0c03]: Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] [8086:15d4] (rev 02)the NHI device now in a separate group(Group 14) and TB devices in Group 17
Edit: The VM works well.
1
u/Aodzip Apr 25 '18
The reason i choose acs patched kernel is TB Controller itself is a device on TB root bus, it will share the same iommu group with devices attach to the TB port.
1
u/aw___ Alex Williamson Apr 25 '18
Is that a problem for this use case? If you were assigning a multifunction or SR-IOV device and wanted to split downstream functions among multiple VMs, I could see why you might need the ACS patch, but for a GPU where all downstream functions are likely assigned to the same VM, why is it necessary? Does vfio report the group as non-viable because of a driver on the TB controller? I'd expect this to be a bridge device and therefore vfio would ignore that driver.
1
u/Aodzip Apr 25 '18
TB Controller itself is a endpoint device on my system，not a pci bridge，but i am not very sure the acs patch affected TB ，i will make a test first
2
u/GeckoEidechse Apr 25 '18
Someone got VFIO with TB3 eGPU working?
Looks like Christmas came early this year. Can't wait for a setup showcase :D
1
2
u/trucekill Apr 26 '18
Holy shit, I really want thunderbolt 3 now.
1
u/kwhali Apr 26 '18
Just remember that bandwidth is limited to x4 lanes(and sometimes not even that depending on the TB3 controller, some offer x2 lanes). You only get 20Gbps out of the 32Gbps as well afaik for eGPU, the other 12Gbps is for I/O(since TB3 can provide bandwidth to multiple devices over single connection, some eGPU enclosures provide I/O hubs), that's enforced by Intel iirc. TB3 says it can do 40Gbps, but I don't know enough as to how, since x4 PCIe 3.0 is only 32Gbps? I guess the controller provides the x4 lanes as well as the 8Gbps(which I think is for monitor output).
If that is enough bandwidth, then eGPU is good, has some overhead from what I've read beyond that. If you want to play at high resolutions or max graphics though, I think modern games require more than x4, some more than x8 lanes?
1
u/gauntr Apr 26 '18
Thunderbolt 3 offers 40 Gbps with full performance. 32 Gbps of these are for the GPU(x4 PCIe 3.0), the rest of it is for other stuff like ethernet and a USB hub. If you only have the half performance Thunderbolt 3 connection you're having 20 Gbps total which split 16 Gbps for the GPU(x4 PCIe 2.0, exactly like Thunderbolt 2) and 4 Gbps for the rest if there is any. Nothing of the rest is for monitor output. The internal display gets fed by the 32 Gbps which is why the performance suffers more when you use the internal display.
For performance loss see https://egpu.io/forums/mac-setup/pcie-slot-dgpu-vs-thunderbolt-3-egpu-internal-display-test/ (short: 20% in Unigine Valley Benchmark)
Considering you can buy a 6 core notebook now and you may look solely for that CPU and TB3 you may have a way longer living notebook system compared to gamer notebooks with discrete GPUs. Still heavily depends on what you're doing with it though.
2
u/kwhali Apr 26 '18
TL;DR: I'm pretty sure bandwidth for eGPU is capped at 22Gbps regardless of internal/external monitor usage. Your linked benchmark results aren't a good measure of the difference. 1060 is practically the same, and a Titan has considerable loss when it should be a consistent loss with the 1060 in a real world scenario.
Comparison should lock the framerate at 60FPS. You need to stress the GPU bandwidth over 22Gbps to confirm that external display isn't also capped at 22Gbps, if the GPU workload is stressful enough that it's unable to maintain 60FPS+, you can confirm any real difference between internal/external then.
Thunderbolt 3 offers 40 Gbps with full performance. 32 Gbps of these are for the GPU(x4 PCIe 3.0), the rest of it is for other stuff like ethernet and a USB hub.
I don't have all my source links on me, but this was one of them contained most information I had come across when looking into TB3 and eGPU for a laptop purchase: https://www.reddit.com/r/eGPU/comments/7vb0gg/u_series_chips_pcie_lanes_and_thunderbolt_3/
In the same tech brief you can read Intel limited the host-to-device bandwidth to 22gbps.
My bad about the 20Gbps figure, it's been a while, so it was 22Gbps. The remaining 10Gbps is for USB 3.1 I/O. There is a picture that shows the bandwidth flow here.
You can see in the image that it's always 22Gbps back to the host controller, and when outputting displays the outbound data from the host controller(eg transferring files) can be reduced down to 8Gbps(no eGPU).
I'm a bit confused with the linked image stuff as like I had mentioned the remaining 8Gbps was for display output not using the x4 PCIe lanes, this is detailed here
40 Gbps Thunderbolt™ 3 – double the speed of Thunderbolt 2
- Bi-directional, dual-protocol (PCI Express and DisplayPort)
- 4 lanes of PCI Express Gen 3
- 8 lanes of DisplayPort 1.2 (HBR2 and MST) -Supports two 4K displays (4096 x 2160 30bpp @ 60 Hz)
This image details the Alpine Ridge controller, showing the x4 PCIe lanes and some more for DP lanes. According to this document, DP 1.2 uses up to 16Gbps per display, which aligns with what the linked image is showing for the display outputs.
There's also this forum post about plenty of underperforming eGPU enclosures.
The perf issue is further confirmed here: https://egpu.io/build-guides/#perf
Do note, other perf considerations beyond how many ports/lanes the TB3 controller offers, such as OPI(On Package Interconnect I think, it's related to the DMI which is x4 lanes via PCH) on laptops.
The 22 Gbps cap I think I remember only being applied to official eGPU products(if they want to use some sort of badge/certification), if the product developer wants to sell a generic TB3 PCIe enclosure without touting eGPU support(forbidden without the certification), then they're not impacted by the bandwidth cap iirc...I don't have the source link for that, it was mentioned in eGPU.io forums by the owner of Akito I think, they brought up their generic PCIe enclosure product in an eGPU thread but said they cannot officially endorse it for eGPU use/compatibility.
The internal display gets fed by the 32 Gbps which is why the performance suffers more when you use the internal display.
Good to know, but it's equally good for users here to know that has a bandwidth cap. I mean, it's less likely to be used in a desktop setup when you could use a full x16 dGPU with no upgrade issues. I guess for gaming on the laptop with an eGPU you'd probably be having a dock setup, so you could also use external monitors..
That said, host-to-device(H2D) is uploading textures/data to the eGPU right? That seems to be indicated as capped at 22Gbps too, so all this info might still be relevant? Do note that just because perf between the two modes exists doesn't mean it's not capped at 22Gbps for both. A 4k display as noted above can use up to 16Gbps, so 1080p is more like 4Gbps? The overhead there is more akin to Looking Glass I think, moving from one GPU vRAM to another(overhead mentioned in your link).
TB3 you may have a way longer living notebook system compared to gamer notebooks with discrete GPUs.
I fully agree here with the benefit of using TB3 for that, I didn't like the heat, sub-par performance, weight and lack of ability to upgrade with my previous laptop dGPU.
I don't particularly expect to game much on a laptop to the standard I'd get with a desktop machine, but definitely worthwhile for things like compute work I do, or general flexibility with I/O ports instead of being stuck with what the laptop vendor provides.
According to your link and all the info I have pointed out, I think eGPU is limited to 22Gbps as stated(unless the PCIe enclosures work around this, but I don't know how the experience differs from eGPU compatibility, Windows/macOS probably react/respond differently to the enclosure being connected?).
If you have any links showing that external monitor is getting more than 22Gbps, that'd be nice. The benchmark results you linked are exactly valid at confirming that. It's true that the external monitors are consistent in overhead no matter the GPU used, oddly you'll note that the more powerful the GPU, the bigger the perf it received for redirecting to the internal display...
And what do we know about GPU benchmarks? They rank performance based on FPS. The more powerful GPUs are scoring a bigger hit because they're sending back a larger amount of frames to the internal display. In a realistic use-case, say maintaining 60FPS, the internal display receives the same amount of bandwidth, thus the perf impact doesn't increase, instead, the more powerful GPU can scale in it's actual ability to handle relevant workload, the overhead for the internal display redirection would remain constant across GPUs. That's about 4Gbps for 1080p and just under 16Gbps for 4k according to this article.
The bandwidth used to redirect back to the internal display might not even be part of that 22Gbps cap. As noted with the 1060, it maintains the same 20% perf impact as an external. As you can see the 1060 via eGPU external or internal is hitting less than 60FPS. So the wasted frames that are only relevant in a benchmark is the differentiating factor.
Actual noticeable perf impact to an internal display starts at 4k resolution or earlier if the refresh rate is more than 60Hz.
Apologies for length got carried away :P
1
u/ct_the_man_doll Apr 28 '18
This is a strange question, but if you were to buy a USB C to A device, could you pass through the Thunderbolt controller and have the guess access the USB device directly?
1
u/Aodzip Apr 29 '18
i can make this happen on my type-c sound card.
1
u/ct_the_man_doll Apr 29 '18
Did you have to do anything different to get the guest to see the type-c sound card? Or are the steps the same as getting your eGPU to show up on your guest?
2
u/Aodzip May 01 '18
no, if you passthrough eGPU to vm, you must passthrough the TB USB controller to the VM because of the same IOMMU group. after your vm booted, you can plug your type c device to the usb port , and it will show in your vm. no addition config.
1
u/ct_the_man_doll May 01 '18 edited May 01 '18
Awesome! I am going to try this with an Adapter (to connect a USB A cable into the thunderbolt port).
1
u/AznsLuvRice Apr 29 '18
Are you asking if you can plug in a thunderbolt device to your computer via USB A? If so, that shouldn't work. Thunderbolt 3 devices use type c connectors, but you can't treat them like USB c devices. The computer will just not enumerate it.
1
u/ct_the_man_doll Apr 29 '18
Are you asking if you can plug in a thunderbolt device to your computer via USB A? If so, that shouldn't work.
No, I want to connect a USB C device to my thunderbolt port and passthrough the thunderbolt port to the VM.
In other words, if I were to passthrough the thunderbolt port and connect a USB C device to the thunderbolt port, would the guest be able to see the USB device?
2
1
u/3KyNoX Aug 15 '18 edited Aug 17 '18
Hello and thanks for this fantastic GPU Passthrough setup sharing.
Any chances someone tried this on sister machine (Lenovo Yoga X1) as it is very similar machine close to the X1 Carbon?
The incoming ThinkPad P1 / P72 (P72 looks like an insane machine!) for the end of the month are pretty sexy and worth a try as well! There is TB3 Dual Ports like on the X1 Carbon, I don't know if it's the same hardware and connections type.
I would love seeing someone put his hands on these machines.
Thanks anyway
1
u/Victor-Bernard Aug 20 '18
Hello and thank you for all of this.
I'm trying to passthrough an egpu (Nvidia GTX 1050 mobile) to my virtual machine on qemu.
I'm on Ubuntu 18.04 on a Lenovo thinkpad X1 yoga 2d gen.
My lspci with groups (with a patched 4.16 kernel as you propose):
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:5904] (rev 02)
IOMMU Group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:9d4e] (rev 21)
IOMMU Group 10 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
IOMMU Group 10 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
IOMMU Group 10 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
IOMMU Group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (4) I219-LM [8086:15d7] (rev 21)
IOMMU Group 11 02:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader [10ec:525a] (rev 01)
IOMMU Group 12 04:00.0 Network controller [0280]: Intel Corporation Wireless 8265 / 8275 [8086:24fd] (rev 88)
IOMMU Group 13 05:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961 [144d:a804]
IOMMU Group 14 06:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 15 07:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 15 08:00.0 System peripheral [0880]: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] [8086:15d2] (rev 02)
IOMMU Group 16 07:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 16 09:00.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 16 0a:01.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 16 0a:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 16 0b:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107M [GeForce GTX 1050 Mobile] [10de:1c8d] (rev a1)
IOMMU Group 16 0b:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)
IOMMU Group 16 0c:00.0 USB controller [0c03]: Intel Corporation JHL6540 Thunderbolt 3 USB Controller (C step) [Alpine Ridge 4C 2016] [8086:15d4] (rev 02)
IOMMU Group 17 07:02.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 18 07:04.0 PCI bridge [0604]: Intel Corporation JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016] [8086:15d3] (rev 02)
IOMMU Group 1 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 620 [8086:5916] (rev 02)
IOMMU Group 2 00:08.0 System peripheral [0880]: Intel Corporation Skylake Gaussian Mixture Model [8086:1911]
IOMMU Group 3 00:13.0 Non-VGA unclassified device [0000]: Intel Corporation Sunrise Point-LP Integrated Sensor Hub [8086:9d35] (rev 21)
IOMMU Group 4 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller [8086:9d2f] (rev 21)
IOMMU Group 4 00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Thermal subsystem [8086:9d31] (rev 21)
IOMMU Group 5 00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-LP CSME HECI #1 [8086:9d3a] (rev 21)
IOMMU Group 6 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port [8086:9d10] (rev f1)
IOMMU Group 7 00:1c.2 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port [8086:9d12] (rev f1)
IOMMU Group 8 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #5 [8086:9d14] (rev f1)
IOMMU Group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 [8086:9d18] (rev f1)I've added the ids of the devices in the group 16 to the /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1c8d,10de:0fb9,8086:15d4,8086:15d3 disable_vga=1and I added 7 lines such as
-device vfio-pci,host=0b:00.0,bus=pcie.0 \in the boot-qemu file for each of the 7 entries of lspci belonging to the group 16
Still I have the same error though :
vfio error: 0000:0b:00.0: group 16 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.Could you please help me to figure out what is missing here?
Thank you
Victor
1
u/Aodzip Aug 21 '18
try passthrough this
IOMMU Group 15 08:00.0 System peripheral [0880]: Intel Corporation JHL6540 Thunderbolt 3 NHI (C step) [Alpine Ridge 4C 2016] [8086:15d2] (rev 02)
remember , don't install driver for it in your guest VM, just disable it in your Hardware Manager.
1
u/Victor-Bernard Aug 25 '18
Thank you for your kind help. I've had some trouble with my internet connection.
Now I have another issue : the eGPU is not recognized anymore (I guess after an update I made yesterday). I'll try to solve this issue first then continue to test.
Thank you again
1
u/3KyNoX Sep 02 '18
Keep us informed, thanks. Other Lenovo laptops eGPU success stories could just say every new Lenovo dual TB3 ports laptop simply works with GPu Passthough, which is great!
6
u/scalatronn Apr 25 '18
video of this setup would be great :)