* PCI passthrough fails with Intel 82574L
@ 2014-05-08 14:55 Derek Piper
2014-05-08 15:10 ` Alex Williamson
0 siblings, 1 reply; 4+ messages in thread
From: Derek Piper @ 2014-05-08 14:55 UTC (permalink / raw
To: kvm@vger.kernel.org
Hi,
I am unable to get PCI passthrough to work on my Opteron system. I
keep getting an 'adapter reset' within the VM that has the PCI ethernet
assigned to it. It is running 2 x 6 core Opterons on a SuperMicro H8DCL
motherboard that has 2 Intel 82574L network interfaces on it.
Host's lspci (relevant card):
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
Connection
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
Connection
Both host and VM are running Debian 7 (Wheezy). I have the latest qemu-kvm
from debian backports (1.7.0).
I am running the virtual hosts via virt-manager and assigning the PCI express
ethernet card at 03:00.0. There is no configuration on the host for that
ethernet card, only the first one (02:00.0) is in use. I have enabled the
options kvm allow_unsafe_assigned_interrupts=1
When running the VM with the PCI express adapter assigned to it, and configured
for the network it is connected to, it does not work. I simply get the
'adapter reset' message again and again.
The host shows the device being claimed by pci-stub as it gets assigned to the
guest and getting the IRQs. On the guest there is a stack trace and also
adapter resets.
I am hoping to run a number of different virtual machines off of this host. Due
to network policies in place (no devices in promiscuous mode get routed) I
cannot simply place the host's interface to be shared with different IPs, so
must do one physical ethernet interface each.
Any help would be greatly appreciated.
Thanks,
Derek
--
Derek Piper - dcpiper@indiana.edu - (812) 855 5560
System Administrator / Informatics Specialist
Molecular Structure Center (IUMSC), Chemistry
Indiana University, Bloomington, Indiana
http://www.iumsc.indiana.edu/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCI passthrough fails with Intel 82574L
2014-05-08 14:55 PCI passthrough fails with Intel 82574L Derek Piper
@ 2014-05-08 15:10 ` Alex Williamson
[not found] ` <201405081128.24349.dcpiper@indiana.edu>
0 siblings, 1 reply; 4+ messages in thread
From: Alex Williamson @ 2014-05-08 15:10 UTC (permalink / raw
To: Derek Piper; +Cc: kvm@vger.kernel.org
On Thu, 2014-05-08 at 10:55 -0400, Derek Piper wrote:
>
> Hi,
>
> I am unable to get PCI passthrough to work on my Opteron system. I
> keep getting an 'adapter reset' within the VM that has the PCI ethernet
> assigned to it. It is running 2 x 6 core Opterons on a SuperMicro H8DCL
> motherboard that has 2 Intel 82574L network interfaces on it.
>
> Host's lspci (relevant card):
> 02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
> 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
>
> Both host and VM are running Debian 7 (Wheezy). I have the latest qemu-kvm
> from debian backports (1.7.0).
Latest is 2.0, also what's the kernel version?
> I am running the virtual hosts via virt-manager and assigning the PCI express
> ethernet card at 03:00.0. There is no configuration on the host for that
> ethernet card, only the first one (02:00.0) is in use. I have enabled the
>
> options kvm allow_unsafe_assigned_interrupts=1
>
> When running the VM with the PCI express adapter assigned to it, and configured
> for the network it is connected to, it does not work. I simply get the
> 'adapter reset' message again and again.
>
> The host shows the device being claimed by pci-stub as it gets assigned to the
> guest and getting the IRQs. On the guest there is a stack trace and also
> adapter resets.
Relevant guest dmesg would be helpful.
> I am hoping to run a number of different virtual machines off of this host. Due
> to network policies in place (no devices in promiscuous mode get routed) I
> cannot simply place the host's interface to be shared with different IPs, so
> must do one physical ethernet interface each.
>
> Any help would be greatly appreciated.
Does it work using vfio-pci to do the assignment rather than pci-assign?
Newer versions of libvirt will use this by default, you may need to add
a driver tag with an older version, or upgrade if it's too old for vfio.
See xml example here:
http://libvirt.org/formatdomain.html#elementsNICSHostdev
Thanks,
Alex
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCI passthrough fails with Intel 82574L
[not found] ` <201405081128.24349.dcpiper@indiana.edu>
@ 2014-05-08 15:38 ` Alex Williamson
2014-05-09 16:04 ` Derek Piper
0 siblings, 1 reply; 4+ messages in thread
From: Alex Williamson @ 2014-05-08 15:38 UTC (permalink / raw
To: Derek Piper; +Cc: kvm@vger.kernel.org
On Thu, 2014-05-08 at 11:28 -0400, Derek Piper wrote:
>
> Attached is the dmesg from the host and the vm. I tried email these
> before but it seemed to make the message get rejected from the mailing list.
>
> I attached the PCIe device via the virt-manager, which modified the config
> like this:
>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
> </hostdev>
>
> Debian wheezy kernel is 3.2.0-4-amd64. Yes, maybe what I have is too old.
> Seeing what was said about VFIO, I will give that a go.
That's quite old. You'll need 3.6 or (preferably) newer for vfio. I
expect there have also been plenty of legacy KVM assignment fixes since
that kernel. Please upgrade and report back. Thanks,
Alex
> On Thursday, May 08, 2014 11:10:49 AM Alex Williamson wrote:
> > On Thu, 2014-05-08 at 10:55 -0400, Derek Piper wrote:
> > > Hi,
> > >
> > > I am unable to get PCI passthrough to work on my Opteron system.
> > > I
> > >
> > > keep getting an 'adapter reset' within the VM that has the PCI ethernet
> > > assigned to it. It is running 2 x 6 core Opterons on a SuperMicro H8DCL
> > > motherboard that has 2 Intel 82574L network interfaces on it.
> > >
> > > Host's lspci (relevant card):
> > > 02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > > Connection
> > > 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > > Connection
> > >
> > > Both host and VM are running Debian 7 (Wheezy). I have the latest
> > > qemu-kvm from debian backports (1.7.0).
> >
> > Latest is 2.0, also what's the kernel version?
> >
> > > I am running the virtual hosts via virt-manager and assigning the PCI
> > > express ethernet card at 03:00.0. There is no configuration on the host
> > > for that ethernet card, only the first one (02:00.0) is in use. I have
> > > enabled the
> > >
> > > options kvm allow_unsafe_assigned_interrupts=1
> > >
> > > When running the VM with the PCI express adapter assigned to it, and
> > > configured for the network it is connected to, it does not work. I
> > > simply get the 'adapter reset' message again and again.
> > >
> > > The host shows the device being claimed by pci-stub as it gets assigned
> > > to the guest and getting the IRQs. On the guest there is a stack trace
> > > and also adapter resets.
> >
> > Relevant guest dmesg would be helpful.
> >
> > > I am hoping to run a number of different virtual machines off of this
> > > host. Due to network policies in place (no devices in promiscuous mode
> > > get routed) I cannot simply place the host's interface to be shared with
> > > different IPs, so must do one physical ethernet interface each.
> > >
> > > Any help would be greatly appreciated.
> >
> > Does it work using vfio-pci to do the assignment rather than pci-assign?
> > Newer versions of libvirt will use this by default, you may need to add
> > a driver tag with an older version, or upgrade if it's too old for vfio.
> > See xml example here:
> >
> > http://libvirt.org/formatdomain.html#elementsNICSHostdev
> >
> > Thanks,
> >
> > Alex
>
> --
> Derek Piper - dcpiper@indiana.edu - (812) 855 5560
> System Administrator / Informatics Specialist
> Molecular Structure Center (IUMSC), Chemistry
> Indiana University, Bloomington, Indiana
> http://www.iumsc.indiana.edu/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: PCI passthrough fails with Intel 82574L
2014-05-08 15:38 ` Alex Williamson
@ 2014-05-09 16:04 ` Derek Piper
0 siblings, 0 replies; 4+ messages in thread
From: Derek Piper @ 2014-05-09 16:04 UTC (permalink / raw
To: Alex Williamson; +Cc: kvm@vger.kernel.org
I'm happy to report that after updating the kernel to 3.14.3 (and
enabling the IPv4 NAT in the config) I was able to assign the device using
virt-manager (so it was a regular 'hostdev' assignment). Networking works
great, and the virtual machine can have both an internal IP on the virtual
network and also have its own ethernet card on the external network. Packets
travel well and no adapter resets any more.
Thanks for prodding me in the right direction with regard to the kernel.
Derek
On Thursday, May 08, 2014 11:38:47 AM Alex Williamson wrote:
> On Thu, 2014-05-08 at 11:28 -0400, Derek Piper wrote:
> > Attached is the dmesg from the host and the vm. I tried email
> > these
> >
> > before but it seemed to make the message get rejected from the mailing
> > list.
> >
> > I attached the PCIe device via the virt-manager, which modified
> > the config
> >
> > like this:
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> >
> > <source>
> >
> > <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
> >
> > </source>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> >
> > function='0x0'/>
> >
> > </hostdev>
> >
> > Debian wheezy kernel is 3.2.0-4-amd64. Yes, maybe what I have is
> > too old.
> >
> > Seeing what was said about VFIO, I will give that a go.
>
> That's quite old. You'll need 3.6 or (preferably) newer for vfio. I
> expect there have also been plenty of legacy KVM assignment fixes since
> that kernel. Please upgrade and report back. Thanks,
>
> Alex
>
> > On Thursday, May 08, 2014 11:10:49 AM Alex Williamson wrote:
> > > On Thu, 2014-05-08 at 10:55 -0400, Derek Piper wrote:
> > > > Hi,
> > > >
> > > > I am unable to get PCI passthrough to work on my Opteron
> > > > system. I
> > > >
> > > > keep getting an 'adapter reset' within the VM that has the PCI
> > > > ethernet assigned to it. It is running 2 x 6 core Opterons on a
> > > > SuperMicro H8DCL motherboard that has 2 Intel 82574L network
> > > > interfaces on it.
> > > >
> > > > Host's lspci (relevant card):
> > > > 02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > > > Connection
> > > > 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > > > Connection
> > > >
> > > > Both host and VM are running Debian 7 (Wheezy). I have the latest
> > > > qemu-kvm from debian backports (1.7.0).
> > >
> > > Latest is 2.0, also what's the kernel version?
> > >
> > > > I am running the virtual hosts via virt-manager and assigning the PCI
> > > > express ethernet card at 03:00.0. There is no configuration on the
> > > > host for that ethernet card, only the first one (02:00.0) is in use.
> > > > I have enabled the
> > > >
> > > > options kvm allow_unsafe_assigned_interrupts=1
> > > >
> > > > When running the VM with the PCI express adapter assigned to it, and
> > > > configured for the network it is connected to, it does not work. I
> > > > simply get the 'adapter reset' message again and again.
> > > >
> > > > The host shows the device being claimed by pci-stub as it gets
> > > > assigned to the guest and getting the IRQs. On the guest there is a
> > > > stack trace and also adapter resets.
> > >
> > > Relevant guest dmesg would be helpful.
> > >
> > > > I am hoping to run a number of different virtual machines off of this
> > > > host. Due to network policies in place (no devices in promiscuous
> > > > mode get routed) I cannot simply place the host's interface to be
> > > > shared with different IPs, so must do one physical ethernet
> > > > interface each.
> > > >
> > > > Any help would be greatly appreciated.
> > >
> > > Does it work using vfio-pci to do the assignment rather than
> > > pci-assign? Newer versions of libvirt will use this by default, you
> > > may need to add a driver tag with an older version, or upgrade if it's
> > > too old for vfio. See xml example here:
> > >
> > > http://libvirt.org/formatdomain.html#elementsNICSHostdev
> > >
> > > Thanks,
> > >
> > > Alex
--
Derek Piper - dcpiper@indiana.edu - (812) 855 5560
System Administrator / Informatics Specialist
Molecular Structure Center (IUMSC), Chemistry
Indiana University, Bloomington, Indiana
http://www.iumsc.indiana.edu/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-05-09 16:04 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-05-08 14:55 PCI passthrough fails with Intel 82574L Derek Piper
2014-05-08 15:10 ` Alex Williamson
[not found] ` <201405081128.24349.dcpiper@indiana.edu>
2014-05-08 15:38 ` Alex Williamson
2014-05-09 16:04 ` Derek Piper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.