Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Xen: Users

NIC PCI-Passthrough problems (PV domU driver crashes)

 

 

Xen users RSS feed   Index | Next | Previous | View Threaded


jmark2850 at gmail

Aug 5, 2013, 12:37 PM

Post #1 of 7 (51 views)
Permalink
NIC PCI-Passthrough problems (PV domU driver crashes)

Hi,

I have been trying to get NIC PCI Passthrough to work on Xen 4.3.0 and have
hit a wall. On attaching the NIC the domU network driver (igb) crashes.

I followed the Xen PCI Passthrough wiki
http://wiki.xen.org/wiki/Xen_PCI_Passthrough and am using dynamic device
assignment with xl.

Trying to passthrough this device:

root [at] dom:~# lspci|grep 82575EB
05:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network
Connection (rev 02)

xl shows the pci devices are ready to be plugged-in:

root [at] dom:~# xl pci-assignable-list
0000:00:1d.2
0000:05:00.1

Setting up the PV domU (ubuntu precise):

root [at] ubunt:~# mount -t xenfs none /proc/xen
root [at] ubunt:~# modprobe xen_pcifront
root [at] ubunt:~# modprobe xen_evtchn
root [at] ubunt:~# modprobe xen_gntdev
root [at] ubunt:~# modprobe xenfs
root [at] ubunt:~# ls /proc/xen
capabilities privcmd xenbus

Attaching the device with xl:

root [at] dom:~# xl -f pci-attach 54 0000:05:00.1,permissive=1

This causes the domU driver crash (please see log after the jump). The pci
device is visible though:

root [at] ubunt:~# lspci
00:00.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network
Connection (rev 02)

And xl confirms:

root [at] dom:~# xl pci-list 54
Vdev Device
00.0 0000:05:00.1

System configuration:
--------------------------------

Dom0 kernel: 3.9.0-030900-generic
DomU kernel: 3.2.0-51-generic
Dom0 memory: 8GB (kernel cmdline: dom0_mem=8G,max:8G)
IO Virtualisation: Disabled/Unavailable

Also tried a more recent domU kernel with no luck.

DomU config:
--------------------

name = "ubuntu"
memory = 2048
disk = ['/stor/ubuntu-domu.img,raw,xvda,w']
vif = [' ']
bootloader="pygrub"
on_reboot="restart"
extra = "iommu=soft swiotlb=force console=hvc0"
e820_host=1

Am I missing steps? Any help would be great, this is driving me nuts.
I also tried passing-through a USB controller and it worked fine.

Thanks for your time.
Mark

--snip--
kern.log

Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Initializing cgroup subsys
cpuset
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Initializing cgroup subsys cpu
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Linux version
3.2.0-51-generic (buildd [at] allspic) (gcc version 4.6.3 (Ubuntu/Linaro
4.6.3-1ubuntu5) ) #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 (Ubuntu
3.2.0-51.77-generic 3.2.48)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Command line:
root=/dev/mapper/ubuntu--vg-root ro console=hvc0 quiet splash $vt_handoff
iommu=soft swiotlb=force console=hvc0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] KERNEL supported cpus:
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Intel GenuineIntel
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] AMD AuthenticAMD
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Centaur CentaurHauls
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] ACPI in unprivileged domain
disabled
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] 1-1 mapping on 80000->100000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Released 0 pages of unused
memory
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Set 524288 page(s) to 1-1
mapping
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] BIOS-provided physical RAM
map:
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 0000000000000000 -
00000000000a0000 (usable)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000000a0000 -
0000000000100000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 0000000000100000 -
0000000080000000 (usable)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 0000000080000000 -
00000000bf78e000 (unusable)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000bf78e000 -
00000000bf790000 type 9
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000bf790000 -
00000000bf79e000 (ACPI data)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000bf79e000 -
00000000bf7d0000 (ACPI NVS)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000bf7d0000 -
00000000bf7e0000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000bf7ec000 -
00000000c0000000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000e0000000 -
00000000f0000000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000fee00000 -
00000000fee01000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: 00000000ffc00000 -
0000000100000000 (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] NX (Execute Disable)
protection: active
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMI not present or invalid.
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] e820 update range:
0000000000000000 - 0000000000010000 (usable) ==> (reserved)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] e820 remove range:
00000000000a0000 - 0000000000100000 (usable)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] No AGP bridge found
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] last_pfn = 0x80000
max_arch_pfn = 0x400000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] initial memory mapped : 0 -
047c1000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Base memory trampoline at
[ffff88000009b000] 9b000 size 20480
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] init_memory_mapping:
0000000000000000-0000000080000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] 0000000000 - 0080000000 page
4k
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] kernel direct mapping tables
up to 80000000 @ bfd000-1000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] xen: setting RW the range
fd2000 - 1000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] RAMDISK: 02060000 - 047c1000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] NUMA turned off
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Faking a node at
0000000000000000-0000000080000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Initmem setup node 0
0000000000000000-0000000080000000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] NODE_DATA [000000007fffb000
- 000000007fffffff]
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Zone PFN ranges:
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA 0x00000010 ->
0x00001000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA32 0x00001000 ->
0x00100000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Normal empty
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Movable zone start PFN for
each node
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] early_node_map[2] active PFN
ranges
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] 0: 0x00000010 ->
0x000000a0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] 0: 0x00000100 ->
0x00080000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] On node 0 totalpages: 524176
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA zone: 64 pages used for
memmap
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA zone: 986 pages reserved
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA zone: 2934 pages, LIFO
batch:0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA32 zone: 8128 pages used
for memmap
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] DMA32 zone: 512064 pages,
LIFO batch:31
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] SFI: Simple Firmware
Interface v0.81 http://simplefirmware.org
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] SMP: Allowing 1 CPUs, 0
hotplug CPUs
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] No local APIC present
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] APIC: disable apic facility
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] APIC: switched to apic NOOP
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] nr_irqs_gsi: 16
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] PM: Registered nosave memory:
00000000000a0000 - 0000000000100000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Allocating PCI resources
starting at c0000000 (gap: c0000000:20000000)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Booting paravirtualized
kernel on Xen
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen version: 4.3.0
(preserve-AD)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] setup_percpu: NR_CPUS:256
nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] PERCPU: Embedded 28 pages/cpu
@ffff88007fc00000 s83136 r8192 d23360 u2097152
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] pcpu-alloc: s83136 r8192
d23360 u2097152 alloc=1*2097152
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] pcpu-alloc: [0] 0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Built 1 zonelists in Node
order, mobility grouping on. Total pages: 514998
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Policy zone: DMA32
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Kernel command line:
root=/dev/mapper/ubuntu--vg-root ro console=hvc0 quiet splash $vt_handoff
iommu=soft swiotlb=force console=hvc0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] PID hash table entries: 4096
(order: 3, 32768 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Placing 64MB software IO TLB
between ffff880079600000 - ffff88007d600000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] software IO TLB at phys
0x79600000 - 0x7d600000
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Memory: 1933356k/2097152k
available (6583k kernel code, 448k absent, 163348k reserved, 6623k data,
924k init)
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] SLUB: Genslabs=15,
HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Hierarchical RCU
implementation.
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] RCU dyntick-idle grace-period
acceleration is enabled.
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] NR_IRQS:16640 nr_irqs:256 16
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Console: colour dummy device
80x25
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] console [tty0] enabled
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] console [hvc0] enabled
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] allocated 16777216 bytes of
page_cgroup
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] please try
'cgroup_disable=memory' option if you don't want memory cgroups
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Xen: using vcpuop timer
interface
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] installing Xen timer for CPU 0
Aug 5 15:03:00 ubuntu kernel: [ 0.000000] Detected 2666.864 MHz
processor.
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Calibrating delay loop
(skipped), value calculated using timer frequency.. 5333.72 BogoMIPS
(lpj=10667456)
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] pid_max: default: 32768
minimum: 301
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Security Framework initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] AppArmor: AppArmor initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Yama: becoming mindful.
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Dentry cache hash table
entries: 262144 (order: 9, 2097152 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Inode-cache hash table
entries: 131072 (order: 8, 1048576 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Mount-cache hash table
entries: 256
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
cpuacct
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
memory
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
devices
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
freezer
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
blkio
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] Initializing cgroup subsys
perf_event
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] ENERGY_PERF_BIAS: Set to
'normal', was 'performance'
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] ENERGY_PERF_BIAS: View and
update with x86_energy_perf_policy(8)
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] CPU: Physical Processor ID: 0
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] CPU: Processor Core ID: 0
Aug 5 15:03:00 ubuntu kernel: [ 0.004000] SMP alternatives: switching
to UP code
Aug 5 15:03:00 ubuntu kernel: [ 0.034445] Freeing SMP alternatives: 24k
freed
Aug 5 15:03:00 ubuntu kernel: [ 0.034455] ftrace: allocating 26587
entries in 105 pages
Aug 5 15:03:00 ubuntu kernel: [ 0.036046] cpu 0 spinlock event irq 17
Aug 5 15:03:00 ubuntu kernel: [ 0.036057] Performance Events:
unsupported p6 CPU model 44 no PMU driver, software events only.
Aug 5 15:03:00 ubuntu kernel: [ 0.036166] NMI watchdog disabled (cpu0):
hardware events not enabled
Aug 5 15:03:00 ubuntu kernel: [ 0.036181] Brought up 1 CPUs
Aug 5 15:03:00 ubuntu kernel: [ 0.036295] devtmpfs: initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.037004] EVM: security.selinux
Aug 5 15:03:00 ubuntu kernel: [ 0.037005] EVM: security.SMACK64
Aug 5 15:03:00 ubuntu kernel: [ 0.037006] EVM: security.capability
Aug 5 15:03:00 ubuntu kernel: [ 0.037031] PM: Registering ACPI NVS
region at bf79e000 (204800 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.037663] Grant table initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.037704] print_constraints: dummy:
Aug 5 15:03:00 ubuntu kernel: [ 0.057311] RTC time: 165:165:165, date:
165/165/65
Aug 5 15:03:00 ubuntu kernel: [ 0.057344] NET: Registered protocol
family 16
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] PCI: setting up Xen PCI
frontend stub
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] PCI: pci_cache_line_size set
to 64 bytes
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] bio: create slab <bio-0> at 0
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] ACPI: Interpreter disabled.
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] xen/balloon: Initialising
balloon driver.
Aug 5 15:03:00 ubuntu kernel: [ 0.057505] xen-balloon: Initialising
balloon driver.
Aug 5 15:03:00 ubuntu kernel: [ 0.060043] vgaarb: loaded
Aug 5 15:03:00 ubuntu kernel: [ 0.060118] i2c-core: driver [aat2870]
using legacy suspend method
Aug 5 15:03:00 ubuntu kernel: [ 0.060120] i2c-core: driver [aat2870]
using legacy resume method
Aug 5 15:03:00 ubuntu kernel: [ 0.060169] SCSI subsystem initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.060201] libata version 3.00 loaded.
Aug 5 15:03:00 ubuntu kernel: [ 0.060238] usbcore: registered new
interface driver usbfs
Aug 5 15:03:00 ubuntu kernel: [ 0.060246] usbcore: registered new
interface driver hub
Aug 5 15:03:00 ubuntu kernel: [ 0.060279] usbcore: registered new
device driver usb
Aug 5 15:03:00 ubuntu kernel: [ 0.060370] PCI: System does not support
PCI
Aug 5 15:03:00 ubuntu kernel: [ 0.060372] PCI: System does not support
PCI
Aug 5 15:03:00 ubuntu kernel: [ 0.060444] NetLabel: Initializing
Aug 5 15:03:00 ubuntu kernel: [ 0.060445] NetLabel: domain hash size =
128
Aug 5 15:03:00 ubuntu kernel: [ 0.060446] NetLabel: protocols =
UNLABELED CIPSOv4
Aug 5 15:03:00 ubuntu kernel: [ 0.060454] NetLabel: unlabeled traffic
allowed by default
Aug 5 15:03:00 ubuntu kernel: [ 0.060495] Switching to clocksource xen
Aug 5 15:03:00 ubuntu kernel: [ 0.066220] AppArmor: AppArmor Filesystem
Enabled
Aug 5 15:03:00 ubuntu kernel: [ 0.066232] pnp: PnP ACPI: disabled
Aug 5 15:03:00 ubuntu kernel: [ 0.067733] PCI: max bus depth: 0
pci_try_num: 1
Aug 5 15:03:00 ubuntu kernel: [ 0.067759] NET: Registered protocol
family 2
Aug 5 15:03:00 ubuntu kernel: [ 0.067906] IP route cache hash table
entries: 65536 (order: 7, 524288 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.068906] TCP established hash table
entries: 262144 (order: 10, 4194304 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.069714] TCP bind hash table entries:
65536 (order: 8, 1048576 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.069859] TCP: Hash tables configured
(established 262144 bind 65536)
Aug 5 15:03:00 ubuntu kernel: [ 0.069861] TCP reno registered
Aug 5 15:03:00 ubuntu kernel: [ 0.069872] UDP hash table entries: 1024
(order: 3, 32768 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.069886] UDP-Lite hash table entries:
1024 (order: 3, 32768 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.069937] NET: Registered protocol
family 1
Aug 5 15:03:00 ubuntu kernel: [ 0.069945] PCI: CLS 0 bytes, default 64
Aug 5 15:03:00 ubuntu kernel: [ 0.070003] platform rtc_cmos: registered
platform RTC device (no PNP device found)
Aug 5 15:03:00 ubuntu kernel: [ 0.070188] audit: initializing netlink
socket (disabled)
Aug 5 15:03:00 ubuntu kernel: [ 0.070194] type=2000
audit(1375729377.495:1): initialized
Aug 5 15:03:00 ubuntu kernel: [ 0.088968] Trying to unpack rootfs image
as initramfs...
Aug 5 15:03:00 ubuntu kernel: [ 0.112400] HugeTLB registered 2 MB page
size, pre-allocated 0 pages
Aug 5 15:03:00 ubuntu kernel: [ 0.123314] Freeing initrd memory: 40324k
freed
Aug 5 15:03:00 ubuntu kernel: [ 0.133822] VFS: Disk quotas dquot_6.5.2
Aug 5 15:03:00 ubuntu kernel: [ 0.133870] Dquot-cache hash table
entries: 512 (order 0, 4096 bytes)
Aug 5 15:03:00 ubuntu kernel: [ 0.134283] fuse init (API version 7.17)
Aug 5 15:03:00 ubuntu kernel: [ 0.134358] msgmni has been set to 3854
Aug 5 15:03:00 ubuntu kernel: [ 0.134629] Block layer SCSI generic
(bsg) driver version 0.4 loaded (major 253)
Aug 5 15:03:00 ubuntu kernel: [ 0.134656] io scheduler noop registered
Aug 5 15:03:00 ubuntu kernel: [ 0.134658] io scheduler deadline
registered
Aug 5 15:03:00 ubuntu kernel: [ 0.134684] io scheduler cfq registered
(default)
Aug 5 15:03:00 ubuntu kernel: [ 0.134745] pci_hotplug: PCI Hot Plug PCI
Core version: 0.5
Aug 5 15:03:00 ubuntu kernel: [ 0.134763] pciehp: PCI Express Hot Plug
Controller Driver version: 0.4
Aug 5 15:03:00 ubuntu kernel: [ 0.135030] Serial: 8250/16550 driver, 32
ports, IRQ sharing enabled
Aug 5 15:03:00 ubuntu kernel: [ 0.135934] Linux agpgart interface v0.103
Aug 5 15:03:00 ubuntu kernel: [ 0.136964] brd: module loaded
Aug 5 15:03:00 ubuntu kernel: [ 0.137510] loop: module loaded
Aug 5 15:03:00 ubuntu kernel: [ 0.140488] blkfront device/vbd/51712
num-ring-pages 1 nr_ents 32.
Aug 5 15:03:00 ubuntu kernel: [ 0.140935] Fixed MDIO Bus: probed
Aug 5 15:03:00 ubuntu kernel: [ 0.140950] tun: Universal TUN/TAP device
driver, 1.6
Aug 5 15:03:00 ubuntu kernel: [ 0.140952] tun: (C) 1999-2004 Max
Krasnyansky <maxk [at] qualcomm>
Aug 5 15:03:00 ubuntu kernel: [ 0.140986] PPP generic driver version
2.4.2
Aug 5 15:03:00 ubuntu kernel: [ 0.141016] Initialising Xen virtual
ethernet driver.
Aug 5 15:03:00 ubuntu kernel: [ 0.142408] ehci_hcd: USB 2.0 'Enhanced'
Host Controller (EHCI) Driver
Aug 5 15:03:00 ubuntu kernel: [ 0.142426] ohci_hcd: USB 1.1 'Open' Host
Controller (OHCI) Driver
Aug 5 15:03:00 ubuntu kernel: [ 0.142433] uhci_hcd: USB Universal Host
Controller Interface driver
Aug 5 15:03:00 ubuntu kernel: [ 0.142482] usbcore: registered new
interface driver libusual
Aug 5 15:03:00 ubuntu kernel: [ 0.142509] i8042: PNP: No PS/2
controller found. Probing ports directly.
Aug 5 15:03:00 ubuntu kernel: [ 1.143504] i8042: No controller found
Aug 5 15:03:00 ubuntu kernel: [ 1.143585] mousedev: PS/2 mouse device
common for all mice
Aug 5 15:03:00 ubuntu kernel: [ 1.183368] rtc_cmos rtc_cmos: rtc core:
registered rtc_cmos as rtc0
Aug 5 15:03:00 ubuntu kernel: [ 1.183399] rtc_cmos: probe of rtc_cmos
failed with error -38
Aug 5 15:03:00 ubuntu kernel: [ 1.183461] device-mapper: uevent:
version 1.0.3
Aug 5 15:03:00 ubuntu kernel: [ 1.183515] device-mapper: ioctl:
4.22.0-ioctl (2011-10-19) initialised: dm-devel [at] redhat
Aug 5 15:03:00 ubuntu kernel: [ 1.183520] EFI Variables Facility v0.08
2004-May-17
Aug 5 15:03:00 ubuntu kernel: [ 1.183732] TCP cubic registered
Aug 5 15:03:00 ubuntu kernel: [ 1.183821] NET: Registered protocol
family 10
Aug 5 15:03:00 ubuntu kernel: [ 1.184451] NET: Registered protocol
family 17
Aug 5 15:03:00 ubuntu kernel: [ 1.184455] Registering the dns_resolver
key type
Aug 5 15:03:00 ubuntu kernel: [ 1.184541] PM: Hibernation image not
present or could not be loaded.
Aug 5 15:03:00 ubuntu kernel: [ 1.184552] registered taskstats version 1
Aug 5 15:03:00 ubuntu kernel: [ 1.197518] blkfront: xvda: flush
diskcache: enabled
Aug 5 15:03:00 ubuntu kernel: [ 1.199535] xvda: xvda1 xvda2 < xvda5 >
Aug 5 15:03:00 ubuntu kernel: [ 1.292087] Magic number: 1:252:3141
Aug 5 15:03:00 ubuntu kernel: [ 1.292112]
/build/buildd/linux-3.2.0/drivers/rtc/hctosys.c: unable to open rtc device
(rtc0)
Aug 5 15:03:00 ubuntu kernel: [ 1.292117] BIOS EDD facility v0.16
2004-Jun-25, 0 devices found
Aug 5 15:03:00 ubuntu kernel: [ 1.292119] EDD information not available.
Aug 5 15:03:00 ubuntu kernel: [ 1.292772] Freeing unused kernel memory:
924k freed
Aug 5 15:03:00 ubuntu kernel: [ 1.293033] Write protecting the kernel
read-only data: 12288k
Aug 5 15:03:00 ubuntu kernel: [ 1.301058] Freeing unused kernel memory:
1592k freed
Aug 5 15:03:00 ubuntu kernel: [ 1.302164] Freeing unused kernel memory:
1188k freed
Aug 5 15:03:00 ubuntu kernel: [ 1.660898] EXT4-fs (dm-0): INFO:
recovery required on readonly filesystem
Aug 5 15:03:00 ubuntu kernel: [ 1.660902] EXT4-fs (dm-0): write access
will be enabled during recovery
Aug 5 15:03:00 ubuntu kernel: [ 1.669882] EXT4-fs (dm-0): recovery
complete
Aug 5 15:03:00 ubuntu kernel: [ 1.670674] EXT4-fs (dm-0): mounted
filesystem with ordered data mode. Opts: (null)
Aug 5 15:03:00 ubuntu kernel: [ 2.036595] Adding 1048572k swap on
/dev/mapper/ubuntu--vg-swap_1. Priority:-1 extents:1 across:1048572k SS
Aug 5 15:03:00 ubuntu kernel: [ 2.053532] ADDRCONF(NETDEV_UP): eth0:
link is not ready
Aug 5 15:03:00 ubuntu kernel: [ 2.137511] EXT4-fs (dm-0): re-mounted.
Opts: errors=remount-ro
Aug 5 15:03:00 ubuntu kernel: [ 2.152251] lp: driver loaded but no
devices found
Aug 5 15:03:00 ubuntu kernel: [ 2.842758] type=1400
audit(1375729380.267:2): apparmor="STATUS" operation="profile_load"
name="/sbin/dhclient" pid=463 comm="apparmor_parser"
Aug 5 15:03:00 ubuntu kernel: [ 2.844323] type=1400
audit(1375729380.271:3): apparmor="STATUS" operation="profile_load"
name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=463
comm="apparmor_parser"
Aug 5 15:03:00 ubuntu kernel: [ 2.844519] type=1400
audit(1375729380.271:4): apparmor="STATUS" operation="profile_load"
name="/usr/lib/connman/scripts/dhclient-script" pid=463
comm="apparmor_parser"
Aug 5 15:03:01 ubuntu kernel: [ 4.260146] init: failsafe main process
(521) killed by TERM signal
Aug 5 15:03:01 ubuntu kernel: [ 4.321428] type=1400
audit(1375729381.747:5): apparmor="STATUS" operation="profile_replace"
name="/sbin/dhclient" pid=783 comm="apparmor_parser"
Aug 5 15:03:01 ubuntu kernel: [ 4.321789] type=1400
audit(1375729381.747:6): apparmor="STATUS" operation="profile_replace"
name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=783
comm="apparmor_parser"
Aug 5 15:03:01 ubuntu kernel: [ 4.321987] type=1400
audit(1375729381.747:7): apparmor="STATUS" operation="profile_replace"
name="/usr/lib/connman/scripts/dhclient-script" pid=783
comm="apparmor_parser"
Aug 5 15:03:01 ubuntu kernel: [ 4.327211] type=1400
audit(1375729381.751:8): apparmor="STATUS" operation="profile_load"
name="/usr/sbin/tcpdump" pid=785 comm="apparmor_parser"
Aug 5 15:03:02 ubuntu kernel: [ 5.445164] init: udev-fallback-graphics
main process (898) terminated with status 1
Aug 5 15:03:02 ubuntu kernel: [ 5.473583] init: plymouth-splash main
process (904) terminated with status 1
Aug 5 15:03:10 ubuntu kernel: [ 13.184119] eth0: no IPv6 routers present
Aug 5 15:03:45 ubuntu kernel: [ 48.306564] Event-channel device
installed.
Aug 5 15:05:17 ubuntu kernel: [ 139.755233] pcifront pci-0: Installing
PCI frontend
Aug 5 15:05:17 ubuntu kernel: [ 139.755392] pcifront pci-0: Creating PCI
Frontend Bus 0000:00
Aug 5 15:05:17 ubuntu kernel: [ 139.755999] pci 0000:00:00.1: [8086:10a7]
type 0 class 0x000200
Aug 5 15:05:17 ubuntu kernel: [ 139.756171] pci 0000:00:00.1: reg 10:
[mem 0xfa3e0000-0xfa3fffff]
Aug 5 15:05:17 ubuntu kernel: [ 139.756295] pci 0000:00:00.1: reg 14:
[mem 0xfa600000-0xfa7fffff]
Aug 5 15:05:17 ubuntu kernel: [ 139.756410] pci 0000:00:00.1: reg 18: [io
0xe880-0xe89f]
Aug 5 15:05:17 ubuntu kernel: [ 139.756525] pci 0000:00:00.1: reg 1c:
[mem 0xfa3dc000-0xfa3dffff]
Aug 5 15:05:17 ubuntu kernel: [ 139.757636] pcifront pci-0: New device on
0000:00:00.01 found.
Aug 5 15:05:17 ubuntu kernel: [ 139.767322] pcifront pci-0: claiming
resource 0000:00:00.1/0
Aug 5 15:05:17 ubuntu kernel: [ 139.767328] pcifront pci-0: claiming
resource 0000:00:00.1/1
Aug 5 15:05:17 ubuntu kernel: [ 139.767331] pcifront pci-0: claiming
resource 0000:00:00.1/2
Aug 5 15:05:17 ubuntu kernel: [ 139.767335] pcifront pci-0: claiming
resource 0000:00:00.1/3
Aug 5 15:05:17 ubuntu kernel: [ 139.775084] dca service started, version
1.12.1
Aug 5 15:05:17 ubuntu kernel: [ 139.781460] Intel(R) Gigabit Ethernet
Network Driver - version 3.2.10-k
Aug 5 15:05:17 ubuntu kernel: [ 139.781465] Copyright (c) 2007-2011 Intel
Corporation.
Aug 5 15:05:17 ubuntu kernel: [ 139.781534] igb 0000:00:00.1: enabling
device (0000 -> 0002)
Aug 5 15:05:17 ubuntu kernel: [ 139.781688] igb 0000:00:00.1: Xen PCI
mapped GSI42 to IRQ27
Aug 5 15:05:17 ubuntu kernel: [ 139.781816] igb 0000:00:00.1: setting
latency timer to 64
Aug 5 15:05:17 ubuntu kernel: [ 139.793427] BUG: unable to handle kernel
paging request at ffffc9000031800c
Aug 5 15:05:17 ubuntu kernel: [ 139.793442] IP: [<ffffffff8134b30b>]
msix_program_entries+0x7b/0xa0
Aug 5 15:05:17 ubuntu kernel: [ 139.793457] PGD 77e1c067 PUD 77e1d067 PMD
77f23067 PTE 80100000fa3dc465
Aug 5 15:05:17 ubuntu kernel: [ 139.793471] Oops: 0003 [#1] SMP
Aug 5 15:05:17 ubuntu kernel: [ 139.793479] CPU 0
Aug 5 15:05:17 ubuntu kernel: [ 139.793483] Modules linked in: igb(+) dca
xen_gntdev xen_evtchn xen_pcifront xenfs ext2 lp parport
Aug 5 15:05:17 ubuntu kernel: [ 139.793502]
Aug 5 15:05:17 ubuntu kernel: [ 139.793507] Pid: 1163, comm: modprobe Not
tainted 3.2.0-51-generic #77-Ubuntu
Aug 5 15:05:17 ubuntu kernel: [ 139.793517] RIP:
e030:[<ffffffff8134b30b>] [<ffffffff8134b30b>]
msix_program_entries+0x7b/0xa0
Aug 5 15:05:17 ubuntu kernel: [ 139.793528] RSP: e02b:ffff880076c77aa8
EFLAGS: 00010286
Aug 5 15:05:17 ubuntu kernel: [ 139.793534] RAX: ffffc9000031800c RBX:
ffff8800771b2ec0 RCX: 00000000fffffffa
Aug 5 15:05:17 ubuntu kernel: [ 139.793542] RDX: 0000000000000001 RSI:
ffff880075550200 RDI: ffff880075550200
Aug 5 15:05:17 ubuntu kernel: [ 139.793549] RBP: ffff880076c77ad8 R08:
ffff880075558000 R09: ffff880078c000f8
Aug 5 15:05:17 ubuntu kernel: [ 139.793556] R10: 0000000000000000 R11:
0000000000000000 R12: 0000000000000000
Aug 5 15:05:17 ubuntu kernel: [ 139.793564] R13: ffffc9000031800c R14:
ffff880074ee3918 R15: ffff880076161800
Aug 5 15:05:17 ubuntu kernel: [ 139.793576] FS: 00007fa3fcb91700(0000)
GS:ffff88007fc00000(0000) knlGS:0000000000000000
Aug 5 15:05:17 ubuntu kernel: [ 139.793584] CS: e033 DS: 0000 ES: 0000
CR0: 000000008005003b
Aug 5 15:05:17 ubuntu kernel: [ 139.793592] CR2: ffff8000007d1ee0 CR3:
0000000076f1a000 CR4: 0000000000002660
Aug 5 15:05:17 ubuntu kernel: [ 139.793600] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Aug 5 15:05:17 ubuntu kernel: [ 139.793608] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Aug 5 15:05:17 ubuntu kernel: [ 139.793616] Process modprobe (pid: 1163,
threadinfo ffff880076c76000, task ffff880003d4ae00)
Aug 5 15:05:17 ubuntu kernel: [ 139.793625] Stack:
Aug 5 15:05:17 ubuntu kernel: [ 139.793629] 0000000000000062
ffff880074ee3000 0000000000000000 0000000000000003
Aug 5 15:05:17 ubuntu kernel: [ 139.793641] ffff880076161800
0000000000000062 ffff880076c77b48 ffffffff8134b501
Aug 5 15:05:17 ubuntu kernel: [ 139.793654] 0000000000000002
ffff880074ee3918 0000000274ee3000 ffffc90000318000
Aug 5 15:05:17 ubuntu kernel: [ 139.793666] Call Trace:
Aug 5 15:05:17 ubuntu kernel: [ 139.793674] [<ffffffff8134b501>]
msix_capability_init+0x1d1/0x260
Aug 5 15:05:17 ubuntu kernel: [ 139.793683] [<ffffffff8134c31a>]
pci_enable_msix.part.11+0x9a/0x100
Aug 5 15:05:17 ubuntu kernel: [ 139.793692] [<ffffffff8134c3c6>]
pci_enable_msix+0x46/0x60
Aug 5 15:05:17 ubuntu kernel: [ 139.793705] [<ffffffffa0051688>]
igb_set_interrupt_capability+0x1b8/0x1d0 [igb]
Aug 5 15:05:17 ubuntu kernel: [ 139.793720] [<ffffffffa0054448>]
igb_init_interrupt_scheme+0x28/0xf0 [igb]
Aug 5 15:05:17 ubuntu kernel: [ 139.793734] [<ffffffffa00634c5>]
igb_sw_init+0x182/0x1d6 [igb]
Aug 5 15:05:17 ubuntu kernel: [ 139.793747] [<ffffffffa00638a8>]
igb_probe+0x38f/0xa21 [igb]
Aug 5 15:05:17 ubuntu kernel: [ 139.793758] [<ffffffff816606ee>] ?
_raw_spin_unlock_irqrestore+0x1e/0x30
Aug 5 15:05:17 ubuntu kernel: [ 139.793767] [<ffffffff816606ee>] ?
_raw_spin_unlock_irqrestore+0x1e/0x30
Aug 5 15:05:17 ubuntu kernel: [ 139.793778] [<ffffffff81337bdc>]
local_pci_probe+0x5c/0xd0
Aug 5 15:05:17 ubuntu kernel: [ 139.793787] [<ffffffff813394d9>]
__pci_device_probe+0xf9/0x100
Aug 5 15:05:17 ubuntu kernel: [ 139.793796] [<ffffffff813103fa>] ?
kobject_get+0x1a/0x30
Aug 5 15:05:17 ubuntu kernel: [ 139.793804] [<ffffffff8133951a>]
pci_device_probe+0x3a/0x60
Aug 5 15:05:17 ubuntu kernel: [ 139.793814] [<ffffffff813f9658>]
really_probe+0x68/0x190
Aug 5 15:05:17 ubuntu kernel: [ 139.793822] [<ffffffff813f98e5>]
driver_probe_device+0x45/0x70
Aug 5 15:05:17 ubuntu kernel: [ 139.793830] [<ffffffff813f99bb>]
__driver_attach+0xab/0xb0
Aug 5 15:05:17 ubuntu kernel: [ 139.793838] [<ffffffff813f9910>] ?
driver_probe_device+0x70/0x70
Aug 5 15:05:17 ubuntu kernel: [ 139.793847] [<ffffffff813f9910>] ?
driver_probe_device+0x70/0x70
Aug 5 15:05:17 ubuntu kernel: [ 139.793856] [<ffffffff813f8744>]
bus_for_each_dev+0x64/0xa0
Aug 5 15:05:17 ubuntu kernel: [ 139.793864] [<ffffffff813f941e>]
driver_attach+0x1e/0x20
Aug 5 15:05:17 ubuntu kernel: [ 139.793872] [<ffffffff813f9070>]
bus_add_driver+0x1a0/0x270
Aug 5 15:05:17 ubuntu kernel: [ 139.793880] [<ffffffffa006f000>] ?
0xffffffffa006efff
Aug 5 15:05:17 ubuntu kernel: [ 139.793888] [<ffffffff813f9f26>]
driver_register+0x76/0x140
Aug 5 15:05:17 ubuntu kernel: [ 139.793898] [<ffffffff81647ccc>] ?
printk+0x51/0x53
Aug 5 15:05:17 ubuntu kernel: [ 139.793906] [<ffffffffa006f000>] ?
0xffffffffa006efff
Aug 5 15:05:17 ubuntu kernel: [ 139.793914] [<ffffffff813391b6>]
__pci_register_driver+0x56/0xd0
Aug 5 15:05:17 ubuntu kernel: [ 139.793927] [<ffffffffa006f05b>]
igb_init_module+0x5b/0x1000 [igb]
Aug 5 15:05:17 ubuntu kernel: [ 139.793938] [<ffffffff81002040>]
do_one_initcall+0x40/0x180
Aug 5 15:05:17 ubuntu kernel: [ 139.793948] [<ffffffff810a9eae>]
sys_init_module+0xbe/0x230
Aug 5 15:05:17 ubuntu kernel: [ 139.793958] [<ffffffff81668d02>]
system_call_fastpath+0x16/0x1b
Aug 5 15:05:17 ubuntu kernel: [ 139.793965] Code: c1 e5 04 41 83 c5 0c e8
34 20 d9 ff 4d 63 ed 4c 03 6b 20 41 8b 55 00 0f b7 43 02 89 53 08 83 ca 01
c1 e0 04 83 c0 0c 48 03 43 20 <89> 10 48 8b 43 10 41 83 c4 01 89 53 08 49
39 c6 48 8d 58 f0 75
Aug 5 15:05:17 ubuntu kernel: [ 139.794041] RIP [<ffffffff8134b30b>]
msix_program_entries+0x7b/0xa0
Aug 5 15:05:17 ubuntu kernel: [ 139.794051] RSP <ffff880076c77aa8>
Aug 5 15:05:17 ubuntu kernel: [ 139.794055] CR2: ffffc9000031800c
Aug 5 15:05:17 ubuntu kernel: [ 139.794061] ---[ end trace
dea4e19be79d2d73 ]---


Ian.Campbell at citrix

Aug 6, 2013, 3:04 AM

Post #2 of 7 (46 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

On Tue, 2013-08-06 at 01:07 +0530, Mark Jones wrote:
> Hi,
>
>
> I have been trying to get NIC PCI Passthrough to work on Xen 4.3.0 and
> have hit a wall. On attaching the NIC the domU network driver (igb)
> crashes.

Is 4.3.0 the only version you have tried or did it used to work on an
older Xen?

> root [at] dom:~# xl -f pci-attach 54 0000:05:00.1,permissive=1

What is the -f for? Are you working around some other issue?

Does it work if you list the device in the config file, i.e. do a cold
plug? (isolating to a hotplug only vs. a general issue is useful)

It might also be interesting to disable MSI for the guest, there's a
guest kernel commandline option I think, not sure what it is though,
maybe pci=nomsi or perhaps just nomsi?

I assume this device works fine if you leave it to dom0? What about if
you boot the domU 3.2 kernel natively on the host? (I think this is
unlikely to be a driver bug, but it is worth ruling it out).

Google doesn't show up any relevant looking results for
msix_program_entries or msix_capability_init. I'm not really sure how
MSIs work with PCI passthrough, it might be worth escalating this to the
-devel list.

> Aug 5 15:05:17 ubuntu kernel: [ 139.781460] Intel(R) Gigabit Ethernet Network Driver - version 3.2.10-k
> Aug 5 15:05:17 ubuntu kernel: [ 139.781465] Copyright (c) 2007-2011 Intel Corporation.
> Aug 5 15:05:17 ubuntu kernel: [ 139.781534] igb 0000:00:00.1: enabling device (0000 -> 0002)
> Aug 5 15:05:17 ubuntu kernel: [ 139.781688] igb 0000:00:00.1: Xen PCI mapped GSI42 to IRQ27
> Aug 5 15:05:17 ubuntu kernel: [ 139.781816] igb 0000:00:00.1: setting latency timer to 64
> Aug 5 15:05:17 ubuntu kernel: [ 139.793427] BUG: unable to handle kernel paging request at ffffc9000031800c
> Aug 5 15:05:17 ubuntu kernel: [ 139.793442] IP: [<ffffffff8134b30b>] msix_program_entries+0x7b/0xa0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793457] PGD 77e1c067 PUD 77e1d067 PMD 77f23067 PTE 80100000fa3dc465
> Aug 5 15:05:17 ubuntu kernel: [ 139.793471] Oops: 0003 [#1] SMP
> Aug 5 15:05:17 ubuntu kernel: [ 139.793479] CPU 0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793483] Modules linked in: igb(+) dca xen_gntdev xen_evtchn xen_pcifront xenfs ext2 lp parport
> Aug 5 15:05:17 ubuntu kernel: [ 139.793502]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793507] Pid: 1163, comm: modprobe Not tainted 3.2.0-51-generic #77-Ubuntu
> Aug 5 15:05:17 ubuntu kernel: [ 139.793517] RIP: e030:[<ffffffff8134b30b>] [<ffffffff8134b30b>] msix_program_entries+0x7b/0xa0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793528] RSP: e02b:ffff880076c77aa8 EFLAGS: 00010286
> Aug 5 15:05:17 ubuntu kernel: [ 139.793534] RAX: ffffc9000031800c RBX: ffff8800771b2ec0 RCX: 00000000fffffffa
> Aug 5 15:05:17 ubuntu kernel: [ 139.793542] RDX: 0000000000000001 RSI: ffff880075550200 RDI: ffff880075550200
> Aug 5 15:05:17 ubuntu kernel: [ 139.793549] RBP: ffff880076c77ad8 R08: ffff880075558000 R09: ffff880078c000f8
> Aug 5 15:05:17 ubuntu kernel: [ 139.793556] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> Aug 5 15:05:17 ubuntu kernel: [ 139.793564] R13: ffffc9000031800c R14: ffff880074ee3918 R15: ffff880076161800
> Aug 5 15:05:17 ubuntu kernel: [ 139.793576] FS: 00007fa3fcb91700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
> Aug 5 15:05:17 ubuntu kernel: [ 139.793584] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> Aug 5 15:05:17 ubuntu kernel: [ 139.793592] CR2: ffff8000007d1ee0 CR3: 0000000076f1a000 CR4: 0000000000002660
> Aug 5 15:05:17 ubuntu kernel: [ 139.793600] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Aug 5 15:05:17 ubuntu kernel: [ 139.793608] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Aug 5 15:05:17 ubuntu kernel: [ 139.793616] Process modprobe (pid: 1163, threadinfo ffff880076c76000, task ffff880003d4ae00)
> Aug 5 15:05:17 ubuntu kernel: [ 139.793625] Stack:
> Aug 5 15:05:17 ubuntu kernel: [ 139.793629] 0000000000000062 ffff880074ee3000 0000000000000000 0000000000000003
> Aug 5 15:05:17 ubuntu kernel: [ 139.793641] ffff880076161800 0000000000000062 ffff880076c77b48 ffffffff8134b501
> Aug 5 15:05:17 ubuntu kernel: [ 139.793654] 0000000000000002 ffff880074ee3918 0000000274ee3000 ffffc90000318000
> Aug 5 15:05:17 ubuntu kernel: [ 139.793666] Call Trace:
> Aug 5 15:05:17 ubuntu kernel: [ 139.793674] [<ffffffff8134b501>] msix_capability_init+0x1d1/0x260
> Aug 5 15:05:17 ubuntu kernel: [ 139.793683] [<ffffffff8134c31a>] pci_enable_msix.part.11+0x9a/0x100
> Aug 5 15:05:17 ubuntu kernel: [ 139.793692] [<ffffffff8134c3c6>] pci_enable_msix+0x46/0x60
> Aug 5 15:05:17 ubuntu kernel: [ 139.793705] [<ffffffffa0051688>] igb_set_interrupt_capability+0x1b8/0x1d0 [igb]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793720] [<ffffffffa0054448>] igb_init_interrupt_scheme+0x28/0xf0 [igb]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793734] [<ffffffffa00634c5>] igb_sw_init+0x182/0x1d6 [igb]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793747] [<ffffffffa00638a8>] igb_probe+0x38f/0xa21 [igb]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793758] [<ffffffff816606ee>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
> Aug 5 15:05:17 ubuntu kernel: [ 139.793767] [<ffffffff816606ee>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
> Aug 5 15:05:17 ubuntu kernel: [ 139.793778] [<ffffffff81337bdc>] local_pci_probe+0x5c/0xd0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793787] [<ffffffff813394d9>] __pci_device_probe+0xf9/0x100
> Aug 5 15:05:17 ubuntu kernel: [ 139.793796] [<ffffffff813103fa>] ? kobject_get+0x1a/0x30
> Aug 5 15:05:17 ubuntu kernel: [ 139.793804] [<ffffffff8133951a>] pci_device_probe+0x3a/0x60
> Aug 5 15:05:17 ubuntu kernel: [ 139.793814] [<ffffffff813f9658>] really_probe+0x68/0x190
> Aug 5 15:05:17 ubuntu kernel: [ 139.793822] [<ffffffff813f98e5>] driver_probe_device+0x45/0x70
> Aug 5 15:05:17 ubuntu kernel: [ 139.793830] [<ffffffff813f99bb>] __driver_attach+0xab/0xb0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793838] [<ffffffff813f9910>] ? driver_probe_device+0x70/0x70
> Aug 5 15:05:17 ubuntu kernel: [ 139.793847] [<ffffffff813f9910>] ? driver_probe_device+0x70/0x70
> Aug 5 15:05:17 ubuntu kernel: [ 139.793856] [<ffffffff813f8744>] bus_for_each_dev+0x64/0xa0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793864] [<ffffffff813f941e>] driver_attach+0x1e/0x20
> Aug 5 15:05:17 ubuntu kernel: [ 139.793872] [<ffffffff813f9070>] bus_add_driver+0x1a0/0x270
> Aug 5 15:05:17 ubuntu kernel: [ 139.793880] [<ffffffffa006f000>] ? 0xffffffffa006efff
> Aug 5 15:05:17 ubuntu kernel: [ 139.793888] [<ffffffff813f9f26>] driver_register+0x76/0x140
> Aug 5 15:05:17 ubuntu kernel: [ 139.793898] [<ffffffff81647ccc>] ? printk+0x51/0x53
> Aug 5 15:05:17 ubuntu kernel: [ 139.793906] [<ffffffffa006f000>] ? 0xffffffffa006efff
> Aug 5 15:05:17 ubuntu kernel: [ 139.793914] [<ffffffff813391b6>] __pci_register_driver+0x56/0xd0
> Aug 5 15:05:17 ubuntu kernel: [ 139.793927] [<ffffffffa006f05b>] igb_init_module+0x5b/0x1000 [igb]
> Aug 5 15:05:17 ubuntu kernel: [ 139.793938] [<ffffffff81002040>] do_one_initcall+0x40/0x180
> Aug 5 15:05:17 ubuntu kernel: [ 139.793948] [<ffffffff810a9eae>] sys_init_module+0xbe/0x230
> Aug 5 15:05:17 ubuntu kernel: [ 139.793958] [<ffffffff81668d02>] system_call_fastpath+0x16/0x1b
> Aug 5 15:05:17 ubuntu kernel: [ 139.793965] Code: c1 e5 04 41 83 c5 0c e8 34 20 d9 ff 4d 63 ed 4c 03 6b 20 41 8b 55 00 0f b7 43 02 89 53 08 83 ca 01 c1 e0 04 83 c0 0c 48 03 43 20 <89> 10 48 8b 43 10 41 83 c4 01 89 53 08 49 39 c6 48 8d 58 f0 75
> Aug 5 15:05:17 ubuntu kernel: [ 139.794041] RIP [<ffffffff8134b30b>] msix_program_entries+0x7b/0xa0
> Aug 5 15:05:17 ubuntu kernel: [ 139.794051] RSP <ffff880076c77aa8>
> Aug 5 15:05:17 ubuntu kernel: [ 139.794055] CR2: ffffc9000031800c
> Aug 5 15:05:17 ubuntu kernel: [ 139.794061] ---[ end trace dea4e19be79d2d73 ]---
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users [at] lists
> http://lists.xen.org/xen-users



_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users


jmark2850 at gmail

Aug 6, 2013, 5:43 AM

Post #3 of 7 (46 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

Ian,

> I have been trying to get NIC PCI Passthrough to work on Xen 4.3.0 and
> > have hit a wall. On attaching the NIC the domU network driver (igb)
> > crashes.
>
> Is 4.3.0 the only version you have tried or did it used to work on an
> older Xen?
>
> > root [at] dom:~# xl -f pci-attach 54 0000:05:00.1,permissive=1
>
> What is the -f for? Are you working around some other issue?
>
>
xl complains that xend is running if /etc/init.d/xend exists.


> Does it work if you list the device in the config file, i.e. do a cold
> plug? (isolating to a hotplug only vs. a general issue is useful)
>
> It might also be interesting to disable MSI for the guest, there's a
> guest kernel commandline option I think, not sure what it is though,
> maybe pci=nomsi or perhaps just nomsi?
>


Much thanks, pci=nomsi in the kernel command-line works perfectly.

Mark


Ian.Campbell at citrix

Aug 6, 2013, 6:24 AM

Post #4 of 7 (46 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

On Tue, 2013-08-06 at 18:13 +0530, Mark Jones wrote:

>
> What is the -f for? Are you working around some other issue?
>
>
>
> xl complains that xend is running if /etc/init.d/xend exists.

xl complains that xend is running if xend is running. /etc/init.d/xend
can exist but not be run at startup. What xl actually checks for is some
lockfiles created by xend at start of day.

You can confirm with "ps aux" whether xend is running, if it is not then
there is a false positive in libxl's check (and I'll ask for some other
info if that is the case).

If xend is running then you really should arrange for it not to be (e.g.
"update-rc.d xend disable" I think).

xend and xl do not collaborate very well together, xend really needs to
not be running.

Thanks,
Ian.




_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users


jmark2850 at gmail

Aug 6, 2013, 1:06 PM

Post #5 of 7 (41 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

On Tue, Aug 6, 2013 at 6:54 PM, Ian Campbell <Ian.Campbell [at] citrix>wrote:

> On Tue, 2013-08-06 at 18:13 +0530, Mark Jones wrote:
>
> >
> > What is the -f for? Are you working around some other issue?
> >
> >
> >
> > xl complains that xend is running if /etc/init.d/xend exists.
>
> xl complains that xend is running if xend is running. /etc/init.d/xend
> can exist but not be run at startup. What xl actually checks for is some
> lockfiles created by xend at start of day.
>

Pretty sure xend was not running.
I removed the startup links via update-rc.d and it required a reboot to
stop complaining.
Perhaps xend did start and was shutdown, hence the lockfiles (I did not see
them, rebooted the system since).


You can confirm with "ps aux" whether xend is running, if it is not then
> there is a false positive in libxl's check (and I'll ask for some other
> info if that is the case).
>
>
I did upgrade from Xen 4.1.2 to 4.3.0. I also purged the existing xen
installation before the upgrade so it should not matter.

Thanks,
Mark




> If xend is running then you really should arrange for it not to be (e.g.
> "update-rc.d xend disable" I think).
>
> xend and xl do not collaborate very well together, xend really needs to
> not be running.
>
> Thanks,
> Ian.
>
>
>
>


Ian.Campbell at citrix

Aug 7, 2013, 1:31 AM

Post #6 of 7 (41 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

On Wed, 2013-08-07 at 01:36 +0530, Mark Jones wrote:
>
> On Tue, Aug 6, 2013 at 6:54 PM, Ian Campbell <Ian.Campbell [at] citrix>
> wrote:
> On Tue, 2013-08-06 at 18:13 +0530, Mark Jones wrote:
>
> >
> > What is the -f for? Are you working around some
> other issue?
> >
> >
> >
> > xl complains that xend is running if /etc/init.d/xend
> exists.
>
>
> xl complains that xend is running if xend is
> running. /etc/init.d/xend
> can exist but not be run at startup. What xl actually checks
> for is some
> lockfiles created by xend at start of day.
>
>
> Pretty sure xend was not running.
> I removed the startup links via update-rc.d and it required a reboot
> to stop complaining.
> Perhaps xend did start and was shutdown, hence the lockfiles (I did
> not see them, rebooted the system since).

And now xl is happy without the -f?

Ian.



_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users


jmark2850 at gmail

Aug 7, 2013, 7:51 AM

Post #7 of 7 (41 views)
Permalink
Re: NIC PCI-Passthrough problems (PV domU driver crashes) [In reply to]

On Wed, Aug 7, 2013 at 2:01 PM, Ian Campbell <Ian.Campbell [at] citrix>wrote:

> On Wed, 2013-08-07 at 01:36 +0530, Mark Jones wrote:
> >
> > On Tue, Aug 6, 2013 at 6:54 PM, Ian Campbell <Ian.Campbell [at] citrix>
> > wrote:
> > On Tue, 2013-08-06 at 18:13 +0530, Mark Jones wrote:
> >
> > >
> > > What is the -f for? Are you working around some
> > other issue?
> > >
> > >
> > >
> > > xl complains that xend is running if /etc/init.d/xend
> > exists.
> >
> >
> > xl complains that xend is running if xend is
> > running. /etc/init.d/xend
> > can exist but not be run at startup. What xl actually checks
> > for is some
> > lockfiles created by xend at start of day.
> >
> >
> > Pretty sure xend was not running.
> > I removed the startup links via update-rc.d and it required a reboot
> > to stop complaining.
> > Perhaps xend did start and was shutdown, hence the lockfiles (I did
> > not see them, rebooted the system since).
>
> And now xl is happy without the -f?
>
>
Yup, seems to be chugging along nicely.


Mark

Xen users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.