Mailing List Archive: Xen: Users

PCI Passthrough, Radeon 7950 and Windows 7 64-bit

Index | Next | Previous | View Threaded

astralstorm at gmail

Jun 25, 2012, 6:50 AM

Post #1 of 18 (1076 views)
Permalink

PCI Passthrough, Radeon 7950 and Windows 7 64-bit

Hello,
I've been consistently attempting to make the setup mentioned in the
subject work on the following hardware:

Xeon E3 1275 (v1, Sandy Bridge)
Asus P8B WS (Firmware: 0904, 2009 and 2102)
16 GB RAM
Sapphire Radeon HD 7950 OC (Firmware: 015.013.000.010.000705)
Linux 3.4.1 dom0, x86-64, pvops.
Xen 4.1.2 (+ security patch) and 4.1.3-rc1.
Old qemu-dm setup. (not new mainline qemu)

Windows 7 64-bit (actually debug build), 4 GB allocated memory
Signed Uninvention GPLPV drivers 0.11.0.356 (in use for disk and networking)

I've heard others managed to get this setup running without issues.

What I've been able to get is the passthrough to work exactly once per
start of the HVM.
Second start of the HVM Windows 7 64-bit results in a bluescreen
mentioning a page fault.

I've also attempted to force a PCI bus reset on the card via sysfs and
also a forced detach and rescan - trying to passthrough the card
afterwards results in a hard hang.
Same result if xen-pciback.passthrough=1.

I haven't found a way to force a power state reset yet (d3->d0->d3),
it'd be useful for a test.

Any other ideas how to debug this? I can provide various logs as necessary.

[Sorry about any possible double-post. I've registered to the list.]

--
Radosław Szkodziński

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

astralstorm at gmail

Jun 24, 2012, 4:48 AM

Post #2 of 18 (1043 views)
Permalink

PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Hello,
I've been consistently attempting to make the setup mentioned in the
subject work on the following hardware:

Xeon E3 1275 (v1, Sandy Bridge)
Asus P8B WS (Firmware: 0904 and 2009)
16 GB RAM
Sapphire Radeon HD 7950 OC (Firmware: 015.013.000.010.000705)
Linux 3.4.1 dom0, x86-64, pvops.
Xen 4.1.2 (+ security patch) and 4.1.3-rc1.
Old qemu-dm setup. (not new mainline qemu)

Windows 7 64-bit (actually debug build), 4 GB allocated memory
Signed Uninvention GPLPV drivers 0.11.0.356 (in use for disk and networking)

I've heard others managed to get this setup running without issues.

What I've been able to get is the passthrough to work exactly once per
start of the HVM.
Second start of the HVM Windows 7 64-bit results in a bluescreen
mentioning a page fault.

I've also attempted to force a PCI bus reset on the card via sysfs and
also a forced detach and rescan - trying to passthrough the card
afterwards results in a hard hang.
Same result if xen-pciback is set with passthrough=1.

I haven't found a way to force a power state reset yet (d3->d0->d3),
it'd be useful for a test.

Any other ideas how to debug this? I can provide various logs as necessary.

[Note: I'm not subscribed to the list, please CC.]
--
Radosław Szkodziński

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

astralstorm at gmail

Jun 25, 2012, 8:09 AM

Post #3 of 18 (1052 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

On Mon, Jun 25, 2012 at 4:52 PM, Matthias
<matthias.kannenberg [at] googlemail> wrote:
> Hi,
>
> I got a quite similar setup working with vga passthrough. Can you
> provide some more information?
>
> What kernel are you using? With xen patches?
> Everything xen related compiled from source or any distro packages?

Qubes packages for libs (that's Xen 4.1.2 patched), the other
4.1.3-rc1 hypervisor by myself from Gentoo ebuild.

> Are you using gfx_passthrough=1 ?

No, secondary PCI card.

> qemu-dm or stubdom?

Mentioned in the post, qemu-dm. I was unable to make stubdom work with
the rest of the networking setup yet.

> xl or xm?

Xl. No xm here anymore, as xend is not started.

> Hiding the vga card on boot and passing it to the domu via config in
> pci = [] or doing everything on the fly?

Statically, as in pci = [ '0000:01:00.0', '0000:01:00.1' ].

>
> If this is any help: my setup is running on a debian wheezy with an
> openSuse Kernel (3.4.2) and their xen patches. I'm hiding my vga on
> boot via pciback.hide (no other pciback params)

Here, it's being hidden by the initramfs using sysfs, before modules
being loaded.
I can attach the script that does this - it just adds the slots and
binds the devices to pciback.

> For pagefaults, i find the dmesg and xm dmesg outputs more informative
> than the actual xen logs for most of the time.. anything suspicious
> coming up there?

Nothing obvious, that's the problem. I'll attach a whole collection of
logs later.

Thanks,
--
Radosław Szkodziński

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

matthias.kannenberg at googlemail

Jun 25, 2012, 8:21 AM

Post #4 of 18 (1059 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Maybe you should give xm a try just to see if it does the trick. I
never got vga passthrough working with xl (and from my understanding,
it's a lot mor complicated there with compiling the vga bios into xen
and manual calculating vga adress ranges.. with xm, I'm doing neither
of it).

Also, do you increase the log level for xen? my kernel line is:
multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all guest_loglvl=all

What kernel are you using? If you want i can provide my build commands
for the xen-patched openSuse Kernel..

2012/6/25 Radoslaw Szkodzinski <astralstorm [at] gmail>:
> On Mon, Jun 25, 2012 at 4:52 PM, Matthias
> <matthias.kannenberg [at] googlemail> wrote:
>> Hi,
>>
>> I got a quite similar setup working with vga passthrough. Can you
>> provide some more information?
>>
>> What kernel are you using? With xen patches?
>> Everything xen related compiled from source or any distro packages?
>
> Qubes packages for libs (that's Xen 4.1.2 patched), the other
> 4.1.3-rc1 hypervisor by myself from Gentoo ebuild.
>
>> Are you using gfx_passthrough=1 ?
>
> No, secondary PCI card.
>
>> qemu-dm or stubdom?
>
> Mentioned in the post, qemu-dm. I was unable to make stubdom work with
> the rest of the networking setup yet.
>
>> xl or xm?
>
> Xl. No xm here anymore, as xend is not started.
>
>> Hiding the vga card on boot and passing it to the domu via config in
>> pci = [] or doing everything on the fly?
>
> Statically, as in pci = [ '0000:01:00.0', '0000:01:00.1' ].
>
>>
>> If this is any help: my setup is running on a debian wheezy with an
>> openSuse Kernel (3.4.2) and their xen patches. I'm hiding my vga on
>> boot via pciback.hide (no other pciback params)
>
> Here, it's being hidden by the initramfs using sysfs, before modules
> being loaded.
> I can attach the script that does this - it just adds the slots and
> binds the devices to pciback.
>
>> For pagefaults, i find the dmesg and xm dmesg outputs more informative
>> than the actual xen logs for most of the time.. anything suspicious
>> coming up there?
>
> Nothing obvious, that's the problem. I'll attach a whole collection of
> logs later.
>
> Thanks,
> --
> Rados�aw Szkodzi�ski

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

astralstorm at gmail

Jun 25, 2012, 8:32 AM

Post #5 of 18 (1052 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

On Mon, Jun 25, 2012 at 5:21 PM, Matthias
<matthias.kannenberg [at] googlemail> wrote:
> Maybe you should give xm a try just to see if it does the trick. I
> never got vga passthrough working with xl (and from my understanding,
> it's a lot mor complicated there with compiling the vga bios into xen
> and manual calculating vga adress ranges.. with xm, I'm doing neither
> of it).

Why? I'm not using the VGA Passthrough, The card is set up as a secondary.
That shouldn't need any VGA BIOS. Also, it works fine for the first
boot very well!

The problem occurs on the second start of the same VM.

> Also, do you increase the log level for xen? my kernel line is:
> multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all guest_loglvl=all

Will do. (except dom0_mem)

> What kernel are you using? If you want i can provide my build commands
> for the xen-patched openSuse Kernel..

I don't want to use the XenLinux kernel. PVOps only please.
Unless the Xen kernel is actually Pvops-based, in which case why would
I want to use OpenSUSE one instead of vanilla?

As I've mentioned, vanilla 3.4.1.

--
Radosław Szkodziński

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

jocelyn.falempe at free

Jun 25, 2012, 12:51 PM

Post #6 of 18 (1099 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Hi,

I've got a similar setup :

core i7-3770 + ASRock z77 pro 4M
and a sapphire radeon HD7950

Ubuntu 12.04 as Dom0, and Win8 release preview as HVM guest.

I can start my VM multiple time without issue.
as I have 3 USB controller and 2 SATA controller, I use pci passthrough
to pass USB and SATA devices too
so my razer mouse, and secondary sata drive is "native" on windows too.

here is my script (copied from someone) to reserve the PCI devices for
my HVM :

remove_device () {
BDF=$1
# Unbind a PCI function from its driver as necessary
[ ! -e /sys/bus/pci/devices/$BDF/driver/unbind ] || \
echo -n $BDF > /sys/bus/pci/devices/$BDF/driver/unbind
# Add a new slot to the PCI Backend's list
echo -n $BDF > /sys/bus/pci/drivers/pciback/new_slot
# Now that the backend is watching for the slot, bind to it
echo -n $BDF > /sys/bus/pci/drivers/pciback/bind
}

#USB Controller :
remove_device "0000:00:1a.0"
#SATA Controller :
remove_device "0000:04:00.0"
#Radeon 7950
remove_device "0000:01:00.0"
#Radeon 7950 audio
remove_device "0000:01:00.1"

and here is my .cfg file to launch the VM (xm create win8_lan.cfg) :
cat win8_lan.cfg
kernel="/usr/lib/xen-default/boot/hvmloader"
builder='hvm'
memory = 8096
vcpus=6
name = "win8_new"
vif = [ 'type=ioemu, bridge=br0' ]
disk = ['file:/xen-images/win8_2.img,ioemu:hda,w']
acpi = 1
boot="c"
sdl=0
serial='pty'
vnc=1
pci=[ '00:1a.0', '01:00.0', '01:00.1', '04:00.0' ]

hope it helps

Regards,

Jocelyn

On 06/25/2012 05:32 PM, Radoslaw Szkodzinski wrote:
> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
> <matthias.kannenberg [at] googlemail> wrote:
>> Maybe you should give xm a try just to see if it does the trick. I
>> never got vga passthrough working with xl (and from my understanding,
>> it's a lot mor complicated there with compiling the vga bios into xen
>> and manual calculating vga adress ranges.. with xm, I'm doing neither
>> of it).
> Why? I'm not using the VGA Passthrough, The card is set up as a secondary.
> That shouldn't need any VGA BIOS. Also, it works fine for the first
> boot very well!
>
> The problem occurs on the second start of the same VM.
>
>> Also, do you increase the log level for xen? my kernel line is:
>> multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all guest_loglvl=all
> Will do. (except dom0_mem)
>
>> What kernel are you using? If you want i can provide my build commands
>> for the xen-patched openSuse Kernel..
> I don't want to use the XenLinux kernel. PVOps only please.
> Unless the Xen kernel is actually Pvops-based, in which case why would
> I want to use OpenSUSE one instead of vanilla?
>
> As I've mentioned, vanilla 3.4.1.
>

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

cdelorme at gmail

Jun 25, 2012, 3:47 PM

Post #7 of 18 (1054 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Hello,

I am fairly certain you are experiencing the second-boot degradation,
probably combined with a half-working driver installation.

I will try to explain it but the topic is fairly complex and difficult to
comprehend without a solid understanding of what is happening.

I have attached a flowchart as a visual aid to depict the problem.

When Xen, the physical machine, boots the card is passed and the state
remains "uninitialized". If the card were to be initialized normally,
rebooting the physical machine would reset the state due to a change in
power supplied to the card.

So the question to ask is how does a virtual machine reset physical
hardware? Tobias Geiger wrote up an email on the subject of FLR (Function
Level Reset), which I believe is the virtual machine solution to resetting
device state.

Windows fails to issue an FLR to passed GPU's.

The first pass to Windows works great because it is the first time the card
is "initialized", but Windows does not reset the device when you shutdown
or restart it.

With me so far?

His email chains also suggested a solution, which was to use the safely
remove hardware menu when the system has started back up, this restores the
state of the card (your monitor will go black for a few seconds as the card
resets).

However, now you are stuck at the crossroads because you can't get to the
second restart due to a BlueScreen.

I would be almost certain that you are receiving a bluescreen because the
driver installation was run on a second boot previously, where the card
state had been initialized during a previous boot of the system.

This "first boot" is still under investigation, I believe it initializes if
you pass it to another VM, or if you pass it during the installation
process, but I haven't spent the time to verify this.

Most people will boot into safe mode and remove the drivers via VNC
Console, but if you do this during a second boot without resetting the card
first, you will end up with leftover .NET data that prevents ATI Driver
Installation AND cannot be removed, meaning you would need to reinstall
Windows.

So, my suggestion to you is to reboot the physical system, boot Windows and
remove the drivers. Reboot the physical system a second time to be sure,
and install the drivers again during first boot of Windows.

Then give it another spin.

If this did not make sense, please let me know where I lost you and I will
try to explain it better.

~Casey

On Mon, Jun 25, 2012 at 11:32 AM, Radoslaw Szkodzinski <
astralstorm [at] gmail> wrote:

> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
> <matthias.kannenberg [at] googlemail> wrote:
> > Maybe you should give xm a try just to see if it does the trick. I
> > never got vga passthrough working with xl (and from my understanding,
> > it's a lot mor complicated there with compiling the vga bios into xen
> > and manual calculating vga adress ranges.. with xm, I'm doing neither
> > of it).
>
> Why? I'm not using the VGA Passthrough, The card is set up as a secondary.
> That shouldn't need any VGA BIOS. Also, it works fine for the first
> boot very well!
>
> The problem occurs on the second start of the same VM.
>
> > Also, do you increase the log level for xen? my kernel line is:
> > multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all guest_loglvl=all
>
> Will do. (except dom0_mem)
>
> > What kernel are you using? If you want i can provide my build commands
> > for the xen-patched openSuse Kernel..
>
> I don't want to use the XenLinux kernel. PVOps only please.
> Unless the Xen kernel is actually Pvops-based, in which case why would
> I want to use OpenSUSE one instead of vanilla?
>
> As I've mentioned, vanilla 3.4.1.
>
> --
> Radosław Szkodziński
>
> _______________________________________________
> Xen-users mailing list
> Xen-users [at] lists
> http://lists.xen.org/xen-users
>

Attachments:

Diagrams.pdf (110 KB)

matthias.kannenberg at googlemail

Jun 25, 2012, 4:29 PM

Post #8 of 18 (1049 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Interesting explanation, but wouldn't that imply that the domU is the
only thing responsible for resetting the graphic card?
I'm asking because I can simply kill my domU via xm destroy and still
be able to boot the domU just fine afterwards (actually, this was an
issue up till 3 or so month ago but then was fixed in one of the
change sets in the testing repo).

i actually remember an IRQ warning after killing the domU,but xen can
recover this, so if i understand your explanation correctly, there
have to be a mechanism within the dom0 to reset pci devices, too..

2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
> Hello,
>
> I am fairly certain you are experiencing the second-boot degradation,
> probably combined with a half-working driver installation.
>
> I will try to explain it but the topic is fairly complex and difficult to
> comprehend without a solid understanding of what is happening.
>
> I have attached a flowchart as a visual aid to depict the problem.
>
>
> When Xen, the physical machine, boots the card is passed and the state
> remains "uninitialized". If the card were to be initialized normally,
> rebooting the physical machine would reset the state due to a change in
> power supplied to the card.
>
> So the question to ask is how does a virtual machine reset physical
> hardware? Tobias Geiger wrote up an email on the subject of FLR (Function
> Level Reset), which I believe is the virtual machine solution to resetting
> device state.
>
> Windows fails to issue an FLR to passed GPU's.
>
>
> The first pass to Windows works great because it is the first time the card
> is "initialized", but Windows does not reset the device when you shutdown or
> restart it.
>
> With me so far?
>
> His email chains also suggested a solution, which was to use the safely
> remove hardware menu when the system has started back up, this restores the
> state of the card (your monitor will go black for a few seconds as the card
> resets).
>
>
> However, now you are stuck at the crossroads because you can't get to the
> second restart due to a BlueScreen.
>
> I would be almost certain that you are receiving a bluescreen because the
> driver installation was run on a second boot previously, where the card
> state had been initialized during a previous boot of the system.
>
> This "first boot" is still under investigation, I believe it initializes if
> you pass it to another VM, or if you pass it during the installation
> process, but I haven't spent the time to verify this.
>
>
> Most people will boot into safe mode and remove the drivers via VNC Console,
> but if you do this during a second boot without resetting the card first,
> you will end up with leftover .NET data that prevents ATI Driver
> Installation AND cannot be removed, meaning you would need to reinstall
> Windows.
>
>
>
> So, my suggestion to you is to reboot the physical system, boot Windows and
> remove the drivers. Reboot the physical system a second time to be sure,
> and install the drivers again during first boot of Windows.
>
> Then give it another spin.
>
> If this did not make sense, please let me know where I lost you and I will
> try to explain it better.
>
> ~Casey
>
> On Mon, Jun 25, 2012 at 11:32 AM, Radoslaw Szkodzinski
> <astralstorm [at] gmail> wrote:
>>
>> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
>> <matthias.kannenberg [at] googlemail> wrote:
>> > Maybe you should give xm a try just to see if it does the trick. I
>> > never got vga passthrough working with xl (and from my understanding,
>> > it's a lot mor complicated there with compiling the vga bios into xen
>> > and manual calculating vga adress ranges.. with xm, I'm doing neither
>> > of it).
>>
>> Why? I'm not using the VGA Passthrough, The card is set up as a secondary.
>> That shouldn't need any VGA BIOS. Also, it works fine for the first
>> boot very well!
>>
>> The problem occurs on the second start of the same VM.
>>
>> > Also, do you increase the log level for xen? my kernel line is:
>> > multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all
>> > guest_loglvl=all
>>
>> Will do. (except dom0_mem)
>>
>> > What kernel are you using? If you want i can provide my build commands
>> > for the xen-patched openSuse Kernel..
>>
>> I don't want to use the XenLinux kernel. PVOps only please.
>> Unless the Xen kernel is actually Pvops-based, in which case why would
>> I want to use OpenSUSE one instead of vanilla?
>>
>> As I've mentioned, vanilla 3.4.1.
>>
>> --
>> Radosław Szkodziński
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users [at] lists
>> http://lists.xen.org/xen-users
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users [at] lists
> http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

cdelorme at gmail

Jun 25, 2012, 5:04 PM

Post #9 of 18 (1046 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

If you are passing the card to xen-pciback then Dom0 has nothing to do with
the device. Short of physically resetting the device via power-cycling the
physical system, then yes that is exactly what I am implying.

Please understand, I only ran tests with Windows, I have yet to setup
passthrough with Linux or Unix, so the results may vary if you are using aa
DomU with a different operating system.

To my understanding Xen does not inform the card that it needs to reset, I
am pretty sure it was remarked in the aforementioned email chain that this
is the responsibility of the DomU. I can only hazard a guess as to why,
either it isn't sending one because Windows isn't exactly advocating
running their system as a Virtual Machine, OR because we are using
secondary passthrough the card is not available at early boot time, and
perhaps that is when Windows is issuing a reset. Again, no facts
supporting either of those, only wild speculation.

I don't think IRQ is relative but I could be wrong. To my understanding
IRQ gives the OS a way to send signals to the card, it doesn't tell the OS
what signals to send to the card, which would mean the OS chooses whether
to send the reset signal.

If the card is passed using xen-pciback then Dom0 never bothers the device.
With late binding this may be an entirely different story. I have yet to
hear of a Windows Dom0, so again not something my tests can yield
conclusions on. However, to my understanding Dom0 is tied to Xen, I
haven't heard of anyone restarting Dom0 without rebooting the physical
machine, and going back to our original logic, resetting the machine
changes the power which physically resets devices attached to the machine
(this would include devices sent to xen-pciback).

Another speculation of mine is that the reason behind the performance drop
is at second initialization the card has been told to serve two owners,
which throws a wrench into its operations.

I am no expert, I am only supplying my attempt at drawing a logical
conclusion from the problem and my tests. I could supply the exact same
flowchart without terms like FLR and device State, and it wouldn't change
as the process and logic remain the same.

Let me know if that helps clear things up any.

On Mon, Jun 25, 2012 at 7:29 PM, Matthias <
matthias.kannenberg [at] googlemail> wrote:

> Interesting explanation, but wouldn't that imply that the domU is the
> only thing responsible for resetting the graphic card?
> I'm asking because I can simply kill my domU via xm destroy and still
> be able to boot the domU just fine afterwards (actually, this was an
> issue up till 3 or so month ago but then was fixed in one of the
> change sets in the testing repo).
>
> i actually remember an IRQ warning after killing the domU,but xen can
> recover this, so if i understand your explanation correctly, there
> have to be a mechanism within the dom0 to reset pci devices, too..
>
> 2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
> > Hello,
> >
> > I am fairly certain you are experiencing the second-boot degradation,
> > probably combined with a half-working driver installation.
> >
> > I will try to explain it but the topic is fairly complex and difficult to
> > comprehend without a solid understanding of what is happening.
> >
> > I have attached a flowchart as a visual aid to depict the problem.
> >
> >
> > When Xen, the physical machine, boots the card is passed and the state
> > remains "uninitialized". If the card were to be initialized normally,
> > rebooting the physical machine would reset the state due to a change in
> > power supplied to the card.
> >
> > So the question to ask is how does a virtual machine reset physical
> > hardware? Tobias Geiger wrote up an email on the subject of FLR
> (Function
> > Level Reset), which I believe is the virtual machine solution to
> resetting
> > device state.
> >
> > Windows fails to issue an FLR to passed GPU's.
> >
> >
> > The first pass to Windows works great because it is the first time the
> card
> > is "initialized", but Windows does not reset the device when you
> shutdown or
> > restart it.
> >
> > With me so far?
> >
> > His email chains also suggested a solution, which was to use the safely
> > remove hardware menu when the system has started back up, this restores
> the
> > state of the card (your monitor will go black for a few seconds as the
> card
> > resets).
> >
> >
> > However, now you are stuck at the crossroads because you can't get to the
> > second restart due to a BlueScreen.
> >
> > I would be almost certain that you are receiving a bluescreen because the
> > driver installation was run on a second boot previously, where the card
> > state had been initialized during a previous boot of the system.
> >
> > This "first boot" is still under investigation, I believe it initializes
> if
> > you pass it to another VM, or if you pass it during the installation
> > process, but I haven't spent the time to verify this.
> >
> >
> > Most people will boot into safe mode and remove the drivers via VNC
> Console,
> > but if you do this during a second boot without resetting the card first,
> > you will end up with leftover .NET data that prevents ATI Driver
> > Installation AND cannot be removed, meaning you would need to reinstall
> > Windows.
> >
> >
> >
> > So, my suggestion to you is to reboot the physical system, boot Windows
> and
> > remove the drivers. Reboot the physical system a second time to be sure,
> > and install the drivers again during first boot of Windows.
> >
> > Then give it another spin.
> >
> > If this did not make sense, please let me know where I lost you and I
> will
> > try to explain it better.
> >
> > ~Casey
> >
> > On Mon, Jun 25, 2012 at 11:32 AM, Radoslaw Szkodzinski
> > <astralstorm [at] gmail> wrote:
> >>
> >> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
> >> <matthias.kannenberg [at] googlemail> wrote:
> >> > Maybe you should give xm a try just to see if it does the trick. I
> >> > never got vga passthrough working with xl (and from my understanding,
> >> > it's a lot mor complicated there with compiling the vga bios into xen
> >> > and manual calculating vga adress ranges.. with xm, I'm doing neither
> >> > of it).
> >>
> >> Why? I'm not using the VGA Passthrough, The card is set up as a
> secondary.
> >> That shouldn't need any VGA BIOS. Also, it works fine for the first
> >> boot very well!
> >>
> >> The problem occurs on the second start of the same VM.
> >>
> >> > Also, do you increase the log level for xen? my kernel line is:
> >> > multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all
> >> > guest_loglvl=all
> >>
> >> Will do. (except dom0_mem)
> >>
> >> > What kernel are you using? If you want i can provide my build commands
> >> > for the xen-patched openSuse Kernel..
> >>
> >> I don't want to use the XenLinux kernel. PVOps only please.
> >> Unless the Xen kernel is actually Pvops-based, in which case why would
> >> I want to use OpenSUSE one instead of vanilla?
> >>
> >> As I've mentioned, vanilla 3.4.1.
> >>
> >> --
> >> Rados�aw Szkodzi�ski
> >>
> >> _______________________________________________
> >> Xen-users mailing list
> >> Xen-users [at] lists
> >> http://lists.xen.org/xen-users
> >
> >
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users [at] lists
> > http://lists.xen.org/xen-users
>

matthias.kannenberg at googlemail

Jun 25, 2012, 5:21 PM

Post #10 of 18 (1044 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Hi,

sorry if i was unclear: I will test this tomorow, but i'm pretty sure
i can kill my windows 7 domU (has vga, usb controller and onboard
sound via pciback) via xm and afterwards booting it just fine again
with everything working like normal (also possible with linux domUs)..

It might be right that the dom0 can't actually use the hidden pci
devices, but I think that at least the xen itself has some abilities
to control it (hence for example listing the assignable devices via xm
in my opinion indicates an awareness).

But again, these are just my two cents on your theory.. i will test
this tomorrow with my setup..

2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
>
> If you are passing the card to xen-pciback then Dom0 has nothing to do with
> the device. �Short of physically resetting the device via power-cycling the
> physical system, then yes that is exactly what I am implying.
>
> Please understand, I only ran tests with Windows, I have yet to setup
> passthrough with Linux or Unix, so the results may vary if you are using aa
> DomU with a different operating system.
>
>
> To my understanding Xen does not inform the card that it needs to reset, I
> am pretty sure it was remarked in the aforementioned email chain that this
> is the responsibility of the DomU. �I can only hazard a guess as to why,
> either it isn't sending one because Windows isn't exactly advocating running
> their system as a Virtual Machine, OR because we are using secondary
> passthrough the card is not available at early boot time, and perhaps that
> is when Windows is issuing a reset. �Again, no facts supporting either of
> those, only wild speculation.
>
>
> I don't think IRQ is relative but I could be wrong. �To my understanding IRQ
> gives the OS a way to send signals to the card, it doesn't tell the OS what
> signals to send to the card, which would mean the OS chooses whether to send
> the reset signal.
>
> If the card is passed using xen-pciback then Dom0 never bothers the device.
> �With late binding this may be an entirely different story. �I have yet to
> hear of a Windows Dom0, so again not something my tests can yield
> conclusions on. �However, to my understanding Dom0 is tied to Xen, I haven't
> heard of anyone restarting Dom0 without rebooting the physical machine, and
> going back to our original logic, resetting the machine changes the power
> which physically resets devices attached to the machine (this would include
> devices sent to xen-pciback).
>
>
> Another speculation of mine is that the reason behind the performance drop
> is at second initialization the card has been told to serve two owners,
> which throws a wrench into its operations.
>
>
> I am no expert, I am only supplying my attempt at drawing a logical
> conclusion from the problem and my tests. �I could supply the exact same
> flowchart without terms like FLR and device State, and it wouldn't change as
> the process and logic remain the same.
>
> Let me know if that helps clear things up any.
>
>
> On Mon, Jun 25, 2012 at 7:29 PM, Matthias
> <matthias.kannenberg [at] googlemail> wrote:
>>
>> Interesting explanation, but wouldn't that imply that the domU is the
>> only thing responsible for resetting the graphic card?
>> I'm asking because I can simply kill my domU via xm destroy and still
>> be able to boot the domU just fine afterwards (actually, this was an
>> issue up till 3 or so month ago but then was fixed in one of the
>> change sets in the testing repo).
>>
>> i actually remember an IRQ warning after killing the domU,but xen can
>> recover this, so if i understand your explanation correctly, there
>> have to be a mechanism within the dom0 to reset pci devices, too..
>>
>> 2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
>> > Hello,
>> >
>> > I am fairly certain you are experiencing the second-boot degradation,
>> > probably combined with a half-working driver installation.
>> >
>> > I will try to explain it but the topic is fairly complex and difficult
>> > to
>> > comprehend without a solid understanding of what is happening.
>> >
>> > I have attached a flowchart as a visual aid to depict the problem.
>> >
>> >
>> > When Xen, the physical machine, boots the card is passed and the state
>> > remains "uninitialized". �If the card were to be initialized normally,
>> > rebooting the physical machine would reset the state due to a change in
>> > power supplied to the card.
>> >
>> > So the question to ask is how does a virtual machine reset physical
>> > hardware? �Tobias Geiger wrote up an email on the subject of FLR
>> > (Function
>> > Level Reset), which I believe is the virtual machine solution to
>> > resetting
>> > device state.
>> >
>> > Windows fails to issue an FLR to passed GPU's.
>> >
>> >
>> > The first pass to Windows works great because it is the first time the
>> > card
>> > is "initialized", but Windows does not reset the device when you
>> > shutdown or
>> > restart it.
>> >
>> > With me so far?
>> >
>> > His email chains also suggested a solution, which was to use the safely
>> > remove hardware menu when the system has started back up, this restores
>> > the
>> > state of the card (your monitor will go black for a few seconds as the
>> > card
>> > resets).
>> >
>> >
>> > However, now you are stuck at the crossroads because you can't get to
>> > the
>> > second restart due to a BlueScreen.
>> >
>> > I would be almost certain that you are receiving a bluescreen because
>> > the
>> > driver installation was run on a second boot previously, where the card
>> > state had been initialized during a previous boot of the system.
>> >
>> > This "first boot" is still under investigation, I believe it initializes
>> > if
>> > you pass it to another VM, or if you pass it during the installation
>> > process, but I haven't spent the time to verify this.
>> >
>> >
>> > Most people will boot into safe mode and remove the drivers via VNC
>> > Console,
>> > but if you do this during a second boot without resetting the card
>> > first,
>> > you will end up with leftover .NET data that prevents ATI Driver
>> > Installation AND cannot be removed, meaning you would need to reinstall
>> > Windows.
>> >
>> >
>> >
>> > So, my suggestion to you is to reboot the physical system, boot Windows
>> > and
>> > remove the drivers. �Reboot the physical system a second time to be
>> > sure,
>> > and install the drivers again during first boot of Windows.
>> >
>> > Then give it another spin.
>> >
>> > If this did not make sense, please let me know where I lost you and I
>> > will
>> > try to explain it better.
>> >
>> > ~Casey
>> >
>> > On Mon, Jun 25, 2012 at 11:32 AM, Radoslaw Szkodzinski
>> > <astralstorm [at] gmail> wrote:
>> >>
>> >> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
>> >> <matthias.kannenberg [at] googlemail> wrote:
>> >> > Maybe you should give xm a try just to see if it does the trick. I
>> >> > never got vga passthrough working with xl (and from my understanding,
>> >> > it's a lot mor complicated there with compiling the vga bios into xen
>> >> > and manual calculating vga adress ranges.. with xm, I'm doing neither
>> >> > of it).
>> >>
>> >> Why? I'm not using the VGA Passthrough, The card is set up as a
>> >> secondary.
>> >> That shouldn't need any VGA BIOS. Also, it works fine for the first
>> >> boot very well!
>> >>
>> >> The problem occurs on the second start of the same VM.
>> >>
>> >> > Also, do you increase the log level for xen? my kernel line is:
>> >> > multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all
>> >> > guest_loglvl=all
>> >>
>> >> Will do. (except dom0_mem)
>> >>
>> >> > What kernel are you using? If you want i can provide my build
>> >> > commands
>> >> > for the xen-patched openSuse Kernel..
>> >>
>> >> I don't want to use the XenLinux kernel. PVOps only please.
>> >> Unless the Xen kernel is actually Pvops-based, in which case why would
>> >> I want to use OpenSUSE one instead of vanilla?
>> >>
>> >> As I've mentioned, vanilla 3.4.1.
>> >>
>> >> --
>> >> Rados�aw Szkodzi�ski
>> >>
>> >> _______________________________________________
>> >> Xen-users mailing list
>> >> Xen-users [at] lists
>> >> http://lists.xen.org/xen-users
>> >
>> >
>> >
>> > _______________________________________________
>> > Xen-users mailing list
>> > Xen-users [at] lists
>> > http://lists.xen.org/xen-users
>
>

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

cdelorme at gmail

Jun 25, 2012, 5:51 PM

Post #11 of 18 (1049 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

I agree with you that Xen has an awareness, but what I read suggested that
the DomU is supposed to be responsible for the reset. In any event, please
do post your results. If you don't have the same performance degradation
and you can help identify where our configurations differ, it could help
fix the problem which would be awesome.

I forgot to mention, my Windows can reboot, and the GUI works fine, it's
when I try to run a 3D application like a modern video game that I run into
problems.

Results vary, The Last Remnant by Square Enix runs extremely slow and
usually crashes after a certain point, Borderlands drops from 60~ FPS at
max settings to under 10~ FPS and a lower max resolution.

On Mon, Jun 25, 2012 at 8:21 PM, Matthias <
matthias.kannenberg [at] googlemail> wrote:

> Hi,
>
> sorry if i was unclear: I will test this tomorow, but i'm pretty sure
> i can kill my windows 7 domU (has vga, usb controller and onboard
> sound via pciback) via xm and afterwards booting it just fine again
> with everything working like normal (also possible with linux domUs)..
>
> It might be right that the dom0 can't actually use the hidden pci
> devices, but I think that at least the xen itself has some abilities
> to control it (hence for example listing the assignable devices via xm
> in my opinion indicates an awareness).
>
> But again, these are just my two cents on your theory.. i will test
> this tomorrow with my setup..
>
>
> 2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
> >
> > If you are passing the card to xen-pciback then Dom0 has nothing to do
> with
> > the device. Short of physically resetting the device via power-cycling
> the
> > physical system, then yes that is exactly what I am implying.
> >
> > Please understand, I only ran tests with Windows, I have yet to setup
> > passthrough with Linux or Unix, so the results may vary if you are using
> aa
> > DomU with a different operating system.
> >
> >
> > To my understanding Xen does not inform the card that it needs to reset,
> I
> > am pretty sure it was remarked in the aforementioned email chain that
> this
> > is the responsibility of the DomU. I can only hazard a guess as to why,
> > either it isn't sending one because Windows isn't exactly advocating
> running
> > their system as a Virtual Machine, OR because we are using secondary
> > passthrough the card is not available at early boot time, and perhaps
> that
> > is when Windows is issuing a reset. Again, no facts supporting either of
> > those, only wild speculation.
> >
> >
> > I don't think IRQ is relative but I could be wrong. To my understanding
> IRQ
> > gives the OS a way to send signals to the card, it doesn't tell the OS
> what
> > signals to send to the card, which would mean the OS chooses whether to
> send
> > the reset signal.
> >
> > If the card is passed using xen-pciback then Dom0 never bothers the
> device.
> > With late binding this may be an entirely different story. I have yet
> to
> > hear of a Windows Dom0, so again not something my tests can yield
> > conclusions on. However, to my understanding Dom0 is tied to Xen, I
> haven't
> > heard of anyone restarting Dom0 without rebooting the physical machine,
> and
> > going back to our original logic, resetting the machine changes the power
> > which physically resets devices attached to the machine (this would
> include
> > devices sent to xen-pciback).
> >
> >
> > Another speculation of mine is that the reason behind the performance
> drop
> > is at second initialization the card has been told to serve two owners,
> > which throws a wrench into its operations.
> >
> >
> > I am no expert, I am only supplying my attempt at drawing a logical
> > conclusion from the problem and my tests. I could supply the exact same
> > flowchart without terms like FLR and device State, and it wouldn't
> change as
> > the process and logic remain the same.
> >
> > Let me know if that helps clear things up any.
> >
> >
> > On Mon, Jun 25, 2012 at 7:29 PM, Matthias
> > <matthias.kannenberg [at] googlemail> wrote:
> >>
> >> Interesting explanation, but wouldn't that imply that the domU is the
> >> only thing responsible for resetting the graphic card?
> >> I'm asking because I can simply kill my domU via xm destroy and still
> >> be able to boot the domU just fine afterwards (actually, this was an
> >> issue up till 3 or so month ago but then was fixed in one of the
> >> change sets in the testing repo).
> >>
> >> i actually remember an IRQ warning after killing the domU,but xen can
> >> recover this, so if i understand your explanation correctly, there
> >> have to be a mechanism within the dom0 to reset pci devices, too..
> >>
> >> 2012/6/26 Casey DeLorme <cdelorme [at] gmail>:
> >> > Hello,
> >> >
> >> > I am fairly certain you are experiencing the second-boot degradation,
> >> > probably combined with a half-working driver installation.
> >> >
> >> > I will try to explain it but the topic is fairly complex and difficult
> >> > to
> >> > comprehend without a solid understanding of what is happening.
> >> >
> >> > I have attached a flowchart as a visual aid to depict the problem.
> >> >
> >> >
> >> > When Xen, the physical machine, boots the card is passed and the state
> >> > remains "uninitialized". If the card were to be initialized normally,
> >> > rebooting the physical machine would reset the state due to a change
> in
> >> > power supplied to the card.
> >> >
> >> > So the question to ask is how does a virtual machine reset physical
> >> > hardware? Tobias Geiger wrote up an email on the subject of FLR
> >> > (Function
> >> > Level Reset), which I believe is the virtual machine solution to
> >> > resetting
> >> > device state.
> >> >
> >> > Windows fails to issue an FLR to passed GPU's.
> >> >
> >> >
> >> > The first pass to Windows works great because it is the first time the
> >> > card
> >> > is "initialized", but Windows does not reset the device when you
> >> > shutdown or
> >> > restart it.
> >> >
> >> > With me so far?
> >> >
> >> > His email chains also suggested a solution, which was to use the
> safely
> >> > remove hardware menu when the system has started back up, this
> restores
> >> > the
> >> > state of the card (your monitor will go black for a few seconds as the
> >> > card
> >> > resets).
> >> >
> >> >
> >> > However, now you are stuck at the crossroads because you can't get to
> >> > the
> >> > second restart due to a BlueScreen.
> >> >
> >> > I would be almost certain that you are receiving a bluescreen because
> >> > the
> >> > driver installation was run on a second boot previously, where the
> card
> >> > state had been initialized during a previous boot of the system.
> >> >
> >> > This "first boot" is still under investigation, I believe it
> initializes
> >> > if
> >> > you pass it to another VM, or if you pass it during the installation
> >> > process, but I haven't spent the time to verify this.
> >> >
> >> >
> >> > Most people will boot into safe mode and remove the drivers via VNC
> >> > Console,
> >> > but if you do this during a second boot without resetting the card
> >> > first,
> >> > you will end up with leftover .NET data that prevents ATI Driver
> >> > Installation AND cannot be removed, meaning you would need to
> reinstall
> >> > Windows.
> >> >
> >> >
> >> >
> >> > So, my suggestion to you is to reboot the physical system, boot
> Windows
> >> > and
> >> > remove the drivers. Reboot the physical system a second time to be
> >> > sure,
> >> > and install the drivers again during first boot of Windows.
> >> >
> >> > Then give it another spin.
> >> >
> >> > If this did not make sense, please let me know where I lost you and I
> >> > will
> >> > try to explain it better.
> >> >
> >> > ~Casey
> >> >
> >> > On Mon, Jun 25, 2012 at 11:32 AM, Radoslaw Szkodzinski
> >> > <astralstorm [at] gmail> wrote:
> >> >>
> >> >> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
> >> >> <matthias.kannenberg [at] googlemail> wrote:
> >> >> > Maybe you should give xm a try just to see if it does the trick. I
> >> >> > never got vga passthrough working with xl (and from my
> understanding,
> >> >> > it's a lot mor complicated there with compiling the vga bios into
> xen
> >> >> > and manual calculating vga adress ranges.. with xm, I'm doing
> neither
> >> >> > of it).
> >> >>
> >> >> Why? I'm not using the VGA Passthrough, The card is set up as a
> >> >> secondary.
> >> >> That shouldn't need any VGA BIOS. Also, it works fine for the first
> >> >> boot very well!
> >> >>
> >> >> The problem occurs on the second start of the same VM.
> >> >>
> >> >> > Also, do you increase the log level for xen? my kernel line is:
> >> >> > multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all
> >> >> > guest_loglvl=all
> >> >>
> >> >> Will do. (except dom0_mem)
> >> >>
> >> >> > What kernel are you using? If you want i can provide my build
> >> >> > commands
> >> >> > for the xen-patched openSuse Kernel..
> >> >>
> >> >> I don't want to use the XenLinux kernel. PVOps only please.
> >> >> Unless the Xen kernel is actually Pvops-based, in which case why
> would
> >> >> I want to use OpenSUSE one instead of vanilla?
> >> >>
> >> >> As I've mentioned, vanilla 3.4.1.
> >> >>
> >> >> --
> >> >> Radosław Szkodziński
> >> >>
> >> >> _______________________________________________
> >> >> Xen-users mailing list
> >> >> Xen-users [at] lists
> >> >> http://lists.xen.org/xen-users
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Xen-users mailing list
> >> > Xen-users [at] lists
> >> > http://lists.xen.org/xen-users
> >
> >
>

astralstorm at gmail

Jun 25, 2012, 6:13 PM

Post #12 of 18 (1047 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

On Tue, Jun 26, 2012 at 2:51 AM, Casey DeLorme <cdelorme [at] gmail> wrote:
> I agree with you that Xen has an awareness, but what I read suggested that
> the DomU is supposed to be responsible for the reset.

Quite silly, many OSes and drivers don't care about device shutdown on
"poweroff".
Why should they, the power usually will be off real soon anyway.
I think currently only Linux cares enough - due to the need to support kexec.

Suspend to ram is different matter. Perhaps that avenue would be
useful to explore.
(Attempting to S2R the VM instead of shutdown...)

Still, it'd be nice for Xen to force reset the devices (FLR, then
D0->D3->D0 ASPM, then finally PCI bus reset as a last resort) when the
VM stops using the devices.
Better safe than sorry.

> In any event, please
> do post your results. If you don't have the same performance degradation
> and you can help identify where our configurations differ, it could help fix
> the problem which would be awesome.

--
Radosław Szkodziński

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

matthias.kannenberg at googlemail

Jun 26, 2012, 4:19 AM

Post #13 of 18 (1038 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Hi,

did some testing in my lunch break:

# dmesg -c && xm dmesg -c
# xm destroy WINDOMU
# dmesg
[257342.161103] xen1: port 1(work) entered disabled state
[257342.162777] xen1: port 1(work) entered disabled state
[257345.003372] irq 18: nobody cared (try booting with the "irqpoll" option)
[257345.004847] Pid: 11048, comm: qemu-dm Not tainted 3.4.2-xen #2
[257345.006208] Call Trace:
[257345.006208] [<ffffffff800085d5>] dump_trace+0x85/0x1c0
[257345.006208] [<ffffffff8088c036>] dump_stack+0x69/0x6f
[257345.006208] [<ffffffff800c9936>] __report_bad_irq+0x36/0xe0
[257345.006208] [<ffffffff800c9c8b>] note_interrupt+0x1fb/0x240
[257345.006208] [<ffffffff800c72c4>] handle_irq_event_percpu+0x94/0x1d0
[257345.006208] [<ffffffff800c7464>] handle_irq_event+0x64/0x90
[257345.006208] [<ffffffff800ca7b4>] handle_fasteoi_irq+0x64/0x120
[257345.006208] [<ffffffff80008468>] handle_irq+0x18/0x30
[257345.006208] [<ffffffff805c1cc4>] evtchn_do_upcall+0x1c4/0x2e0
[257345.006208] [<ffffffff808a308e>] do_hypervisor_callback+0x1e/0x30
[257345.006208] [<ffffffff8014802a>] fget_light+0x3a/0xc0
[257345.006208] [<ffffffff80159c65>] do_select+0x315/0x660
[257345.006208] [<ffffffff8015a15a>] core_sys_select+0x1aa/0x2e0
[257345.006208] [<ffffffff8015a347>] sys_select+0xb7/0x110
[257345.006208] [<ffffffff808a29bb>] system_call_fastpath+0x1a/0x1f
[257345.006208] [<00007f1a6d0081d3>] 0x7f1a6d0081d2
[257345.006208] handlers:
[257345.006208] [<ffffffff80658a80>] usb_hcd_irq
[257345.006208] [<ffffffff80658a80>] usb_hcd_irq
[257345.006208] [<ffffffff80658a80>] usb_hcd_irq
[257345.006208] Disabling IRQ #18
# xm dmesg
<nothing here>
# xm create WINDOMU
booted without problems and i played some Diablo 3 for testing
purpose: no performance change, everything normal

Then i analysed the kernel output in dmesg from the kill and found out
that the IRQ 18 is not in fact my graphic card what i always thought,
but my usb host controler which i have forwarded to the domU, too. So
appereantly there is no nothing vga related in the dmesg. Then I
thought okay, if the dom0 in fact doesn't care about the vga state, it
might actually be the domU and the only thing i guess can be
responsible for that is this in my domU config:

#######################
# PCI Power Management:
#
# If it's set, the guest OS will be able to program D0-D3hot states of the
# PCI device for the purpose of low power consumption.
#
pci_power_mgmt=1
#######################

My new theory is that this allowes the domU (and therefor my windows)
to reset the vga on boot so i don't have the problems you have..

Question is: Are you using this option in your domU or might that be
the difference in our configs?

2012/6/26 Radoslaw Szkodzinski <astralstorm [at] gmail>:
> On Tue, Jun 26, 2012 at 2:51 AM, Casey DeLorme <cdelorme [at] gmail> wrote:
>> I agree with you that Xen has an awareness, but what I read suggested that
>> the DomU is supposed to be responsible for the reset.
>
> Quite silly, many OSes and drivers don't care about device shutdown on
> "poweroff".
> Why should they, the power usually will be off real soon anyway.
> I think currently only Linux cares enough - due to the need to support kexec.
>
> Suspend to ram is different matter. Perhaps that avenue would be
> useful to explore.
> (Attempting to S2R the VM instead of shutdown...)
>
> Still, it'd be nice for Xen to force reset the devices (FLR, then
> D0->D3->D0 ASPM, then finally PCI bus reset as a last resort) when the
> VM stops using the devices.
> Better safe than sorry.
>
>> In any event, please
>> do post your results. �If you don't have the same performance degradation
>> and you can help identify where our configurations differ, it could help fix
>> the problem which would be awesome.
>
> --
> Rados�aw Szkodzi�ski

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

astralstorm at gmail

Jun 26, 2012, 7:37 AM

Post #14 of 18 (1040 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Xm seems to work fine. I remember it has its own PCI reset code when
destroying the VM, which calls the FLR first. Or none at all.
Xl on the other hand forces a PCI bus reset, causing the hard hang.

The PAGE_FAULT only happens actually with the older kernel. It seems
that xl uses dom0 kernel to do a PCI reset.
pci_power_mgmt option does nothing particular.

However, I think my VM image got damaged from multiple hard crashes
and I'll have to reinstall.
I'm getting really weird issues with it now, such as bluescreens in xenvbd.sys.

Will check it again after a reinstall.

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

cdelorme at gmail

Jun 26, 2012, 3:24 PM

Post #15 of 18 (1030 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

I am using the xl toolstack, with Xen 4.2. That's probably the difference,
I tried the pci flags (both inline and stand-alone versions of them)
without any changes.

On Tue, Jun 26, 2012 at 7:19 AM, Matthias <
matthias.kannenberg [at] googlemail> wrote:

> Hi,
>
> did some testing in my lunch break:
>
> # dmesg -c && xm dmesg -c
> # xm destroy WINDOMU
> # dmesg
> [257342.161103] xen1: port 1(work) entered disabled state
> [257342.162777] xen1: port 1(work) entered disabled state
> [257345.003372] irq 18: nobody cared (try booting with the "irqpoll"
> option)
> [257345.004847] Pid: 11048, comm: qemu-dm Not tainted 3.4.2-xen #2
> [257345.006208] Call Trace:
> [257345.006208] [<ffffffff800085d5>] dump_trace+0x85/0x1c0
> [257345.006208] [<ffffffff8088c036>] dump_stack+0x69/0x6f
> [257345.006208] [<ffffffff800c9936>] __report_bad_irq+0x36/0xe0
> [257345.006208] [<ffffffff800c9c8b>] note_interrupt+0x1fb/0x240
> [257345.006208] [<ffffffff800c72c4>] handle_irq_event_percpu+0x94/0x1d0
> [257345.006208] [<ffffffff800c7464>] handle_irq_event+0x64/0x90
> [257345.006208] [<ffffffff800ca7b4>] handle_fasteoi_irq+0x64/0x120
> [257345.006208] [<ffffffff80008468>] handle_irq+0x18/0x30
> [257345.006208] [<ffffffff805c1cc4>] evtchn_do_upcall+0x1c4/0x2e0
> [257345.006208] [<ffffffff808a308e>] do_hypervisor_callback+0x1e/0x30
> [257345.006208] [<ffffffff8014802a>] fget_light+0x3a/0xc0
> [257345.006208] [<ffffffff80159c65>] do_select+0x315/0x660
> [257345.006208] [<ffffffff8015a15a>] core_sys_select+0x1aa/0x2e0
> [257345.006208] [<ffffffff8015a347>] sys_select+0xb7/0x110
> [257345.006208] [<ffffffff808a29bb>] system_call_fastpath+0x1a/0x1f
> [257345.006208] [<00007f1a6d0081d3>] 0x7f1a6d0081d2
> [257345.006208] handlers:
> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> [257345.006208] Disabling IRQ #18
> # xm dmesg
> <nothing here>
> # xm create WINDOMU
> booted without problems and i played some Diablo 3 for testing
> purpose: no performance change, everything normal
>
> Then i analysed the kernel output in dmesg from the kill and found out
> that the IRQ 18 is not in fact my graphic card what i always thought,
> but my usb host controler which i have forwarded to the domU, too. So
> appereantly there is no nothing vga related in the dmesg. Then I
> thought okay, if the dom0 in fact doesn't care about the vga state, it
> might actually be the domU and the only thing i guess can be
> responsible for that is this in my domU config:
>
> #######################
> # PCI Power Management:
> #
> # If it's set, the guest OS will be able to program D0-D3hot states of
> the
> # PCI device for the purpose of low power consumption.
> #
> pci_power_mgmt=1
> #######################
>
> My new theory is that this allowes the domU (and therefor my windows)
> to reset the vga on boot so i don't have the problems you have..
>
> Question is: Are you using this option in your domU or might that be
> the difference in our configs?
>
>
>
>
> 2012/6/26 Radoslaw Szkodzinski <astralstorm [at] gmail>:
> > On Tue, Jun 26, 2012 at 2:51 AM, Casey DeLorme <cdelorme [at] gmail>
> wrote:
> >> I agree with you that Xen has an awareness, but what I read suggested
> that
> >> the DomU is supposed to be responsible for the reset.
> >
> > Quite silly, many OSes and drivers don't care about device shutdown on
> > "poweroff".
> > Why should they, the power usually will be off real soon anyway.
> > I think currently only Linux cares enough - due to the need to support
> kexec.
> >
> > Suspend to ram is different matter. Perhaps that avenue would be
> > useful to explore.
> > (Attempting to S2R the VM instead of shutdown...)
> >
> > Still, it'd be nice for Xen to force reset the devices (FLR, then
> > D0->D3->D0 ASPM, then finally PCI bus reset as a last resort) when the
> > VM stops using the devices.
> > Better safe than sorry.
> >
> >> In any event, please
> >> do post your results. If you don't have the same performance
> degradation
> >> and you can help identify where our configurations differ, it could
> help fix
> >> the problem which would be awesome.
> >
> > --
> > Radosław Szkodziński
>

matthias.kannenberg at googlemail

Jun 26, 2012, 3:57 PM

Post #16 of 18 (1031 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

Well, then as i said before: I thing xm is a lot more stable right now
then xl and i don't really see any benefit of using xl right now.

Actually I tried using xl instead of xm last week and but run intu
issues with creating vifs when using nat. Interesting to see that
there are also issues with vga passthrough.

Are there - right now - any benefits of using xl instead of xm? I
thought that it's easier to run stubdoms with xl, but after reading
that stubdoms don't bring you any significant performance increate
when using PVOPS-HVMs, i gave up on trieing to get stubdoms running.
So it would be nice to here if migrating to xl is actually something
to look into in the (near) future.

2012/6/27 Casey DeLorme <cdelorme [at] gmail>:
> I am using the xl toolstack, with Xen 4.2. That's probably the difference,
> I tried the pci flags (both inline and stand-alone versions of them) without
> any changes.
>
> On Tue, Jun 26, 2012 at 7:19 AM, Matthias
> <matthias.kannenberg [at] googlemail> wrote:
>>
>> Hi,
>>
>> did some testing in my lunch break:
>>
>> # dmesg -c && xm dmesg -c
>> # xm destroy WINDOMU
>> # dmesg
>> [257342.161103] xen1: port 1(work) entered disabled state
>> [257342.162777] xen1: port 1(work) entered disabled state
>> [257345.003372] irq 18: nobody cared (try booting with the "irqpoll"
>> option)
>> [257345.004847] Pid: 11048, comm: qemu-dm Not tainted 3.4.2-xen #2
>> [257345.006208] Call Trace:
>> [257345.006208] [<ffffffff800085d5>] dump_trace+0x85/0x1c0
>> [257345.006208] [<ffffffff8088c036>] dump_stack+0x69/0x6f
>> [257345.006208] [<ffffffff800c9936>] __report_bad_irq+0x36/0xe0
>> [257345.006208] [<ffffffff800c9c8b>] note_interrupt+0x1fb/0x240
>> [257345.006208] [<ffffffff800c72c4>] handle_irq_event_percpu+0x94/0x1d0
>> [257345.006208] [<ffffffff800c7464>] handle_irq_event+0x64/0x90
>> [257345.006208] [<ffffffff800ca7b4>] handle_fasteoi_irq+0x64/0x120
>> [257345.006208] [<ffffffff80008468>] handle_irq+0x18/0x30
>> [257345.006208] [<ffffffff805c1cc4>] evtchn_do_upcall+0x1c4/0x2e0
>> [257345.006208] [<ffffffff808a308e>] do_hypervisor_callback+0x1e/0x30
>> [257345.006208] [<ffffffff8014802a>] fget_light+0x3a/0xc0
>> [257345.006208] [<ffffffff80159c65>] do_select+0x315/0x660
>> [257345.006208] [<ffffffff8015a15a>] core_sys_select+0x1aa/0x2e0
>> [257345.006208] [<ffffffff8015a347>] sys_select+0xb7/0x110
>> [257345.006208] [<ffffffff808a29bb>] system_call_fastpath+0x1a/0x1f
>> [257345.006208] [<00007f1a6d0081d3>] 0x7f1a6d0081d2
>> [257345.006208] handlers:
>> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
>> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
>> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
>> [257345.006208] Disabling IRQ #18
>> # xm dmesg
>> <nothing here>
>> # xm create WINDOMU
>> booted without problems and i played some Diablo 3 for testing
>> purpose: no performance change, everything normal
>>
>> Then i analysed the kernel output in dmesg from the kill and found out
>> that the IRQ 18 is not in fact my graphic card what i always thought,
>> but my usb host controler which i have forwarded to the domU, too. So
>> appereantly there is no nothing vga related in the dmesg. Then I
>> thought okay, if the dom0 in fact doesn't care about the vga state, it
>> might actually be the domU and the only thing i guess can be
>> responsible for that is this in my domU config:
>>
>> #######################
>> # PCI Power Management:
>> #
>> # If it's set, the guest OS will be able to program D0-D3hot states of
>> the
>> # PCI device for the purpose of low power consumption.
>> #
>> pci_power_mgmt=1
>> #######################
>>
>> My new theory is that this allowes the domU (and therefor my windows)
>> to reset the vga on boot so i don't have the problems you have..
>>
>> Question is: Are you using this option in your domU or might that be
>> the difference in our configs?
>>
>>
>>
>>
>> 2012/6/26 Radoslaw Szkodzinski <astralstorm [at] gmail>:
>> > On Tue, Jun 26, 2012 at 2:51 AM, Casey DeLorme <cdelorme [at] gmail>
>> > wrote:
>> >> I agree with you that Xen has an awareness, but what I read suggested
>> >> that
>> >> the DomU is supposed to be responsible for the reset.
>> >
>> > Quite silly, many OSes and drivers don't care about device shutdown on
>> > "poweroff".
>> > Why should they, the power usually will be off real soon anyway.
>> > I think currently only Linux cares enough - due to the need to support
>> > kexec.
>> >
>> > Suspend to ram is different matter. Perhaps that avenue would be
>> > useful to explore.
>> > (Attempting to S2R the VM instead of shutdown...)
>> >
>> > Still, it'd be nice for Xen to force reset the devices (FLR, then
>> > D0->D3->D0 ASPM, then finally PCI bus reset as a last resort) when the
>> > VM stops using the devices.
>> > Better safe than sorry.
>> >
>> >> In any event, please
>> >> do post your results. If you don't have the same performance
>> >> degradation
>> >> and you can help identify where our configurations differ, it could
>> >> help fix
>> >> the problem which would be awesome.
>> >
>> > --
>> > Radosław Szkodziński
>
>

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

cdelorme at gmail

Jun 26, 2012, 6:42 PM

Post #17 of 18 (1084 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

I compiled Xen 4.1.2 and 4.1.3 during a variety of GPU tests and saw the
same problems while using the xl toolstack. I never had success with
passthrough using xm, but I read plenty of guides about it. I had no luck
using the packaged versions of Xen for a variety of reasons, and since I
went down the path of compiling from source I didn't bother looking back.
I am working on a UEFI Booted GPT boot disk OCZ Vertex 3 SSD Debian Wheezy
with Xen 4.2, so unstable everything essentially, yet my system has been
rock solid for two months. Just last weekend I upgraded Debian and rebuilt
the latest Xen, a lot of bugfixes to the Xen source since the last compile
(the efi memory problem I encountered was finally fixed).

Xen 4.2 has updates every week, and the xl toolstack is constantly under
improvement (
http://blog.xen.org/index.php/2012/05/22/libxl-event-api-improvements/). I
wouldn't say that xl is stable yet though, when they give us a release date
for Xen 4.2 stable then you may have a better answer.

I can say that the xm toolstack was deprecated as of Xen 4.1, and is no
longer being actively maintained (
http://wiki.xen.org/wiki/Choice_of_Toolstacks). It doesn't compile by
default with Xen 4.2, I don't know if it would succeed if you tried given
the changes to Xen's source either.

I have PVHVM for Windows and Linux, but I have not successfully passed the
card to a Linux DomU yet (my own lack of linux experience and time). It
works fine. I know stubdom is compiled when I build Xen, but I don't know
if it is put to use automatically; I have done no reading on this topic.

The only other topic I can think of is qemu versions. Currently
qemu-traditional is required for pci passthrough, though from testing the
upstream qemu package has a lot of great features I would like to see.

My understanding is that PCI Passthrough is not part of the standard qemu
package, and adding it at this stage would be a lot of work, so they will
probably update the qemu package as or after Xen 4.2 stable is released.
The emulated graphics and disk IO were noticeably better with the upstream
qemu version, so I really hope to see it implemented sooner than later.

My conclusion agrees with yours, provided that unstable qemu-traditional is
the same version used by Xen 4.1.2, the toolstack is one of the most likely
places for these bugs to exist.

On Tue, Jun 26, 2012 at 6:57 PM, Matthias <
matthias.kannenberg [at] googlemail> wrote:

> Well, then as i said before: I thing xm is a lot more stable right now
> then xl and i don't really see any benefit of using xl right now.
>
> Actually I tried using xl instead of xm last week and but run intu
> issues with creating vifs when using nat. Interesting to see that
> there are also issues with vga passthrough.
>
> Are there - right now - any benefits of using xl instead of xm? I
> thought that it's easier to run stubdoms with xl, but after reading
> that stubdoms don't bring you any significant performance increate
> when using PVOPS-HVMs, i gave up on trieing to get stubdoms running.
> So it would be nice to here if migrating to xl is actually something
> to look into in the (near) future.
>
> 2012/6/27 Casey DeLorme <cdelorme [at] gmail>:
> > I am using the xl toolstack, with Xen 4.2. That's probably the
> difference,
> > I tried the pci flags (both inline and stand-alone versions of them)
> without
> > any changes.
> >
> > On Tue, Jun 26, 2012 at 7:19 AM, Matthias
> > <matthias.kannenberg [at] googlemail> wrote:
> >>
> >> Hi,
> >>
> >> did some testing in my lunch break:
> >>
> >> # dmesg -c && xm dmesg -c
> >> # xm destroy WINDOMU
> >> # dmesg
> >> [257342.161103] xen1: port 1(work) entered disabled state
> >> [257342.162777] xen1: port 1(work) entered disabled state
> >> [257345.003372] irq 18: nobody cared (try booting with the "irqpoll"
> >> option)
> >> [257345.004847] Pid: 11048, comm: qemu-dm Not tainted 3.4.2-xen #2
> >> [257345.006208] Call Trace:
> >> [257345.006208] [<ffffffff800085d5>] dump_trace+0x85/0x1c0
> >> [257345.006208] [<ffffffff8088c036>] dump_stack+0x69/0x6f
> >> [257345.006208] [<ffffffff800c9936>] __report_bad_irq+0x36/0xe0
> >> [257345.006208] [<ffffffff800c9c8b>] note_interrupt+0x1fb/0x240
> >> [257345.006208] [<ffffffff800c72c4>] handle_irq_event_percpu+0x94/0x1d0
> >> [257345.006208] [<ffffffff800c7464>] handle_irq_event+0x64/0x90
> >> [257345.006208] [<ffffffff800ca7b4>] handle_fasteoi_irq+0x64/0x120
> >> [257345.006208] [<ffffffff80008468>] handle_irq+0x18/0x30
> >> [257345.006208] [<ffffffff805c1cc4>] evtchn_do_upcall+0x1c4/0x2e0
> >> [257345.006208] [<ffffffff808a308e>] do_hypervisor_callback+0x1e/0x30
> >> [257345.006208] [<ffffffff8014802a>] fget_light+0x3a/0xc0
> >> [257345.006208] [<ffffffff80159c65>] do_select+0x315/0x660
> >> [257345.006208] [<ffffffff8015a15a>] core_sys_select+0x1aa/0x2e0
> >> [257345.006208] [<ffffffff8015a347>] sys_select+0xb7/0x110
> >> [257345.006208] [<ffffffff808a29bb>] system_call_fastpath+0x1a/0x1f
> >> [257345.006208] [<00007f1a6d0081d3>] 0x7f1a6d0081d2
> >> [257345.006208] handlers:
> >> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> >> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> >> [257345.006208] [<ffffffff80658a80>] usb_hcd_irq
> >> [257345.006208] Disabling IRQ #18
> >> # xm dmesg
> >> <nothing here>
> >> # xm create WINDOMU
> >> booted without problems and i played some Diablo 3 for testing
> >> purpose: no performance change, everything normal
> >>
> >> Then i analysed the kernel output in dmesg from the kill and found out
> >> that the IRQ 18 is not in fact my graphic card what i always thought,
> >> but my usb host controler which i have forwarded to the domU, too. So
> >> appereantly there is no nothing vga related in the dmesg. Then I
> >> thought okay, if the dom0 in fact doesn't care about the vga state, it
> >> might actually be the domU and the only thing i guess can be
> >> responsible for that is this in my domU config:
> >>
> >> #######################
> >> # PCI Power Management:
> >> #
> >> # If it's set, the guest OS will be able to program D0-D3hot states of
> >> the
> >> # PCI device for the purpose of low power consumption.
> >> #
> >> pci_power_mgmt=1
> >> #######################
> >>
> >> My new theory is that this allowes the domU (and therefor my windows)
> >> to reset the vga on boot so i don't have the problems you have..
> >>
> >> Question is: Are you using this option in your domU or might that be
> >> the difference in our configs?
> >>
> >>
> >>
> >>
> >> 2012/6/26 Radoslaw Szkodzinski <astralstorm [at] gmail>:
> >> > On Tue, Jun 26, 2012 at 2:51 AM, Casey DeLorme <cdelorme [at] gmail>
> >> > wrote:
> >> >> I agree with you that Xen has an awareness, but what I read suggested
> >> >> that
> >> >> the DomU is supposed to be responsible for the reset.
> >> >
> >> > Quite silly, many OSes and drivers don't care about device shutdown on
> >> > "poweroff".
> >> > Why should they, the power usually will be off real soon anyway.
> >> > I think currently only Linux cares enough - due to the need to support
> >> > kexec.
> >> >
> >> > Suspend to ram is different matter. Perhaps that avenue would be
> >> > useful to explore.
> >> > (Attempting to S2R the VM instead of shutdown...)
> >> >
> >> > Still, it'd be nice for Xen to force reset the devices (FLR, then
> >> > D0->D3->D0 ASPM, then finally PCI bus reset as a last resort) when the
> >> > VM stops using the devices.
> >> > Better safe than sorry.
> >> >
> >> >> In any event, please
> >> >> do post your results. If you don't have the same performance
> >> >> degradation
> >> >> and you can help identify where our configurations differ, it could
> >> >> help fix
> >> >> the problem which would be awesome.
> >> >
> >> > --
> >> > Radosław Szkodziński
> >
> >
>

kavefish at gmail

Jun 26, 2012, 8:08 PM

Post #18 of 18 (1035 views)
Permalink

Re: PCI Passthrough, Radeon 7950 and Windows 7 64-bit [In reply to]

The configuration file below resolved my problems. I'm passing through
gfx device to Win7 domU (as a secondary device). Interestingly I can
reboot the domU without any gfx issues.

I'm also now running a custom kernel with pciback built-in, but I don't
think that's the solution. I'll do more troubleshooting soon to identify
the culprit and share for posterity.

Thanks all for the useful discussion and sustained interest
-Chris

On 6/25/12 3:51 PM, jocelyn falempe wrote:
> Hi,
>
> I've got a similar setup :
>
> core i7-3770 + ASRock z77 pro 4M
> and a sapphire radeon HD7950
>
> Ubuntu 12.04 as Dom0, and Win8 release preview as HVM guest.
>
> I can start my VM multiple time without issue.
> as I have 3 USB controller and 2 SATA controller, I use pci passthrough
> to pass USB and SATA devices too
> so my razer mouse, and secondary sata drive is "native" on windows too.
>
> here is my script (copied from someone) to reserve the PCI devices for
> my HVM :
>
> remove_device () {
> BDF=$1
> # Unbind a PCI function from its driver as necessary
> [ ! -e /sys/bus/pci/devices/$BDF/driver/unbind ] || \
> echo -n $BDF > /sys/bus/pci/devices/$BDF/driver/unbind
> # Add a new slot to the PCI Backend's list
> echo -n $BDF > /sys/bus/pci/drivers/pciback/new_slot
> # Now that the backend is watching for the slot, bind to it
> echo -n $BDF > /sys/bus/pci/drivers/pciback/bind
> }
>
> #USB Controller :
> remove_device "0000:00:1a.0"
> #SATA Controller :
> remove_device "0000:04:00.0"
> #Radeon 7950
> remove_device "0000:01:00.0"
> #Radeon 7950 audio
> remove_device "0000:01:00.1"
>
> and here is my .cfg file to launch the VM (xm create win8_lan.cfg) :
> cat win8_lan.cfg
> kernel="/usr/lib/xen-default/boot/hvmloader"
> builder='hvm'
> memory = 8096
> vcpus=6
> name = "win8_new"
> vif = [ 'type=ioemu, bridge=br0' ]
> disk = ['file:/xen-images/win8_2.img,ioemu:hda,w']
> acpi = 1
> boot="c"
> sdl=0
> serial='pty'
> vnc=1
> pci=[ '00:1a.0', '01:00.0', '01:00.1', '04:00.0' ]
>
> hope it helps
>
> Regards,
>
> Jocelyn
>
>
> On 06/25/2012 05:32 PM, Radoslaw Szkodzinski wrote:
>> On Mon, Jun 25, 2012 at 5:21 PM, Matthias
>> <matthias.kannenberg [at] googlemail> wrote:
>>> Maybe you should give xm a try just to see if it does the trick. I
>>> never got vga passthrough working with xl (and from my understanding,
>>> it's a lot mor complicated there with compiling the vga bios into xen
>>> and manual calculating vga adress ranges.. with xm, I'm doing neither
>>> of it).
>> Why? I'm not using the VGA Passthrough, The card is set up as a
>> secondary.
>> That shouldn't need any VGA BIOS. Also, it works fine for the first
>> boot very well!
>>
>> The problem occurs on the second start of the same VM.
>>
>>> Also, do you increase the log level for xen? my kernel line is:
>>> multiboot /boot/xen.gz dom0_mem=2048M iommu=1 loglvl=all
>>> guest_loglvl=all
>> Will do. (except dom0_mem)
>>
>>> What kernel are you using? If you want i can provide my build commands
>>> for the xen-patched openSuse Kernel..
>> I don't want to use the XenLinux kernel. PVOps only please.
>> Unless the Xen kernel is actually Pvops-based, in which case why would
>> I want to use OpenSUSE one instead of vanilla?
>>
>> As I've mentioned, vanilla 3.4.1.
>>
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users [at] lists
> http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users [at] lists
http://lists.xen.org/xen-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads