Mailing List Archive: Xen: Devel

On Dom0 disaggregation (was: Re: [RFC PATCH 0/18] Xenstore stub domain)

Index | Next | Previous | View Threaded

joanna at invisiblethingslab

Jan 12, 2012, 3:18 AM

Post #1 of 5 (109 views)
Permalink

On Dom0 disaggregation (was: Re: [RFC PATCH 0/18] Xenstore stub domain)

On 01/12/12 11:48, Tim Deegan wrote:
> At 11:33 +0100 on 12 Jan (1326367997), Joanna Rutkowska wrote:
>> Daniel,
>>
>> Can you explain what is the rationale for moving the xenstored into a
>> stubdom? After all, if an attacker is able to compromise the xenstored,
>> there should be many ways now how to compromise other VMs in the system?
>> And it shouldn't matter whether the xenstored is in stubdom or whether
>> in Dom0. E.g. the attacker might redirect the block fronts to us some
>> false block backends, so that the VMs get compromised fs. One could
>> probably think of other attacks as well...?
>
> I think the point is to protect xenstore from dom0, not dom0 from
> xenstore. With stub-xenstore and driver domains, only the domain
> builder and PCIback need to have any privilege, and they can be moved
> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 ,
> http://tjd.phlegethon.org/words/sosp11-xoar.html)
>

In order for this to make sense from security point of view, you would
need to deprivilige Dom0. When considering this task, one should answer
the following questions:

1) Who manages the chipset (MCH)?
2) Who manages the input (keyboard, mouse)
3) Who manages the output (GPU, specifically critical on client systems)

From the security point of view there is no point of isolating the
entities that manage the above between each other, because a compromise
of any of those entities leads to full system compromise (again, #3
applies to client systems).

Now, as you pointed out, we shall probably add another bullet to the
list, which is:

4) the xenstored

(although I'm not 100% if we couldn't somehow deprivilige xenstored).

So, we end up with a conclusion that there is no point separating those
4 functionalists between each other, and so it only make sense to host
them in the same domain. How about we call this domain "Dom0", then? ;)

(Sure, this "new Dom0" doesn't include net, USB, and perhaps even SATA
drivers, but we already have been doing this on Qubes, except for moving
out SATA/blkbackend from Dom0).

joanna.

Attachments:

signature.asc (0.44 KB)

tim at xen

Jan 12, 2012, 4:13 AM

Post #2 of 5 (109 views)
Permalink

Re: On Dom0 disaggregation (was: Re: [RFC PATCH 0/18] Xenstore stub domain) [In reply to]

At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote:
> On 01/12/12 11:48, Tim Deegan wrote:
> > I think the point is to protect xenstore from dom0, not dom0 from
> > xenstore. With stub-xenstore and driver domains, only the domain
> > builder and PCIback need to have any privilege, and they can be moved
> > out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 ,
> > http://tjd.phlegethon.org/words/sosp11-xoar.html)
>
> In order for this to make sense from security point of view, you would
> need to deprivilige Dom0.

Yep. That's what Xoar did (or, if you prefer, it moved all dom0's
components out and then got rid of it).

> When considering this task, one should answer
> the following questions:
>
> 1) Who manages the chipset (MCH)?

A driver domain that contains PCIback. It doesn't need general
privlege (e.g. for scheduler or memory operations).

> 2) Who manages the input (keyboard, mouse)
> 3) Who manages the output (GPU, specifically critical on client systems)
>
> From the security point of view there is no point of isolating the
> entities that manage the above between each other, because a compromise
> of any of those entities leads to full system compromise (again, #3
> applies to client systems).

#2 is really a client-only thing as well. Serial console input for
servers is owned directly by the hypervisor.

> Now, as you pointed out, we shall probably add another bullet to the
> list, which is:
>
> 4) the xenstored
>
> (although I'm not 100% if we couldn't somehow deprivilige xenstored).

Yes we could (and Xoar did). IIRC the only reason it needs privilege is
to map the rings and that can be sorted out by having the builder
pre-load a grant entry.

> So, we end up with a conclusion that there is no point separating those
> 4 functionalists between each other, and so it only make sense to host
> them in the same domain. How about we call this domain "Dom0", then? ;)

So you're saying that the PCIback domain (which owns the chipsets) might
as well host Xenstore since either of them could hose the system.
Maybe so. In Xoar we had two reasons not to do that:
- in some configurations you could shut that domain down after boot
(though tbh doing that leads to poor error handling!); and
- we were doing other hardening of Xenstore that involved breaking it
up into more than one VM.

Cheers,

Tim.

_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xensource.com/xen-devel

joanna at invisiblethingslab

Jan 12, 2012, 5:30 AM

Post #3 of 5 (111 views)
Permalink

Re: On Dom0 disaggregation [In reply to]

On 01/12/12 13:13, Tim Deegan wrote:
> At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote:
>> On 01/12/12 11:48, Tim Deegan wrote:
>>> I think the point is to protect xenstore from dom0, not dom0 from
>>> xenstore. With stub-xenstore and driver domains, only the domain
>>> builder and PCIback need to have any privilege, and they can be moved
>>> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 ,
>>> http://tjd.phlegethon.org/words/sosp11-xoar.html)
>>
>> In order for this to make sense from security point of view, you would
>> need to deprivilige Dom0.
>
> Yep. That's what Xoar did (or, if you prefer, it moved all dom0's
> components out and then got rid of it).
>
>> When considering this task, one should answer
>> the following questions:
>>
>> 1) Who manages the chipset (MCH)?
>
> A driver domain that contains PCIback. It doesn't need general
> privlege (e.g. for scheduler or memory operations).
>
>> 2) Who manages the input (keyboard, mouse)
>> 3) Who manages the output (GPU, specifically critical on client systems)
>>
>> From the security point of view there is no point of isolating the
>> entities that manage the above between each other, because a compromise
>> of any of those entities leads to full system compromise (again, #3
>> applies to client systems).
>
> #2 is really a client-only thing as well. Serial console input for
> servers is owned directly by the hypervisor.
>
>> Now, as you pointed out, we shall probably add another bullet to the
>> list, which is:
>>
>> 4) the xenstored
>>
>> (although I'm not 100% if we couldn't somehow deprivilige xenstored).
>
> Yes we could (and Xoar did). IIRC the only reason it needs privilege is
> to map the rings and that can be sorted out by having the builder
> pre-load a grant entry.
>
>> So, we end up with a conclusion that there is no point separating those
>> 4 functionalists between each other, and so it only make sense to host
>> them in the same domain. How about we call this domain "Dom0", then? ;)
>
> So you're saying that the PCIback domain (which owns the chipsets) might
> as well host Xenstore since either of them could hose the system.
> Maybe so. In Xoar we had two reasons not to do that:
> - in some configurations you could shut that domain down after boot
> (though tbh doing that leads to poor error handling!); and
> - we were doing other hardening of Xenstore that involved breaking it
> up into more than one VM.
>

But the paper says the PCIBack is destroyed after init -- who manages
the chipset (MCH, ICH, platform power management), then?

Also, if I correctly interpret your architecture, then BlkBack VM must
be trusted, as it has access to the disk, and thus can serve compromised
fs/kernel/initramfs images to VMs, and also modify boot partition to
load compromised Xen on the next boot (unless you use something like
Intel TXT for Xen load). So, what's the reason of separating it from
(the trusted) Xenstored VM?

Unless the kernel/initramfs images (at least for PV domains) come from
some trusted source, not from blkbackend? Then they could contain code
to verify integrity of the fs served from blkbackend. Have you
considered this in your architecture?

Also, if this was a client system, where would you put user input/output
handling?

joanna.

Attachments:

signature.asc (0.44 KB)

tim at xen

Jan 12, 2012, 6:21 AM

Post #4 of 5 (108 views)
Permalink

Re: On Dom0 disaggregation [In reply to]

At 14:30 +0100 on 12 Jan (1326378636), Joanna Rutkowska wrote:
> > So you're saying that the PCIback domain (which owns the chipsets) might
> > as well host Xenstore since either of them could hose the system.
> > Maybe so. In Xoar we had two reasons not to do that:
> > - in some configurations you could shut that domain down after boot
> > (though tbh doing that leads to poor error handling!); and
> > - we were doing other hardening of Xenstore that involved breaking it
> > up into more than one VM.
> >
>
> But the paper says the PCIBack is destroyed after init -- who manages
> the chipset (MCH, ICH, platform power management), then?

Nobody. Like I said, it's not ideal. :)

You don't have to destroy PCIBack -- you can keep it around to do those
things. And if you do, it has a _much_ narrower interface to the rest
of the world than dom0 does. It has no network access, and no direct
interface to customer (i.e. non-driver) VMs. A malicious customer VM
must break at least one other system VM before it can attack it.

> Also, if I correctly interpret your architecture, then BlkBack VM must
> be trusted, as it has access to the disk, and thus can serve compromised
> fs/kernel/initramfs images to VMs, and also modify boot partition to
> load compromised Xen on the next boot (unless you use something like
> Intel TXT for Xen load). So, what's the reason of separating it from
> (the trusted) Xenstored VM?

Well, you can have multiple block driver domains, if you have multiple
HBAs, and arrange that two customers' data don't share a blkback.

And there are the usual reasons for driver domains. For example you're
not tied to a particular OS for all your drivers. And device-driver
crashes don't take out xenstore.

Trusted boot is orthogonal to the Xoar work. We didn't go into it in
the paper but it's fairly obvious how it would fit in. And once you've
got a TPM you could also use vTPMs to protect VMs' disk images.

> Also, if this was a client system, where would you put user input/output
> handling?

I'd probably make a console driver VM that owned the GPU and the USB
controller. It might make sense for the toolstack (or at least the
parts that compose the display and demux the inputs) to live in that VM
too. Or maybe it would be better to isolate those drivers; I haven't
given it a lot of thought.

Cheers,

Tim.

_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xensource.com/xen-devel

mihirn at cs

Jan 12, 2012, 6:23 AM

Post #5 of 5 (110 views)
Permalink

Re: On Dom0 disaggregation [In reply to]

On 2012-01-12, at 5:31 AM, Joanna Rutkowska <joanna [at] invisiblethingslab> wrote:

> On 01/12/12 13:13, Tim Deegan wrote:
>> At 12:18 +0100 on 12 Jan (1326370728), Joanna Rutkowska wrote:
>>> On 01/12/12 11:48, Tim Deegan wrote:
>>>> I think the point is to protect xenstore from dom0, not dom0 from
>>>> xenstore. With stub-xenstore and driver domains, only the domain
>>>> builder and PCIback need to have any privilege, and they can be moved
>>>> out of dom0 too (e.g., http://dl.acm.org/citation.cfm?id=1346278 ,
>>>> http://tjd.phlegethon.org/words/sosp11-xoar.html)
>>>
>>> In order for this to make sense from security point of view, you would
>>> need to deprivilige Dom0.
>>
>> Yep. That's what Xoar did (or, if you prefer, it moved all dom0's
>> components out and then got rid of it).
>>
>>> When considering this task, one should answer
>>> the following questions:
>>>
>>> 1) Who manages the chipset (MCH)?
>>
>> A driver domain that contains PCIback. It doesn't need general
>> privlege (e.g. for scheduler or memory operations).
>>
>>> 2) Who manages the input (keyboard, mouse)
>>> 3) Who manages the output (GPU, specifically critical on client systems)
>>>
>>> From the security point of view there is no point of isolating the
>>> entities that manage the above between each other, because a compromise
>>> of any of those entities leads to full system compromise (again, #3
>>> applies to client systems).
>>
>> #2 is really a client-only thing as well. Serial console input for
>> servers is owned directly by the hypervisor.
>>
>>> Now, as you pointed out, we shall probably add another bullet to the
>>> list, which is:
>>>
>>> 4) the xenstored
>>>
>>> (although I'm not 100% if we couldn't somehow deprivilige xenstored).
>>
>> Yes we could (and Xoar did). IIRC the only reason it needs privilege is
>> to map the rings and that can be sorted out by having the builder
>> pre-load a grant entry.
>>
>>> So, we end up with a conclusion that there is no point separating those
>>> 4 functionalists between each other, and so it only make sense to host
>>> them in the same domain. How about we call this domain "Dom0", then? ;)
>>
>> So you're saying that the PCIback domain (which owns the chipsets) might
>> as well host Xenstore since either of them could hose the system.
>> Maybe so. In Xoar we had two reasons not to do that:
>> - in some configurations you could shut that domain down after boot
>> (though tbh doing that leads to poor error handling!); and
>> - we were doing other hardening of Xenstore that involved breaking it
>> up into more than one VM.
>>
>
> But the paper says the PCIBack is destroyed after init -- who manages
> the chipset (MCH, ICH, platform power management), then?
>
> Also, if I correctly interpret your architecture, then BlkBack VM must
> be trusted, as it has access to the disk, and thus can serve compromised
> fs/kernel/initramfs images to VMs, and also modify boot partition to
> load compromised Xen on the next boot (unless you use something like
> Intel TXT for Xen load). So, what's the reason of separating it from
> (the trusted) Xenstored VM?
>
> Unless the kernel/initramfs images (at least for PV domains) come from
> some trusted source, not from blkbackend? Then they could contain code
> to verify integrity of the fs served from blkbackend. Have you
> considered this in your architecture?

Indeed. The domain builder only builds known good kernels, currently which are served from its ramdisk. There is an alternative architecture where it's served by a special storage domain via shared memory too.

You're right about the block backend needing to be trusted to some extent; it's only a matter of reducing who needs to trust it. On a system with multiple disk controllers, the one serving Xen could be split from the one serving the VMs. With only a single controller, you'd probably use TXT.

>
> Also, if this was a client system, where would you put user input/output
> handling?

I'm afraid I'm not sure I know exactly what you mean here...

~M

>
> joanna.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel [at] lists
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel [at] lists
http://lists.xensource.com/xen-devel

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads