Mailing List Archive: DRBD: Users

Question on ext3 and dual primaries mode

Index | Next | Previous | View Threaded

PieterB at gewis

Sep 13, 2009, 10:57 AM

Post #1 of 7 (1497 views)
Permalink

Question on ext3 and dual primaries mode

Hello,

I'm about to create a DRBD cluster with 2 disks per node. I want to use dual
primary mode, in order to be able to write to both nodes concurrently.

If I interpret http://www.drbd.org/users-guide/s-dual-primary-mode.html
correctly dual-primary-mode requires the use of a shared cluster file system
that utilizes a distributed lock manager, such as GFS(2) or OCFS2 filesystem.

I was wondering if using ext3 is also possible when using dual primary mode.
The DRBD-docs seem to state that this is impossible. I was amazed, that
according to the Ubuntu server guide,
https://help.ubuntu.com/9.04/serverguide/C/drbd.html this seems a possible
setup. The example on that page uses "allow-two-primaries" and ext3 as filesystem.

Can you tell me if the Ubuntu page is wrong, or that it's (now) possible to use
ext3 with dual-primary-mode.

Is it also possible and recommended to use a LVM RAID0 as a DRBD source disk?
E.g. LVM2 RAID0 of 2 disks <-> DRBD <-> EXT3 | OCFS2 | GFS2

If using ext3 is not an option, which is preferable? OCFS2 or GFS2?

I'm using a Ubuntu 9.04 server, with the DRBD-nodes as virtual machines based on KVM.

I'm hoping somebody can answer my questions,
Regards,
Pieter

_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user

florian.haas at linbit

Sep 14, 2009, 12:26 AM

Post #2 of 7 (1451 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

Using ext3 on top of dual primary DRBD is a sure fire way of destroying
data. Does that answer your question?

Cheers,
Florian

Attachments:

signature.asc (0.25 KB)

florian.haas at linbit

Sep 14, 2009, 12:37 AM

Post #3 of 7 (1448 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

On 2009-09-13 19:57, PieterB wrote:
> Hello,
>
> I'm about to create a DRBD cluster with 2 disks per node. I want to use dual
> primary mode, in order to be able to write to both nodes concurrently.
>
> If I interpret http://www.drbd.org/users-guide/s-dual-primary-mode.html
> correctly dual-primary-mode requires the use of a shared cluster file system
> that utilizes a distributed lock manager, such as GFS(2) or OCFS2 filesystem.
>
> I was wondering if using ext3 is also possible when using dual primary mode.
> The DRBD-docs seem to state that this is impossible. I was amazed, that
> according to the Ubuntu server guide,
> https://help.ubuntu.com/9.04/serverguide/C/drbd.html this seems a possible
> setup. The example on that page uses "allow-two-primaries" and ext3 as filesystem.

Ante, can you please drop a note to the Ubuntu Server Guide
documentation maintainers that the example of configuring ext3 on top of
dual-Primary DRBD is utter nonsense? AFAICS the example does not
actually mention *mounting* the device on both nodes, so what they are
documenting is actually a single-Primary setup anyway. So they probably
just want to delete the "allow-two-primaries;" line from their example
conf and be fine.

Thanks,
Florian

Attachments:

signature.asc (0.25 KB)

m.watts at eris

Sep 14, 2009, 1:09 AM

Post #4 of 7 (1456 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

On Mon, 2009-09-14 at 09:26 +0200, Florian Haas wrote:
> Using ext3 on top of dual primary DRBD is a sure fire way of destroying
> data. Does that answer your question?

I have seen (and used) allow-two-primaries with ext3, when using the
drbd block device driver for Xen to allow live migration of a DomU
between two nodes where the DomU is given a full dedicated drbd block
device for its backing store.

However, for 99.999% of the time DRBD is running in Primary/Secondary
mode and it is Xen which controls promoting and demoting each side as a
migration. Note that I believe Xen is using this in order to prove to
itself that the remote node is capable of accepting disk-writes and is
thus safe to migrate to; it does not, I believe, allow writes to happen
on both nodes at the same time in the general case.

This is the _only_ case I know of where allow-two-primaries and ext3 can
co-exist, if only for a very short and controlled time. For anything
else Florian is right - kiss goodbye to your data.

Mark.

--
Mark Watts BSc RHCE MBCS
Senior Systems Engineer, Managed Services Manpower
www.QinetiQ.com
QinetiQ - Delivering customer-focused solutions
GPG Key: http://www.linux-corner.info/mwatts.gpg

Attachments:

signature.asc (0.19 KB)

m.watts at eris

Sep 14, 2009, 1:20 AM

Post #5 of 7 (1443 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

On Mon, 2009-09-14 at 09:09 +0100, Mark Watts wrote:
> On Mon, 2009-09-14 at 09:26 +0200, Florian Haas wrote:
> > Using ext3 on top of dual primary DRBD is a sure fire way of destroying
> > data. Does that answer your question?
>
> I have seen (and used) allow-two-primaries with ext3, when using the
> drbd block device driver for Xen to allow live migration of a DomU
> between two nodes where the DomU is given a full dedicated drbd block
> device for its backing store.
>
> However, for 99.999% of the time DRBD is running in Primary/Secondary
> mode and it is Xen which controls promoting and demoting each side as a
> migration. Note that I believe Xen is using this in order to prove to
> itself that the remote node is capable of accepting disk-writes and is
> thus safe to migrate to; it does not, I believe, allow writes to happen
> on both nodes at the same time in the general case.
>
> This is the _only_ case I know of where allow-two-primaries and ext3 can
> co-exist, if only for a very short and controlled time. For anything
> else Florian is right - kiss goodbye to your data.

Looking back on my notes, I'm totally wrong here. DRBD/Xen live
migration does NOT involve ext3. I'm thinking about something entirely
different.

http://www.drbd.org/users-guide/ch-xen.html

ext3 + allow-two-primaries will destroy data.

/me goes for coffee.

--
Mark Watts BSc RHCE MBCS
Senior Systems Engineer, Managed Services Manpower
www.QinetiQ.com
QinetiQ - Delivering customer-focused solutions
GPG Key: http://www.linux-corner.info/mwatts.gpg

Attachments:

signature.asc (0.19 KB)

florian.haas at linbit

Sep 14, 2009, 1:23 AM

Post #6 of 7 (1443 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

On 2009-09-14 10:20, Mark Watts wrote:
> Looking back on my notes, I'm totally wrong here. DRBD/Xen live
> migration does NOT involve ext3. I'm thinking about something entirely
> different.
>
> http://www.drbd.org/users-guide/ch-xen.html
>
> ext3 + allow-two-primaries will destroy data.
>
> /me goes for coffee.

Mark just beat me for the correction. :)

I'm 100% positive Mark is well aware of what follows, but just in case
some is pulling this thread from the list archives:

In the case Mark describes (which is documented in the User's Guide
chapter he mentioned in his follow up), you would not have ext3 directly
on top of your DRBD device. Instead, your DRBD resource would be
configured to act as a Virtual Block Device for Xen, that VBD would most
likely contain a partition table, and you'd have ext3 inside one of the
partitions. Which means that as long as you have Xen behaving correctly
you're fine. And which also means it's actually rather hard to mount the
ext3 filesystem from within your host, as it requires a kpartx detour
which makes it a little harder shooting yourself in the foot.

Having ext3 right inside dual-Primary DRBD, by contrast, makes it really
easy to shoot yourself in the foot, and royally mess up your data. Don't
do that.

Cheers,
Florian

Attachments:

signature.asc (0.25 KB)

PieterB at gewis

Sep 14, 2009, 11:14 AM

Post #7 of 7 (1455 views)
Permalink

Re: Question on ext3 and dual primaries mode [In reply to]

On Mon, Sep 14, 2009 at 09:26:45AM +0200, Florian Haas wrote:
> Using ext3 on top of dual primary DRBD is a sure fire way of destroying
> data. Does that answer your question?
That does answer my question. I think I will leave ext3 out of the viable
options for creating a fault-tolerant storage with dual primary mode...

Is either OCFS2 or GFS2 recommended over the other filesystem? I've used
other Oracle software in the past, and would not generally recommend it for
fault-tolerant and stable systems. I've read through the Oracle OCFS2 and
DRBD OCFS2 documentation, and I was surprised. It seemed better than average
Oracle documentation. Does the state of the documentation reflect the state
of the OCFS2 software?

Any thoughts/preferences for either OCFS2 or GFS2 and more considerations
are more than welcome.

Regards,
Pieter

PS I will create a Ubuntu bug report on the documentation, in order to improve it.
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads