Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Pacemaker

Searching for a viable Debian solution

 

 

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded


paul at gear

Apr 23, 2010, 9:01 PM

Post #1 of 6 (1320 views)
Permalink
Searching for a viable Debian solution

Hi folks,

Over the last several days i've been reading, asking questions (special
thanks to beekhof, tserong, fghass and the other kind folks on #linux-ha
putting up with my questions), and experimenting with my test setup, and
i'm yet to find a viable combination of options for creating a VM
cluster on Debian.

The preferred components of my setup are:
- Debian lenny
- Xen VMs
- OCFS2 on an iSCSI NAS for shared storage
- whatever openais/corosync/pacemaker combination will do the job
- I'd like to have live migration of VMs but it's not essential

The point of my setup is to have a test cluster with which i can play
(and run my own home network) without breaking the ones i have to
support at customers. I work with small businesses, educational
institutions, and non-profits, none of whom have the means to support a
fully working test configuration, or to buy time from big name
consultants who can employ full-time clustering specialists.

My experiments thus far have been with corosync & pacemaker from
http://people.debian.org/~madkiss/ha/, and with libvirt from
lenny-backports.

When i tried the latest versions of corosync & pacemaker in the above
repo, i was able to get the cluster running appropriately, but when i
tried to add the DLM clone resource, i got this error: "controld[3526]:
ERROR: Setup problem: Couldn't find utility dlm_controld.pcmk". So it
seems that OCFS2 is not viable with the current state of the
corosync/openais/pacemaker stack. I've investigated a couple of
alternatives to my preferred list, including:

- cLVM - requires the Red Hat cluster stack and conflicts with the
current corosync/openais/pacemaker versions. This would be my preferred
solution if it were possible to use LVs from the same VG on different
nodes simultaneously. I'm not sure whether this is possible, because
the cLVM documentation is almost non-existent, and i couldn't find
answers to any of my big questions.

- libvirt with iSCSI as the backend for the storage pool. I've tried
this and it looks promising, but it appears non-functional on iSCSI
storage backends at the moment. I get this message when i try to create
a volume: "libvirtd: ... error : storageVolumeCreateXML:1301 : this
function is not supported by the hypervisor: storage pool does not
support volume creation"

I substituted KVM for Xen, and the result was the same.

So, the question remains: what is a viable HA stack for Debian with
virtualization and shared storage? I'm happy to switch technologies
where it's necessary; here are the things i would be willing to try if
necessary:

- Different clustered file storage setup. What are the possibilities?
GFS2? DRBD seems feasible, but it doesn't actually solve anything in
the above equation, so i'd rather keep the iSCSI NAS.

- Switching distros? Possible - my order of preference: Debian squeeze,
Debian sid, Ubuntu lucid, openSUSE 11.2, CentOS 5.4.

- Switching cluster stacks? I've worked with clusters on HP-UX,
NetWare, OES/Linux 1, heartbeat 1, and heartbeat 2, so i'm sure one more
change won't kill me, but the more similar in feel it is to pacemaker or
heartbeat 2, the better. If using non-integrated OCFS2 cluster
membership and an older openais or heartbeat version is viable, i'm
happy to look at that as well, although obviously i'd rather go forward
than backward.

- Switching VM technologies? I guess KVM, VirtualBox, or OpenVZ would
be viable, but again, it doesn't seem to solve much in the current scenario.

What's the path of least resistance here?

Thanks in advance,
Paul


_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


quentin at MIT

Apr 23, 2010, 10:05 PM

Post #2 of 6 (1298 views)
Permalink
Re: Searching for a viable Debian solution [In reply to]

Hi Paul,

Current clvm actually supports multiple locking schemes, including both
the old redhat cluster stack and modern corosync/openais. We use Ubuntu
Hardy with backported corosync and clvm packages, and it works pretty
well. Hand-backporting is not for the faint of heart, though.

--Quentin

On Sat, 24 Apr 2010, Paul Gear wrote:

> Hi folks,
>
> Over the last several days i've been reading, asking questions (special
> thanks to beekhof, tserong, fghass and the other kind folks on #linux-ha
> putting up with my questions), and experimenting with my test setup, and i'm
> yet to find a viable combination of options for creating a VM cluster on
> Debian.
>
> The preferred components of my setup are:
> - Debian lenny
> - Xen VMs
> - OCFS2 on an iSCSI NAS for shared storage
> - whatever openais/corosync/pacemaker combination will do the job
> - I'd like to have live migration of VMs but it's not essential
>
> The point of my setup is to have a test cluster with which i can play (and
> run my own home network) without breaking the ones i have to support at
> customers. I work with small businesses, educational institutions, and
> non-profits, none of whom have the means to support a fully working test
> configuration, or to buy time from big name consultants who can employ
> full-time clustering specialists.
>
> My experiments thus far have been with corosync & pacemaker from
> http://people.debian.org/~madkiss/ha/, and with libvirt from lenny-backports.
>
> When i tried the latest versions of corosync & pacemaker in the above repo, i
> was able to get the cluster running appropriately, but when i tried to add
> the DLM clone resource, i got this error: "controld[3526]: ERROR: Setup
> problem: Couldn't find utility dlm_controld.pcmk". So it seems that OCFS2 is
> not viable with the current state of the corosync/openais/pacemaker stack.
> I've investigated a couple of alternatives to my preferred list, including:
>
> - cLVM - requires the Red Hat cluster stack and conflicts with the current
> corosync/openais/pacemaker versions. This would be my preferred solution if
> it were possible to use LVs from the same VG on different nodes
> simultaneously. I'm not sure whether this is possible, because the cLVM
> documentation is almost non-existent, and i couldn't find answers to any of
> my big questions.
>
> - libvirt with iSCSI as the backend for the storage pool. I've tried this
> and it looks promising, but it appears non-functional on iSCSI storage
> backends at the moment. I get this message when i try to create a volume:
> "libvirtd: ... error : storageVolumeCreateXML:1301 : this function is not
> supported by the hypervisor: storage pool does not support volume creation"
>
> I substituted KVM for Xen, and the result was the same.
>
> So, the question remains: what is a viable HA stack for Debian with
> virtualization and shared storage? I'm happy to switch technologies where
> it's necessary; here are the things i would be willing to try if necessary:
>
> - Different clustered file storage setup. What are the possibilities? GFS2?
> DRBD seems feasible, but it doesn't actually solve anything in the above
> equation, so i'd rather keep the iSCSI NAS.
>
> - Switching distros? Possible - my order of preference: Debian squeeze,
> Debian sid, Ubuntu lucid, openSUSE 11.2, CentOS 5.4.
>
> - Switching cluster stacks? I've worked with clusters on HP-UX, NetWare,
> OES/Linux 1, heartbeat 1, and heartbeat 2, so i'm sure one more change won't
> kill me, but the more similar in feel it is to pacemaker or heartbeat 2, the
> better. If using non-integrated OCFS2 cluster membership and an older
> openais or heartbeat version is viable, i'm happy to look at that as well,
> although obviously i'd rather go forward than backward.
>
> - Switching VM technologies? I guess KVM, VirtualBox, or OpenVZ would be
> viable, but again, it doesn't seem to solve much in the current scenario.
>
> What's the path of least resistance here?
>
> Thanks in advance,
> Paul
>
>
> _______________________________________________
> Pacemaker mailing list: Pacemaker [at] oss
> http://oss.clusterlabs.org/mailman/listinfo/pacemaker
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


paul at gear

Apr 23, 2010, 11:02 PM

Post #3 of 6 (1293 views)
Permalink
Re: Searching for a viable Debian solution [In reply to]

On 24/04/10 15:05, Quentin Smith wrote:
> Hi Paul,
>
> Current clvm actually supports multiple locking schemes, including both
> the old redhat cluster stack and modern corosync/openais. We use Ubuntu
> Hardy with backported corosync and clvm packages, and it works pretty
> well. Hand-backporting is not for the faint of heart, though.
>
> --Quentin
>
> ...
>>
>> - cLVM - requires the Red Hat cluster stack and conflicts with the
>> current corosync/openais/pacemaker versions. This would be my
>> preferred solution if it were possible to use LVs from the same VG on
>> different nodes simultaneously. I'm not sure whether this is possible,
>> because the cLVM documentation is almost non-existent, and i couldn't
>> find answers to any of my big questions.

Thanks for the thoughts Quentin. Would you mind sharing your current
versions (of clvm, corosync, openais, and pacemaker) and VM technology?

I agree about hand-backporting - i'd be more inclined to run unstable
than to hand-backport.

Any ideas on whether LVs in the same VG can be used on different hosts
simultaneously with cLVM?

Paul


_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


quentin at MIT

Apr 23, 2010, 11:30 PM

Post #4 of 6 (1282 views)
Permalink
Re: Searching for a viable Debian solution [In reply to]

On Sat, 24 Apr 2010, Paul Gear wrote:

> On 24/04/10 15:05, Quentin Smith wrote:
>> Hi Paul,
>>
>> Current clvm actually supports multiple locking schemes, including both
>> the old redhat cluster stack and modern corosync/openais. We use Ubuntu
>> Hardy with backported corosync and clvm packages, and it works pretty
>> well. Hand-backporting is not for the faint of heart, though.
>>
>> --Quentin
>
> Thanks for the thoughts Quentin. Would you mind sharing your current
> versions (of clvm, corosync, openais, and pacemaker) and VM technology?

It's not currently in a good state for being deployed elsewhere, but you
can look at http://invirt.mit.edu/. You can also steal our backports out
of our apt repo at http://xvm.mit.edu/invirt/, but they obviously come
with _no warranty_.

>
> I agree about hand-backporting - i'd be more inclined to run unstable than to
> hand-backport.

It looks like semi-recent corosync is in sid - maybe it also has a recent
clvm?

>
> Any ideas on whether LVs in the same VG can be used on different hosts
> simultaneously with cLVM?

Yes, that is the whole premise of cLVM; LVs can be used by any number of
hosts simultaneously. (Obviously, you need to make sure that if you mount
the same LV from multiple places, that you're using a filesystem that can
be mounted simultaneously such as OCFS).

--Quentin

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


quentin at MIT

Apr 23, 2010, 11:30 PM

Post #5 of 6 (1282 views)
Permalink
Re: Searching for a viable Debian solution [In reply to]

On Sat, 24 Apr 2010, Paul Gear wrote:

> On 24/04/10 15:05, Quentin Smith wrote:
>> Hi Paul,
>>
>> Current clvm actually supports multiple locking schemes, including both
>> the old redhat cluster stack and modern corosync/openais. We use Ubuntu
>> Hardy with backported corosync and clvm packages, and it works pretty
>> well. Hand-backporting is not for the faint of heart, though.
>>
>> --Quentin
>
> Thanks for the thoughts Quentin. Would you mind sharing your current
> versions (of clvm, corosync, openais, and pacemaker) and VM technology?

It's not currently in a good state for being deployed elsewhere, but you
can look at http://invirt.mit.edu/. You can also steal our backports out
of our apt repo at http://xvm.mit.edu/invirt/, but they obviously come
with _no warranty_.

>
> I agree about hand-backporting - i'd be more inclined to run unstable than to
> hand-backport.

It looks like semi-recent corosync is in sid - maybe it also has a recent
clvm?

>
> Any ideas on whether LVs in the same VG can be used on different hosts
> simultaneously with cLVM?

Yes, that is the whole premise of cLVM; LVs can be used by any number of
hosts simultaneously. (Obviously, you need to make sure that if you mount
the same LV from multiple places, that you're using a filesystem that can
be mounted simultaneously such as OCFS).

--Quentin

_______________________________________________
Pacemaker mailing list: Pacemaker [at] oss
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf


florian.haas at linbit

Apr 25, 2010, 11:06 AM

Post #6 of 6 (1284 views)
Permalink
Re: Searching for a viable Debian solution [In reply to]

Paul,

I am copying your message over to the Debian HA maintainers' mailing
list. Chances are that one of those guys can share some valuable insight.

Debian maintainers, when you respond would you mind copying the
Pacemaker list?

Cheers,
Florian

On 04/24/2010 06:01 AM, Paul Gear wrote:
> Hi folks,
>
> Over the last several days i've been reading, asking questions (special
> thanks to beekhof, tserong, fghass and the other kind folks on #linux-ha
> putting up with my questions), and experimenting with my test setup, and
> i'm yet to find a viable combination of options for creating a VM
> cluster on Debian.
>
> The preferred components of my setup are:
> - Debian lenny
> - Xen VMs
> - OCFS2 on an iSCSI NAS for shared storage
> - whatever openais/corosync/pacemaker combination will do the job
> - I'd like to have live migration of VMs but it's not essential
>
> The point of my setup is to have a test cluster with which i can play
> (and run my own home network) without breaking the ones i have to
> support at customers. I work with small businesses, educational
> institutions, and non-profits, none of whom have the means to support a
> fully working test configuration, or to buy time from big name
> consultants who can employ full-time clustering specialists.
>
> My experiments thus far have been with corosync & pacemaker from
> http://people.debian.org/~madkiss/ha/, and with libvirt from
> lenny-backports.
>
> When i tried the latest versions of corosync & pacemaker in the above
> repo, i was able to get the cluster running appropriately, but when i
> tried to add the DLM clone resource, i got this error: "controld[3526]:
> ERROR: Setup problem: Couldn't find utility dlm_controld.pcmk". So it
> seems that OCFS2 is not viable with the current state of the
> corosync/openais/pacemaker stack. I've investigated a couple of
> alternatives to my preferred list, including:
>
> - cLVM - requires the Red Hat cluster stack and conflicts with the
> current corosync/openais/pacemaker versions. This would be my preferred
> solution if it were possible to use LVs from the same VG on different
> nodes simultaneously. I'm not sure whether this is possible, because
> the cLVM documentation is almost non-existent, and i couldn't find
> answers to any of my big questions.
>
> - libvirt with iSCSI as the backend for the storage pool. I've tried
> this and it looks promising, but it appears non-functional on iSCSI
> storage backends at the moment. I get this message when i try to create
> a volume: "libvirtd: ... error : storageVolumeCreateXML:1301 : this
> function is not supported by the hypervisor: storage pool does not
> support volume creation"
>
> I substituted KVM for Xen, and the result was the same.
>
> So, the question remains: what is a viable HA stack for Debian with
> virtualization and shared storage? I'm happy to switch technologies
> where it's necessary; here are the things i would be willing to try if
> necessary:
>
> - Different clustered file storage setup. What are the possibilities?
> GFS2? DRBD seems feasible, but it doesn't actually solve anything in
> the above equation, so i'd rather keep the iSCSI NAS.
>
> - Switching distros? Possible - my order of preference: Debian squeeze,
> Debian sid, Ubuntu lucid, openSUSE 11.2, CentOS 5.4.
>
> - Switching cluster stacks? I've worked with clusters on HP-UX,
> NetWare, OES/Linux 1, heartbeat 1, and heartbeat 2, so i'm sure one more
> change won't kill me, but the more similar in feel it is to pacemaker or
> heartbeat 2, the better. If using non-integrated OCFS2 cluster
> membership and an older openais or heartbeat version is viable, i'm
> happy to look at that as well, although obviously i'd rather go forward
> than backward.
>
> - Switching VM technologies? I guess KVM, VirtualBox, or OpenVZ would
> be viable, but again, it doesn't seem to solve much in the current
> scenario.
>
> What's the path of least resistance here?
>
> Thanks in advance,
> Paul
Attachments: signature.asc (0.25 KB)

Linux-HA pacemaker RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.