Mailing List Archive: Linux-HA: HA-WG-Technical

Proposal to drop vgck in LVM RA

Index | Next | Previous | View Threaded

xwhu at suse

Nov 14, 2011, 1:00 AM

Post #1 of 11 (1623 views)
Permalink

Proposal to drop vgck in LVM RA

Hi all,

Here attached a patch, to remove vgck in the LVM RA completely.

As we know, the LVM RA's used to provide a working VG for others to use.
However, vgck is a tool to check the metadata consistency across all
physical volumes. So, it can be the case that the VG is living and everything's
fine, but the metadata is in-consistent on physical volumes. This can be
found oftentimes on mirrored VG.

So I proposed to drop vgck in the RA completely, for
1. avoiding the false alarm by vgck
2. performance imporoving vgck is time and disk consuming cause it scan
all physical volumes for metadata.

Please let me know if you have concerns on this.

Thanks.

Attachments:

vg-monitor-dont-check.DIFF (0.74 KB)

dejan at suse

Nov 15, 2011, 6:44 AM

Post #2 of 11 (1555 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Xinwei,

On Mon, Nov 14, 2011 at 02:00:23AM -0700, Xinwei Hu wrote:
> Hi all,
>
> Here attached a patch, to remove vgck in the LVM RA completely.
>
>
> As we know, the LVM RA's used to provide a working VG for others to use.
> However, vgck is a tool to check the metadata consistency across all
> physical volumes. So, it can be the case that the VG is living and everything's
> fine, but the metadata is in-consistent on physical volumes. This can be
> found oftentimes on mirrored VG.
>
>
> So I proposed to drop vgck in the RA completely, for
> 1. avoiding the false alarm by vgck
> 2. performance imporoving vgck is time and disk consuming cause it scan
> all physical volumes for metadata.
>
>
> Please let me know if you have concerns on this.

It sounds reasonable. At any rate, better monitor resources which
are on top of volume groups.

Cheers,

Dejan

> Thanks.
>
>
>

> Index: heartbeat/LVM
> ===================================================================
> --- heartbeat.orig/LVM 2011-11-14 16:20:10.000000000 +0800
> +++ heartbeat/LVM 2011-11-14 16:25:57.000000000 +0800
> @@ -187,13 +187,11 @@
> return $OCF_NOT_RUNNING
> fi
>
> - VGOUT=`vgck $1 2>&1`
> - rc=$?
> - if [ $rc -ne 0 ]; then
> - ocf_log err "LVM Volume $1 is not found. ${VGOUT}:${rc}"
> - return $OCF_ERR_GENERIC
> - fi
> -
> +# vgck is removed from LVM_monitor because:
> +# . it's checking metadata on the disk, not the current running status of LV
> +# . failed vgck means in-consistency of metadata across disks, but it not nessaryly means
> +# LV is broken.
> +# . it can be very time/disk comsuming if running on a large pool of PV devices.
> return $OCF_SUCCESS
> }
>

> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

xwhu at suse

Nov 16, 2011, 5:54 AM

Post #3 of 11 (1561 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Dejan,

Thanks for reply.

I enhance the patch a little again. It seems to be good idea to
do vgck in probe, and in probe only. So that we can still benefit
from the consistency checking, meanwhile, we won't suffer the
performance and logic penalty.

Comments are welcome, again ;)

Thanks.
>>> Dejan Muhamedagic 11/15/11 10:44 下午 >>>
Hi Xinwei,

On Mon, Nov 14, 2011 at 02:00:23AM -0700, Xinwei Hu wrote:
> Hi all,
>
> Here attached a patch, to remove vgck in the LVM RA completely.
>
>
> As we know, the LVM RA's used to provide a working VG for others to use.
> However, vgck is a tool to check the metadata consistency across all
> physical volumes. So, it can be the case that the VG is living and everything's
> fine, but the metadata is in-consistent on physical volumes. This can be
> found oftentimes on mirrored VG.
>
>
> So I proposed to drop vgck in the RA completely, for
> 1. avoiding the false alarm by vgck
> 2. performance imporoving vgck is time and disk consuming cause it scan
> all physical volumes for metadata.
>
>
> Please let me know if you have concerns on this.

It sounds reasonable. At any rate, better monitor resources which
are on top of volume groups.

Cheers,

Dejan

> Thanks.
>
>
>

> Index: heartbeat/LVM
> ===================================================================
> --- heartbeat.orig/LVM 2011-11-14 16:20:10.000000000 +0800
> +++ heartbeat/LVM 2011-11-14 16:25:57.000000000 +0800
> @@ -187,13 +187,11 @@
> return $OCF_NOT_RUNNING
> fi
>
> - VGOUT=`vgck $1 2>&1`
> - rc=$?
> - if [ $rc -ne 0 ]; then
> - ocf_log err "LVM Volume $1 is not found. ${VGOUT}:${rc}"
> - return $OCF_ERR_GENERIC
> - fi
> -
> +# vgck is removed from LVM_monitor because:
> +# . it's checking metadata on the disk, not the current running status of LV
> +# . failed vgck means in-consistency of metadata across disks, but it not nessaryly means
> +# LV is broken.
> +# . it can be very time/disk comsuming if running on a large pool of PV devices.
> return $OCF_SUCCESS
> }
>

> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

Attachments:

vg-monitor-dont-check.diff (1.02 KB)

renayama19661014 at ybb

Dec 5, 2011, 4:13 PM

Post #4 of 11 (1505 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Dejan,
Hi Xinwei,

We pay attention to this correction, too.

Did the discussion advance afterwards?
Was the method of the correction decided?

Best Regards,
Hideo Yamauchi.

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

dejan at suse

Dec 6, 2011, 2:25 AM

Post #5 of 11 (1505 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Hideo-san,

On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> Hi Dejan,
> Hi Xinwei,
>
> We pay attention to this correction, too.
>
> Did the discussion advance afterwards?
> Was the method of the correction decided?

Actually not, it somehow slipped off my radar.

Xinwei suggested in his latest message to run vgck on probe, but
this may result in error if the device isn't ready yet (it may
happen say with iscsi). So, I think that we should just drop vgck
altogether.

Thanks,

Dejan

> Best Regards,
> Hideo Yamauchi.
>
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

renayama19661014 at ybb

Dec 7, 2011, 4:14 PM

Post #6 of 11 (1493 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Dejan,

Thank you for comment.
We examine a correction of LVM_validate_all.
Because the handling of vgck influences it, I am going to obey the decision of this argument.

For example, even the following simple choice may be good.
* Add "exec_vgck" parameter
* true(default) : Exec vgck command.
* false : Not exec vgck command.

Best Regards,
Hideo Yamauchi.

--- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:

> Hi Hideo-san,
>
> On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > Hi Dejan,
> > Hi Xinwei,
> >�
> > We pay attention to this correction, too.
> >�
> > Did the discussion advance afterwards?
> > Was the method of the correction decided?
>
> Actually not, it somehow slipped off my radar.
>
> Xinwei suggested in his latest message to run vgck on probe, but
> this may result in error if the device isn't ready yet (it may
> happen say with iscsi). So, I think that we should just drop vgck
> altogether.
>
> Thanks,
>
> Dejan
>
> > Best Regards,
> > Hideo Yamauchi.
> >�
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

dejan at suse

Dec 8, 2011, 9:44 PM

Post #7 of 11 (1485 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Hideo-san,

On Thu, Dec 08, 2011 at 09:14:24AM +0900, renayama19661014 [at] ybb wrote:
> Hi Dejan,
>
> Thank you for comment.
> We examine a correction of LVM_validate_all.
> Because the handling of vgck influences it, I am going to obey the decision of this argument.
>
> For example, even the following simple choice may be good.
> * Add "exec_vgck" parameter
> * true(default) : Exec vgck command.
> * false : Not exec vgck command.

Sounds reasonable, but I'd still rather change the default to
false. Even though it does change the current behaviour of the
RA, it seems like the check was anyway superfluous. Or do you see
it differently?

Cheers,

Dejan

> Best Regards,
> Hideo Yamauchi.
>
> --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:
>
> > Hi Hideo-san,
> >
> > On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > > Hi Dejan,
> > > Hi Xinwei,
> > >�
> > > We pay attention to this correction, too.
> > >�
> > > Did the discussion advance afterwards?
> > > Was the method of the correction decided?
> >
> > Actually not, it somehow slipped off my radar.
> >
> > Xinwei suggested in his latest message to run vgck on probe, but
> > this may result in error if the device isn't ready yet (it may
> > happen say with iscsi). So, I think that we should just drop vgck
> > altogether.
> >
> > Thanks,
> >
> > Dejan
> >
> > > Best Regards,
> > > Hideo Yamauchi.
> > >�
> > > _______________________________________________
> > > ha-wg-technical mailing list
> > > ha-wg-technical [at] lists
> > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

renayama19661014 at ybb

Dec 11, 2011, 4:11 PM

Post #8 of 11 (1470 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Dejan,

Thank you for comment.

> > For example, even the following simple choice may be good.
> > * Add "exec_vgck" parameter
> > * true(default) : Exec vgck command.
> > * false : Not exec vgck command.
>
> Sounds reasonable, but I'd still rather change the default to
> false. Even though it does change the current behaviour of the
> RA, it seems like the check was anyway superfluous. Or do you see
> it differently?

I think that the value of the default does not mind either.
However, the change of the default has an influence on the environment where vgck check does not have any problem anymore in a user using RA of LVM.

The user who upgraded RA will have doubt toward what is not checked in vgck.
Because I did not invite the confusion of such a user, the default thought that "true" was good.

Best Regards,
Hideo Yamauchi.

--- On Fri, 2011/12/9, Dejan Muhamedagic <dejan [at] suse> wrote:

> Hi Hideo-san,
>
> On Thu, Dec 08, 2011 at 09:14:24AM +0900, renayama19661014 [at] ybb wrote:
> > Hi Dejan,
> >
> > Thank you for comment.
> > We examine a correction of LVM_validate_all.
> > Because the handling of vgck influences it, I am going to obey the decision of this argument.
> >
> > For example, even the following simple choice may be good.
> >� * Add "exec_vgck" parameter
> >��* true(default) : Exec vgck command.
> >��* false : Not exec vgck command.
>
> Sounds reasonable, but I'd still rather change the default to
> false. Even though it does change the current behaviour of the
> RA, it seems like the check was anyway superfluous. Or do you see
> it differently?
>
> Cheers,
>
> Dejan
>
> > Best Regards,
> > Hideo Yamauchi.
> >
> > --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:
> >
> > > Hi Hideo-san,
> > >
> > > On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > > > Hi Dejan,
> > > > Hi Xinwei,
> > > >�
> > > > We pay attention to this correction, too.
> > > >�
> > > > Did the discussion advance afterwards?
> > > > Was the method of the correction decided?
> > >
> > > Actually not, it somehow slipped off my radar.
> > >
> > > Xinwei suggested in his latest message to run vgck on probe, but
> > > this may result in error if the device isn't ready yet (it may
> > > happen say with iscsi). So, I think that we should just drop vgck
> > > altogether.
> > >
> > > Thanks,
> > >
> > > Dejan
> > >
> > > > Best Regards,
> > > > Hideo Yamauchi.
> > > >�
> > > > _______________________________________________
> > > > ha-wg-technical mailing list
> > > > ha-wg-technical [at] lists
> > > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > > _______________________________________________
> > > ha-wg-technical mailing list
> > > ha-wg-technical [at] lists
> > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > >
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

renayama19661014 at ybb

Jan 9, 2012, 6:28 PM

Post #9 of 11 (1364 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Dejan,

How do you think about this matter?

Best Regards,
Hideo Yamauchi.

--- On Thu, 2011/12/8, renayama19661014 [at] ybb <renayama19661014 [at] ybb> wrote:

> Hi Dejan,
>
> Thank you for comment.
> We examine a correction of LVM_validate_all.
> Because the handling of vgck influences it, I am going to obey the decision of this argument.
>
> For example, even the following simple choice may be good.
> * Add "exec_vgck" parameter
> � * true(default) : Exec vgck command.
> � * false : Not exec vgck command.
>
> Best Regards,
> Hideo Yamauchi.
>
> --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:
>
> > Hi Hideo-san,
> >
> > On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > > Hi Dejan,
> > > Hi Xinwei,
> > >�
> > > We pay attention to this correction, too.
> > >�
> > > Did the discussion advance afterwards?
> > > Was the method of the correction decided?
> >
> > Actually not, it somehow slipped off my radar.
> >
> > Xinwei suggested in his latest message to run vgck on probe, but
> > this may result in error if the device isn't ready yet (it may
> > happen say with iscsi). So, I think that we should just drop vgck
> > altogether.
> >
> > Thanks,
> >
> > Dejan
> >
> > > Best Regards,
> > > Hideo Yamauchi.
> > >�
> > > _______________________________________________
> > > ha-wg-technical mailing list
> > > ha-wg-technical [at] lists
> > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
>
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

dejan at suse

Jan 10, 2012, 5:22 AM

Post #10 of 11 (1355 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hi Hideo-san,

On Tue, Jan 10, 2012 at 11:28:12AM +0900, renayama19661014 [at] ybb wrote:
> Hi Dejan,
>
> How do you think about this matter?

I'm still inclined to drop vgck from monitor and use it just
before start. I wouldn't even consider that a regression.

I'm also not sure what does vgck offer in comparison with
vgdisplay and if both actually work with the on-disk lvm
meta-data. In that case we should drop vgdisplay as well and find
another (and better) way to monitor VGs.

Anybody with deeper knowledge on LVM?

Cheers,

Dejan

> Best Regards,
> Hideo Yamauchi.
>
>
> --- On Thu, 2011/12/8, renayama19661014 [at] ybb <renayama19661014 [at] ybb> wrote:
>
> > Hi Dejan,
> >
> > Thank you for comment.
> > We examine a correction of LVM_validate_all.
> > Because the handling of vgck influences it, I am going to obey the decision of this argument.
> >
> > For example, even the following simple choice may be good.
> > * Add "exec_vgck" parameter
> > � * true(default) : Exec vgck command.
> > � * false : Not exec vgck command.
> >
> > Best Regards,
> > Hideo Yamauchi.
> >
> > --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:
> >
> > > Hi Hideo-san,
> > >
> > > On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > > > Hi Dejan,
> > > > Hi Xinwei,
> > > >�
> > > > We pay attention to this correction, too.
> > > >�
> > > > Did the discussion advance afterwards?
> > > > Was the method of the correction decided?
> > >
> > > Actually not, it somehow slipped off my radar.
> > >
> > > Xinwei suggested in his latest message to run vgck on probe, but
> > > this may result in error if the device isn't ready yet (it may
> > > happen say with iscsi). So, I think that we should just drop vgck
> > > altogether.
> > >
> > > Thanks,
> > >
> > > Dejan
> > >
> > > > Best Regards,
> > > > Hideo Yamauchi.
> > > >�
> > > > _______________________________________________
> > > > ha-wg-technical mailing list
> > > > ha-wg-technical [at] lists
> > > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > > _______________________________________________
> > > ha-wg-technical mailing list
> > > ha-wg-technical [at] lists
> > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > >
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> >
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

dejan at suse

Feb 2, 2012, 8:44 AM

Post #11 of 11 (1284 views)
Permalink

Re: Proposal to drop vgck in LVM RA [In reply to]

Hello all,

Sorry for crossposting, but can anybody comment on the matter
bellow? Thanks!

Dejan

On Tue, Jan 10, 2012 at 02:22:35PM +0100, Dejan Muhamedagic wrote:
> Hi Hideo-san,
>
> On Tue, Jan 10, 2012 at 11:28:12AM +0900, renayama19661014 [at] ybb wrote:
> > Hi Dejan,
> >
> > How do you think about this matter?
>
> I'm still inclined to drop vgck from monitor and use it just
> before start. I wouldn't even consider that a regression.
>
> I'm also not sure what does vgck offer in comparison with
> vgdisplay and if both actually work with the on-disk lvm
> meta-data. In that case we should drop vgdisplay as well and find
> another (and better) way to monitor VGs.
>
> Anybody with deeper knowledge on LVM?
>
> Cheers,
>
> Dejan
>
> > Best Regards,
> > Hideo Yamauchi.
> >
> >
> > --- On Thu, 2011/12/8, renayama19661014 [at] ybb <renayama19661014 [at] ybb> wrote:
> >
> > > Hi Dejan,
> > >
> > > Thank you for comment.
> > > We examine a correction of LVM_validate_all.
> > > Because the handling of vgck influences it, I am going to obey the decision of this argument.
> > >
> > > For example, even the following simple choice may be good.
> > > * Add "exec_vgck" parameter
> > > � * true(default) : Exec vgck command.
> > > � * false : Not exec vgck command.
> > >
> > > Best Regards,
> > > Hideo Yamauchi.
> > >
> > > --- On Tue, 2011/12/6, Dejan Muhamedagic <dejan [at] suse> wrote:
> > >
> > > > Hi Hideo-san,
> > > >
> > > > On Tue, Dec 06, 2011 at 09:13:31AM +0900, renayama19661014 [at] ybb wrote:
> > > > > Hi Dejan,
> > > > > Hi Xinwei,
> > > > >�
> > > > > We pay attention to this correction, too.
> > > > >�
> > > > > Did the discussion advance afterwards?
> > > > > Was the method of the correction decided?
> > > >
> > > > Actually not, it somehow slipped off my radar.
> > > >
> > > > Xinwei suggested in his latest message to run vgck on probe, but
> > > > this may result in error if the device isn't ready yet (it may
> > > > happen say with iscsi). So, I think that we should just drop vgck
> > > > altogether.
> > > >
> > > > Thanks,
> > > >
> > > > Dejan
> > > >
> > > > > Best Regards,
> > > > > Hideo Yamauchi.
> > > > >�
> > > > > _______________________________________________
> > > > > ha-wg-technical mailing list
> > > > > ha-wg-technical [at] lists
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > > > _______________________________________________
> > > > ha-wg-technical mailing list
> > > > ha-wg-technical [at] lists
> > > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > > >
> > > _______________________________________________
> > > ha-wg-technical mailing list
> > > ha-wg-technical [at] lists
> > > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> > >
> > _______________________________________________
> > ha-wg-technical mailing list
> > ha-wg-technical [at] lists
> > https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linuxfoundation.org/mailman/listinfo/ha-wg-technical

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads