Mailing List Archive: Linux-HA: HA-WG-Technical

RA Spec: A new API for utilization probe

Index | Next | Previous | View Threaded

xwhu at novell

Jul 15, 2011, 12:10 AM

Post #1 of 6 (988 views)
Permalink

RA Spec: A new API for utilization probe

Hi all,

This is actually about the possible next step about utilization based
resource allocation in pacemaker. As we know, current pacemaker has the
ability to do resource allocation based on the utilization of nodes and
resources. But all these have to be set prior running, and belong to the
configuration of CIB.

It should be a nature evolution for pacemaker to support dynamic
utilization based resource reallocation. For that, I'd proposal
following changes for you to review.

- pacemaker to store the dynamic utilization value in the status
section of CIB. A changing to the value in configuration triggers a new
transaction, which is not optimal when trying to update the utilization
of a pile of resources. Also, by storing it in status, we can leverage
the attrd_updater with --dempan.

- A new API for RA, for pacemaker to probe the utilization of a
resource instance. I'd suggest a obvious name, like probe_utilization.
This new API requires no parameter, and always return $OCF_SUCCESS. RA
takes the responsibility to update the utilization value when called.

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

andrew at beekhof

Jul 18, 2011, 2:55 PM

Post #2 of 6 (934 views)
Permalink

Re: RA Spec: A new API for utilization probe [In reply to]

On Fri, Jul 15, 2011 at 5:10 PM, Xin Wei Hu <xwhu [at] novell> wrote:
> Hi all,
>
> This is actually about the possible next step about utilization based
> resource allocation in pacemaker. As we know, current pacemaker has the
> ability to do resource allocation based on the utilization of nodes and
> resources. But all these have to be set prior running, and belong to the
> configuration of CIB.
>
> It should be a nature evolution for pacemaker to support dynamic
> utilization based resource reallocation. For that, I'd proposal
> following changes for you to review.
>
> - pacemaker to store the dynamic utilization value in the status
> section of CIB. A changing to the value in configuration triggers a new
> transaction, which is not optimal when trying to update the utilization
> of a pile of resources.

Storing it in the status section wouldn't necessarily change this.
Is there any reason not to do this during a probe instead?

> Also, by storing it in status, we can leverage
> the attrd_updater with --dempan.
>
> - A new API for RA, for pacemaker to probe the utilization of a
> resource instance. I'd suggest a obvious name, like probe_utilization.
> This new API requires no parameter, and always return $OCF_SUCCESS. RA
> takes the responsibility to update the utilization value when called.
>
>
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical
>
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

lmb at suse

Jul 22, 2011, 4:31 AM

Post #3 of 6 (922 views)
Permalink

Re: RA Spec: A new API for utilization probe [In reply to]

On 2011-07-19T07:55:10, Andrew Beekhof <andrew [at] beekhof> wrote:

> Is there any reason not to do this during a probe instead?

Yes. A probe doesn't necessarily have the pre-requisites available to
detect the requirements. (Think "VM configuration on shared storage".)

In fact, this call would, I hate to say it, almost have the same
semantics as the proposal "validate-all" executed on "some" eligible
node prior to doing start. But folding this into "validate-all"
definitely would be a horrible naming ;-)

Regards,
Lars

--
Architect Storage/HA, OPS Engineering, Novell, Inc.
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

andrew at beekhof

Jul 26, 2011, 6:18 PM

Post #4 of 6 (907 views)
Permalink

Re: RA Spec: A new API for utilization probe [In reply to]

On Fri, Jul 22, 2011 at 9:31 PM, Lars Marowsky-Bree <lmb [at] suse> wrote:
> On 2011-07-19T07:55:10, Andrew Beekhof <andrew [at] beekhof> wrote:
>
>> Is there any reason not to do this during a probe instead?
>
> Yes. A probe doesn't necessarily have the pre-requisites available to
> detect the requirements. (Think "VM configuration on shared storage".)
>
> In fact, this call would, I hate to say it, almost have the same
> semantics as the proposal "validate-all" executed on "some" eligible
> node prior to doing start. But folding this into "validate-all"
> definitely would be a horrible naming ;-)

No question that it should not be in validate-all.
My concern is that I don't see how we can bootstrap this well.

The options seem to be:
a) do it as part of the start operation, or
b) teach the PE about a new operation*

But:

a) means blindly starting resources (and those that depend on it),
hoping that the host has enough capacity (and restart the transition
every time new utilisation data is uploaded)

b) gives you a slower startup and only tells us if the destination has
the capacity to host that one resource, not those that depend on it
too.
it does give less random transition restarts though.

Neither seems particularly complete/desirable.

* "capacity" ?

>
>
> Regards,
> Lars
>
> --
> Architect Storage/HA, OPS Engineering, Novell, Inc.
> SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
> "Experience is the name everyone gives to their mistakes." -- Oscar Wilde
>
>
_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

lmb at suse

Jul 27, 2011, 2:29 AM

Post #5 of 6 (901 views)
Permalink

Re: RA Spec: A new API for utilization probe [In reply to]

On 2011-07-27T11:18:59, Andrew Beekhof <andrew [at] beekhof> wrote:

> The options seem to be:
> a) do it as part of the start operation, or
> b) teach the PE about a new operation*
>
> But:
>
> a) means blindly starting resources (and those that depend on it),
> hoping that the host has enough capacity (and restart the transition
> every time new utilisation data is uploaded)

> b) gives you a slower startup and only tells us if the destination has
> the capacity to host that one resource, not those that depend on it
> too. it does give less random transition restarts though.

I thought I'd lean towards b, and wrote a longish mail about how the
semantics of such a "discover" operation could work, how the PE would
handle it etc - just to realize it is mostly pointless.

Because this only matters if the resource is activated for the very
first time, normally. Very few resources are going to change in their
load data frequently.

So for those, we're talking about a mandatory "go back to the PE" cycle
for the first round - but a "start" that updates the capacity data would
already cause that, anyway.

For those that *are* dynamically updated, a "discover" call preceding
start wouldn't be enough anyway, since either their start or monitor
calls would adjust the capacity requirements more often anyway.

Right now, I'm leaning towards a); the agent can update the utilization
data if it has changed ("previously unset" to "now set" would count as a
change, obviously), and then we need to go back to the PE anyway. As an
optimization, it'd be nice if the "start" were allowed to return
"OCF_NOT_RUNNING" so that we don't need to bother with the "stop".

This is "efficient enough" if we assume resources are added one by one,
or at least in smaller numbers.

For "bulk add", a "discover" operation would have an advantage; we could
bring all resources to the state where their dependencies are satisfied,
schedule "discover", and hand it back to the PE then; that would be
quite a win in number of transitions. But I don't think the complexity
added to the PE justifies the gain.

But. To still get the savings, what if the _UI_ called "discover" on
those agents (bypassing the Pacemaker), so that it could directly add
the resources with the appropriate utilization data? Then the bulk-add
wouldn't cause the problem either.

Maybe we could extend the "validate-all" or "discover" semantics to
return more elaborate status (say, "key=value" on stdout/stderr) that
the UI could interpret accordingly - at least for the agents that want
this?

Regards,
Lars

--
Architect Storage/HA, OPS Engineering, Novell, Inc.
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

andrew at beekhof

Jul 31, 2011, 3:44 PM

Post #6 of 6 (877 views)
Permalink

Re: RA Spec: A new API for utilization probe [In reply to]

On 27/07/2011, at 7:29 PM, Lars Marowsky-Bree wrote:

> On 2011-07-27T11:18:59, Andrew Beekhof <andrew [at] beekhof> wrote:
>
>> The options seem to be:
>> a) do it as part of the start operation, or
>> b) teach the PE about a new operation*
>>
>> But:
>>
>> a) means blindly starting resources (and those that depend on it),
>> hoping that the host has enough capacity (and restart the transition
>> every time new utilisation data is uploaded)
>
>> b) gives you a slower startup and only tells us if the destination has
>> the capacity to host that one resource, not those that depend on it
>> too. it does give less random transition restarts though.
>
> I thought I'd lean towards b, and wrote a longish mail about how the
> semantics of such a "discover" operation could work, how the PE would
> handle it etc - just to realize it is mostly pointless.
>
> Because this only matters if the resource is activated for the very
> first time, normally. Very few resources are going to change in their
> load data frequently.
>
> So for those, we're talking about a mandatory "go back to the PE" cycle
> for the first round - but a "start" that updates the capacity data would
> already cause that, anyway.
>
> For those that *are* dynamically updated, a "discover" call preceding
> start wouldn't be enough anyway, since either their start or monitor
> calls would adjust the capacity requirements more often anyway.
>
>
> Right now, I'm leaning towards a); the agent can update the utilization
> data if it has changed ("previously unset" to "now set" would count as a
> change, obviously), and then we need to go back to the PE anyway. As an
> optimization, it'd be nice if the "start" were allowed to return
> "OCF_NOT_RUNNING" so that we don't need to bother with the "stop".

No objection.

>
> This is "efficient enough" if we assume resources are added one by one,
> or at least in smaller numbers.
>
> For "bulk add", a "discover" operation would have an advantage; we could
> bring all resources to the state where their dependencies are satisfied,
> schedule "discover", and hand it back to the PE then; that would be
> quite a win in number of transitions. But I don't think the complexity
> added to the PE justifies the gain.
>
>
> But. To still get the savings, what if the _UI_ called "discover" on
> those agents (bypassing the Pacemaker), so that it could directly add
> the resources with the appropriate utilization data? Then the bulk-add
> wouldn't cause the problem either.

Presumably after everything was started?

>
> Maybe we could extend the "validate-all" or "discover"

not validate-all please

> semantics to
> return more elaborate status (say, "key=value" on stdout/stderr) that
> the UI could interpret accordingly - at least for the agents that want
> this?

I really detest stderr/stdout parsing, but if we have to have it I think XML is the right format (like metadata).

>
>
> Regards,
> Lars
>
> --
> Architect Storage/HA, OPS Engineering, Novell, Inc.
> SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg)
> "Experience is the name everyone gives to their mistakes." -- Oscar Wilde
>
> _______________________________________________
> ha-wg-technical mailing list
> ha-wg-technical [at] lists
> https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

_______________________________________________
ha-wg-technical mailing list
ha-wg-technical [at] lists
https://lists.linux-foundation.org/mailman/listinfo/ha-wg-technical

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads