Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Linux-HA: Dev

ha_log() as defined in /etc/ha,d/shellfuncs

 

 

Linux-HA dev RSS feed   Index | Next | Previous | View Threaded


pica1dilly at yahoo

Jun 20, 2008, 7:18 AM

Post #1 of 8 (518 views)
Permalink
ha_log() as defined in /etc/ha,d/shellfuncs

In /etc/ha.d/shellfuncs.sh, as part of ha_log(), we have the following sequence:

>if
> [ -n "$HA_LOGFILE" ]
> then
> : appending to $HA_LOGFILE
> Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_LOGFILE
> else
> Echo `hadate`"${*}" >&2
>fi
>if
> [ -n "$HA_DEBUGLOG" ]
> then
> : appending to $HA_DEBUGLOG
> Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_DEBUGLOG
>fi
>

If a ***same*** log entry ( that is, a message passed to ha_log() just ***once***, to be processed just ***once***, ha_log() being called just once*** ), is to be written in 2 different files as above, wouldn't the least be that same entry carry the ***same date/time***, whatever the file it is written to and regardless the number of times it is written down, instead of recalculating the date/time each time the same log entry is written down ?




_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


dejanmm at fastmail

Jun 20, 2008, 8:00 AM

Post #2 of 8 (490 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

On Fri, Jun 20, 2008 at 07:18:55AM -0700, Joe Bill wrote:
> In /etc/ha.d/shellfuncs.sh, as part of ha_log(), we have the following sequence:
>
> >if
> > [ -n "$HA_LOGFILE" ]
> > then
> > : appending to $HA_LOGFILE
> > Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_LOGFILE
> > else
> > Echo `hadate`"${*}" >&2
> >fi
> >if
> > [ -n "$HA_DEBUGLOG" ]
> > then
> > : appending to $HA_DEBUGLOG
> > Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_DEBUGLOG
> >fi
> >
>
> If a ***same*** log entry ( that is, a message passed to
> ha_log() just ***once***, to be processed just ***once***,
> ha_log() being called just once*** ), is to be written in 2
> different files as above, wouldn't the least be that same entry
> carry the ***same date/time***, whatever the file it is written
> to and regardless the number of times it is written down,
> instead of recalculating the date/time each time the same log
> entry is written down ?

Definitely. tee(1) would probably be the best option.
_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


lmb at suse

Jun 20, 2008, 1:14 PM

Post #3 of 8 (481 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

On 2008-06-20T07:18:55, Joe Bill <pica1dilly[at]yahoo.com> wrote:

> In /etc/ha.d/shellfuncs.sh, as part of ha_log(), we have the following sequence:
>
> >if
> > [ -n "$HA_LOGFILE" ]
> > then
> > : appending to $HA_LOGFILE
> > Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_LOGFILE
> > else
> > Echo `hadate`"${*}" >&2
> >fi
> >if
> > [ -n "$HA_DEBUGLOG" ]
> > then
> > : appending to $HA_DEBUGLOG
> > Echo "$HA_LOGTAG: "`hadate`"${*}" >> $HA_DEBUGLOG
> >fi
> >
>
> If a ***same*** log entry ( that is, a message passed to ha_log() just ***once***, to be processed just ***once***, ha_log() being called just once*** ), is to be written in 2 different files as above, wouldn't the least be that same entry carry the ***same date/time***, whatever the file it is written to and regardless the number of times it is written down, instead of recalculating the date/time each time the same log entry is written down ?

Sure. The date should be calculated just once.

Keep in mind that this is legacy though. Anybody found to not be using
logd should be taken out and educated.

If anything, what I'd like even more to see is automatically defaulting
to the Linux-HA logging relay daemon ;-), and if that's not configured
explicitly but debuglog and logfile are, to auto-feed these to the
logging system configuration.

Nobody should not be using the logging daemon.


Regards,
Lars

--
Teamlead Kernel, SuSE Labs, Research and Development
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


pica1dilly at yahoo

Jun 24, 2008, 4:12 AM

Post #4 of 8 (452 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

--- On Fri, 6/20/08, Lars Marowsky-Bree <lmb[at]suse.de> wrote:

>> Eventually I end up providing one option with
>> ha_log() "to conform to the standard", regardless
>> of the result returned, and one option without ha_log(),
>> that guarantees the result I want to achieve while
>> improving the reliability and the predictability
>> over time
>
> On the subject of clarity, I have no idea what you're
> refering to here.

I am referring to this snippet of ha_log():
(/etc/ha.d/shellfuncs)

ha_logger -t "${HA_LOGTAG}" "$@"
if [ "$?" -eq "0" ] ; then
return 0 !!! if successful
fi

where, if ha_logger fails then ha_log() attempts to write directly to a logfile and to a debugfile. If ha_logger succeeds, ha_log() completes without writing to the log files.

If there are intermittent network problems, assembling the patches of log written in the syslog managed files, and those patches written in the local logfiles should guarantee maximum fun on a slow day.

As I wrote above, not knowing the intention of the developers regarding ha_log(), I'd rather provide 2 different functions to log, one home-made, and one calling ha_log(), to suit what the user prefers: sticking to the HA standard or retrieving meaningful log information.

>
> For resource agent, I'd recommend ocf_log and not
> ha_log, anyway.

After massaging the priority code a bit, this is what ocf_log() does:

(/usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs)

if [ "${__OCF_PRIO}" = "DEBUG" ]; then
ha_debug "${__OCF_PRIO}: $__OCF_MSG"
else
ha_log "${__OCF_PRIO}: $__OCF_MSG"
fi

Which doesn't make such a difference IMHO.


> Linux-HA logging relay (daemon), if you insist on using the
> full name.

Sounds just right to me

>> The fact is though, the component has two parts:
>> ha_logger and ha_logd.
>> If the daemon lacks the disk-writing, it's missing
>> then one of the main functions that characterizes
>> as a "logging" daemon.
>
> The daemon can write to disk too,

I apologize for having not looked in the source. Just in one line, how, what for, and when does the HA logging relay "write to disk" ?

> ... or relay to syslog.
> It's main function is buffering to decouple the HA
> stack from the disk or syslog system.

I'm afraid I don't see the benefit of "buffering" a message that is intended to be written to disk.

I understand though how buffering makes sense with an unreliable syslog that would drop messages.

But if the network is shakey, isn't your buffer written to disk at one point, after running out of non-paged memory ?

Now of course, if your storage itself is somewhere out there in the same network...

> If anything, what I'd like even more to see is
> automatically defaulting to the Linux-HA logging
> relay daemon ;-), and if that's not configured
> explicitly but debuglog and logfile are,
> to auto-feed these to the logging system
> configuration.
>
> Nobody should not be using the logging daemon.
>
> ha_logger is not a "the". It's just ha_logger,
> a command line tool to call, but no RA should
> be calling it directly.

Yes, but then ha_log() really needs to be fixed because some folks like me really like seeing that all log messages be written to local disk first, ***before*** being sent out on that "lossy" network, not only when the network is down or flakey as handled by ha_log() today.

Would you agree that ha_log() should write the local log file and/or debug file first, before calling ha_logger ? This would fulfill the condition that the message be saved once before being sent out on the network.

> Sure. The date should be calculated just once.
> Keep in mind that this is legacy though.

If writing log messages to local log and debug files is to be considered as "legacy", should we understand that for a RA to effectively and correctly write to local files, the RA should handle it by itself and not rely on ha_log() to do it, or even expect doing it in the future ?

> Anybody found to not be using logd should be taken out and educated.
>

Fair enough.
But for 'ha_logger' to replace 'logger', does require 'ha_logger' to support the same log facilities and log priorities as 'logger', just to ensure that out-of-band priority messaging, where it is already implemented with 'logger', is also supported with ha_logger.

So with ha_logger to support a '-p' option similar to logger, this would require to slightly modify ha_log().

Do we keep specifying to ha_log():

- the log facility through HA_LOGFACILITY
- the log priority as the uppercase prefix in the message itself, extending the available prefixes to *all* those support be 'logger' ?

Kind regards
Eric



_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


sid at kongregate

Jun 24, 2008, 9:06 AM

Post #5 of 8 (452 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

And how would they be educated? Is the use of logd documented anywhere? Is
the fact that ha_log is deprecated documented?

Sid


Keep in mind that this is legacy though. Anybody found to not be using
> logd should be taken out and educated.
>
> Regards,
> Lars
>
>


lmb at suse

Jun 25, 2008, 4:23 AM

Post #6 of 8 (443 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

On 2008-06-24T04:12:48, Joe Bill <pica1dilly[at]yahoo.com> wrote:

> ha_logger -t "${HA_LOGTAG}" "$@"
> if [ "$?" -eq "0" ] ; then
> return 0 !!! if successful
> fi
>
> where, if ha_logger fails then ha_log() attempts to write directly to a logfile and to a debugfile. If ha_logger succeeds, ha_log() completes without writing to the log files.
>
> If there are intermittent network problems, assembling the patches of log written in the syslog managed files, and those patches written in the local logfiles should guarantee maximum fun on a slow day.

Ah, yeah, that is weird.

But I've never actually seen ha_logger fail, in particular not with ha_logd.

However, you must realize that this is safe-guard in case something
/does/ go wrong, for example with writing to the logd (just that I've
never seen that happen, ever). In that case, if it didn't fall back to
recording the logs /somewhere/, they'd be lost.

> After massaging the priority code a bit, this is what ocf_log() does:
>
> (/usr/lib/ocf/resource.d/heartbeat/.ocf-shellfuncs)
>
> if [ "${__OCF_PRIO}" = "DEBUG" ]; then
> ha_debug "${__OCF_PRIO}: $__OCF_MSG"
> else
> ha_log "${__OCF_PRIO}: $__OCF_MSG"
> fi
>
> Which doesn't make such a difference IMHO.

The difference is that ocf_log is portable to RHT, ha_log is not. The
name of the function call is important. ha_log might go away when we
clean up the shell libraries - no OCF script should call that.

> > The daemon can write to disk too,
> I apologize for having not looked in the source. Just in one line,
> how, what for, and when does the HA logging relay "write to disk" ?

It writes to disk if you specify that you don't want it to relay to
syslog, but to logfiles.

It never directly sends messages on the network, though this would
probably be a neat trick to avoid needing to modify syslog on the
clients.

> I'm afraid I don't see the benefit of "buffering" a message that is
> intended to be written to disk.

The point is to decouple the cluster stack from the possibly blocking
write() call, either to the syslog socket or the disk. That's the main
reason for its existence.

> Yes, but then ha_log() really needs to be fixed because some folks
> like me really like seeing that all log messages be written to local
> disk first, ***before*** being sent out on that "lossy" network, not
> only when the network is down or flakey as handled by ha_log() today.

Your analysis is not correct. ha_logger will not write directly to disk
if there's a network problem, but if the local logging system (syslog
or logd) have failed for whatever reason.

> Would you agree that ha_log() should write the local log file and/or
> debug file first, before calling ha_logger ? This would fulfill the
> condition that the message be saved once before being sent out on the
> network.

No. I do not agree.

> > Sure. The date should be calculated just once.
> > Keep in mind that this is legacy though.
> If writing log messages to local log and debug files is to be considered as "legacy", should we understand that for a RA to effectively and correctly write to local files, the RA should handle it by itself and not rely on ha_log() to do it, or even expect doing it in the future ?

I don't know where you got this idea. RAs should call ocf_log().

I meant that ha_log is a legacy call, and that the code path you outline
- with the date being calculated in the shell library is legacy -
ha_logger will handle this.

> Do we keep specifying to ha_log():
>
> - the log facility through HA_LOGFACILITY

You should not set this manually. This is set in the configuration and
the RA should not mess with it.

> - the log priority as the uppercase prefix in the message itself,
> extending the available prefixes to *all* those support be 'logger' ?

ha_logger could be extended to support the additional prefixes, sure. We
accept patches.


Regards,
Lars

--
Teamlead Kernel, SuSE Labs, Research and Development
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


pica1dilly at yahoo

Jun 25, 2008, 11:30 AM

Post #7 of 8 (442 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

--- On Wed, 6/25/08, Lars Marowsky-Bree <lmb[at]suse.de> wrote:

>> ha_logger -t "${HA_LOGTAG}" "$@"
>> if [ "$?" -eq "0" ] ; then
>> return 0 !!! if successful
>> fi
>>
>> where, if ha_logger fails then ha_log() attempts to
>> write directly to a logfile and to a debugfile.
>>
>> If there are intermittent network problems, ...

> Ah, yeah, that is weird.
> But I've never actually seen ha_logger fail, in
> particular not with ha_logd.
> However, you must realize that this is safe-guard
> in case something /does/ go wrong, for example
> with writing to the logd

Ok. What I failed to realize was that any cross-network dialog between ha_logger and syslog via ha_logd was anyways asynchronous, and whatever dialog between ha_logger and ha_logd is local inter-process (independent from the network), and so ha_logger can not be affected by the state of the network. So the risk of ha_logger failing and forcing ha_log() to write to disk is merely the same as ha_logd crashing, which itself represents serious system integrity degradation.

I guess my confusion was caused by assuming ha_logd was a "syslog relay" per RFC 3164.

> ha_log might go away when we clean up the shell
> libraries - no OCF script should call that.

I hear you, loud and clear.

>
>>> The daemon can write to disk too,
> It writes to disk if you specify that you don't want it
> to relay to syslog, but to logfiles.

I've checked cl_log.c, namely the cl_direct_log function.
It appears that the daemon will write to the specified logfile, ***regardless of it relaying to syslog or not***, if a log filename is provided, and the priority of the message to log is not DEBUG.
May I kindly ask you to confirm my interpretation is correct ?

> It never directly sends messages on the network, though
> this would probably be a neat trick to avoid needing to
> modify syslog on the clients.

I'm not sure what you mean by "it" and why it would be necessary to "modify" the syslog.

Personally, I believe "it" would be the "ha_logger", and yes, wouldn't it be cool if the ha_logger could talk directly to a syslog daemon.

>> I'm afraid I don't see the benefit of "buffering"
>> a message that is intended to be written to disk.

> The point is to decouple the cluster stack from the
> possibly blocking write() call, either to the syslog
> socket or the disk.

So does ha_logd save over reboots messages it received from the logger but hasn't had time to process ?

> I don't know where you got this idea.

Sorry my questions sound sometimes a bit naive but are really thought to invite unambiguous ansewers.

> RAs should call ocf_log().

I hear you.

>> Do we keep specifying to ha_log():
>> - the log facility through HA_LOGFACILITY
>
> You should not set this manually. This is set in the
> configuration and the RA should not mess with it.

Well, say I write a RA that I believe should log to the cron log facility.
How am I to do that while using the ocf-shellfuncs ?

Thanks again for your invaluable help.
Eric



_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/


lmb at suse

Jun 26, 2008, 6:01 AM

Post #8 of 8 (430 views)
Permalink
Re: ha_log() as defined in /etc/ha,d/shellfuncs [In reply to]

On 2008-06-25T11:30:54, Joe Bill <pica1dilly[at]yahoo.com> wrote:

> Ok. What I failed to realize was that any cross-network dialog between
> ha_logger and syslog via ha_logd was anyways asynchronous, and
> whatever dialog between ha_logger and ha_logd is local inter-process
> (independent from the network), and so ha_logger can not be affected
> by the state of the network. So the risk of ha_logger failing and
> forcing ha_log() to write to disk is merely the same as ha_logd
> crashing, which itself represents serious system integrity
> degradation.

Right.

> I've checked cl_log.c, namely the cl_direct_log function.
> It appears that the daemon will write to the specified logfile, ***regardless of it relaying to syslog or not***, if a log filename is provided, and the priority of the message to log is not DEBUG.
> May I kindly ask you to confirm my interpretation is correct ?

Right. This allows you to both log to syslog and a file.

> Personally, I believe "it" would be the "ha_logger", and yes, wouldn't it be cool if the ha_logger could talk directly to a syslog daemon.

Definitely not. That would lose the advantages of decoupling our logging
from syslog or file blocking.

> > The point is to decouple the cluster stack from the possibly
> > blocking write() call, either to the syslog socket or the disk.
> So does ha_logd save over reboots messages it received from the logger
> but hasn't had time to process ?

The messages are kept in memory and flushed before shutdown, yes. So
there is no need to save anything over a reboot.

> > You should not set this manually. This is set in the
> > configuration and the RA should not mess with it.
> Well, say I write a RA that I believe should log to the cron log facility.
> How am I to do that while using the ocf-shellfuncs ?

RAs should follow the logging configuration of the HA subsystem. I think
that is a very good idea.

If you want to violate that or do anything special, you would probably
be calling logger directly.


Regards,
Lars

--
Teamlead Kernel, SuSE Labs, Research and Development
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

_______________________________________________________
Linux-HA-Dev: Linux-HA-Dev[at]lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/

Linux-HA dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.