Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ModPerl: ModPerl

Apache2::ModLogConfig help needed

 

 

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded


torsten.foertsch at gmx

Mar 2, 2011, 1:02 PM

Post #1 of 4 (600 views)
Permalink
Apache2::ModLogConfig help needed

Hi,

I am in the process of developing an interface to mod_log_config. It can do 2
things:

* make CustomLog call perl handlers
* work as a log drain or log file

Log Drain
=========

CustomLog "@perl: My::Handler" "%I%O"

Assuming that My::Handler is defined it will receive 4 parameters here, the
request object, the input bytes according to mod_logio (the %I format), the
output bytes according to mod_logio (%O) and a newline string. Other
information can of course be specified in the format string.

Make it call perl handlers
==========================

CustomLog log/file.log "%r %{My::Fmt}^ ..."

It allocates the ^ format character and then calls My::Fmt with the request
object as the only parameter. The <> format modifiers can be used to specify
if the initial request is meant or the final one.

My::Fmt is expected to return a string that is then inserted in place of the
format specification.


Now, I have found a bug in apache that has been fixed somewhere between 2.2.9
and 2.2.15. I have also figured out a workaround. But I expect it to lead to
segfaults when the server is restarted by means of SIGHUP or SIGUSR1 and in
the old config BufferedLogs were on while in the new one they are off.

The reason for this mail is that I ask for help to figure out which apache
version has fixed the bug. So, if you have a httpd between 2.2.9 and 2.2.15
could you please try out the module.

perl Makefile.PL
make test

If the test hangs in starting up the web server and the only lines in t/logs
error_log look similar to these:

$ cat t/logs/error_log
V=2002009.
V=2002009.
buggy.

where V=... is the apache version then the bug is present.

If you like you can then go to line 169 in ModLogConfig.xs and change the
2002009 to your version+1 and retry.

The current state can be downloaded under:

http://foertsch.name/Apache2-ModLogConfig-0.01.tar.gz


BTW, if you feel urge to comment the idea or the implementation please don't
hesitate.

Thanks,
Torsten Förtsch

--
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net


randolf at modperl

Mar 2, 2011, 10:43 PM

Post #2 of 4 (564 views)
Permalink
Re: Apache2::ModLogConfig help needed (feature suggestion) [In reply to]

> Hi,
>
> I am in the process of developing an interface to mod_log_config. It can do 2
> things:
>
> * make CustomLog call perl handlers
> * work as a log drain or log file
[sNip]
> BTW, if you feel urge to comment the idea or the implementation please don't
> hesitate.

What I think would be a very nice addition to that list of two
things is to be able to specify a customized filename for the log
files. I'm unsure if this is even possible though.

I currently pipe through "cronolog" to include the YEAR and MONTH
numbers in the filenames. If this could be handled without piping
out to another process, then it would reduce a lot of pipe process
overhead (for each VirtualHost there are two "cronolog" processes,
one for the CustomLog, and the other for the ErrorLog).

An example of the filenames for log files for this month are
"access.2011-03.log" and "errors.2011-03.log."

If this feature is possible, then this module could also help to
make ModPerl 2 a little bit more popular if other administrators are
interested in reducing the number of piped processes.

Randolf Richardson - randolf [at] inter-corporate
Inter-Corporate Computer & Network Services, Inc.
Vancouver, British Columbia, Canada
http://www.inter-corporate.com


torsten.foertsch at gmx

Mar 3, 2011, 12:42 AM

Post #3 of 4 (564 views)
Permalink
Re: Apache2::ModLogConfig help needed (feature suggestion) [In reply to]

On Thursday, March 03, 2011 07:43:54 Randolf Richardson wrote:
> > I am in the process of developing an interface to mod_log_config. It can
> > do 2
> >
> > things:
> >
> >
> > * make CustomLog call perl handlers
> > * work as a log drain or log file
>
> [sNip]
>
> > BTW, if you feel urge to comment the idea or the implementation please
> > don't hesitate.
>
> What I think would be a very nice addition to that list of two
> things is to be able to specify a customized filename for the log
> files. I'm unsure if this is even possible though.

Not taking into account the security implications this would be possible:

<Perl>
package My::LogWriter;
use Apache2::RequestRec ();
use Apache2::Const -compile=>'OK';

sub handler {
my ($r, @string)=@_;

my $filehandle=compute_filehandle_based_on_request_time($r->request_time);
print $filehandle join '', @string;

return Apache2::Const::OK;
}
</Perl>
PerlLoadModule Apache2::ModLogConfig
CustomLog "@perl: My::LogWriter" "format spec"

You even can use anonymous subs here. Thus, the handler can be made more
general:

CustomLog "@perl: sub {My::LogWriter::handler(q/%YYYY-%MM-%DD.log/, @_)}" \
"format spec"

> I currently pipe through "cronolog" to include the YEAR and MONTH
> numbers in the filenames. If this could be handled without piping
> out to another process, then it would reduce a lot of pipe process
> overhead (for each VirtualHost there are two "cronolog" processes,
> one for the CustomLog, and the other for the ErrorLog).
>
> An example of the filenames for log files for this month are
> "access.2011-03.log" and "errors.2011-03.log."
>
> If this feature is possible, then this module could also help to
> make ModPerl 2 a little bit more popular if other administrators are
> interested in reducing the number of piped processes.

I think it depends on the number of workers if this approach is really better.

Why don't you log an VHost ID as well and and use exactly 1 reader? Don't know
if cronolog can do that but generally you can use the additional ID to split
the log into parts for each VHost. That way the opening and file name guessing
is done in one place.

Anyway, the reason for me to implement this was to get hands on values that
are otherwise really hard to get. mod_logio measures the number of bytes that
come in an go out on the wire for each request. For the input it installs a
network level filter. Modperl implements only connection and request level
filters. But the output counting is done by the core output filter and
reported to mod_logio by means of an optional function. So, measuring these
values in perl is impossible.

Now with this module I can do (provided mod_perl, mod_log_config and mod_logio
are loaded)

<Perl>
package My::IO;
use Apache2::RequestRec ();
use Apache2::Const -compile=>'OK';

sub handler {
my ($r, $inbytes, $outbytes)=@_;

$r->pnotes->{inout}=[$inbytes, $outbytes];

return Apache2::Const::OK;
}
</Perl>
PerlLoadModule Apache2::ModLogConfig
CustomLog "@perl: My::IO" "%I%O"

A request pool cleanup handler for example would then be able to use the
values.

Torsten Förtsch

--
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net


torsten.foertsch at gmx

Mar 3, 2011, 3:02 AM

Post #4 of 4 (563 views)
Permalink
Re: Apache2::ModLogConfig help needed [In reply to]

On Wednesday, March 02, 2011 22:02:21 Torsten Förtsch wrote:
> Now, I have found a bug in apache that has been fixed somewhere between
> 2.2.9 and 2.2.15.

Well, it's probably not a bug per se. The behavior simply differs when
mod_log_config is compiled in compared to a shared module.

Torsten Förtsch

--
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.