Gossamer Forum
Home : General : Perl Programming :

Log reading script (ftp,web,error....etc) ...

Quote Reply
Log reading script (ftp,web,error....etc) ...
Hi guys,

I've just had a few spare minutes so decided to write a script that will read the last 10 lines of your error/web/ftp or any other log you have read access to, and display them to your browser.

I have seen a few posts from people who were interested in something like this so as I had a spare moment I thought I'd do it (I wanted it for myself also)

Here it is:

Code:
#!/usr/bin/perl
Code:
use CGI qw(:standard);
Code:
&main();
Code:
sub main {

my $dir = "/PATH/TO/LOGS";
my $file = "web.log";
my $lastx = 10;

my ($i,$n) = 0;
Code:
open(FILE,"<$dir/$file") || &error("Couldn't open $dir/$file : $!");
@data=<FILE>;
close(FILE);
Code:
foreach (@data) {
$i++;
}
Code:
print header;
foreach (@data) {
$n++;
next if ($n <= ($i - $lastx));
print "$_< br >";
}

}
Code:
sub error {

my ($msg) = shift;

print header;
print $msg;
exit();

}
The red variables are the only parts that need changing.

Obviously $dir is the path to the logs directory - no trailing slash.
$file is the name of the log file and $lastx is the number of the newest entries you want to view. So 10 shows the last 10 entries, 20 the last 20 entries etc...

Anyway hope you find it useful.

Oh, you'll also need to change < br > to a proper br tag.

I hate opening huge log files so this is perfect. It took about 1-2 seconds to load the last 10 entries of my 2.5MB web log.



Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
that's awesome.
except most web servers that get any traffic have log files larger or close to the same size as the amount of RAM they have. log files in general are known for being enormous so you probably don't want to slurp them into an array.

also, you don't need to loop through it twice.

Code:
my $lines = 10;
my $file = '/path/to/file';
my @stack = ();

open(LF, $file) or die "Error opening $file : $!\n";
while (<LF>) {
shift @stack if ($stack[$lines - 1]);
push @stack, $_;
}
close(LF);

# now @stack contains the last $lines number of lines
print foreach @stack;
of course, if you use a real OS you don't need any of this;)
tail -<lines> file

-g


s/(\d{2})/chr($1)/ge + print if $_ = '8284703280698276687967';
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Take a look at the following tail implementation (in pure Perl, of course) that I wrote. This version is designed only for shell scripts and to NOT be run from the web, but it wouldn't be too tough to change it a bit - though the follow mode would be a little difficult to implement over the web.

http://jagerman.com/ptail.txt

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
In Reply To:
that's awesome.
No need for sarcasm, I was only trying to help those who don't have huge log filesFrown...and I am on a RAQ and it automatically gzips log files for storage so that is why it is 2.5MB at the moment Wink

I appreciate you showing an example of how you'd do it - I'm always willing to learn. Guess I shouldn't have bothered with my example...

We all have to learn somewhere.

Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
i wasn't being sarcastic. i was being genuine. it's always good to see people learning and trying.
trust me, i'm not subtle about my sarcasm.

-g


s/(\d{2})/chr($1)/ge + print if $_ = '8284703280698276687967';
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
OK, sorry for pre-judging.

I know what you mean about the 2 foreach loops but I couldn't think of another way of finding the total number of lines to minus the 10 from to allow the "next" statement to work.

But now I know from your example, however I will be paranoid when showing my code in future Frown

Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
that's up to you. if it makes you double-check your code before posting it then i think it's a good thing.

i consider this a very laid back board. if you want to see harsh or sarcastic go to the comp.lang.perl.misc news group. you won't know what hit ya:)

or just ask me and i'll be happy to oblige;)

all i'm saying is, expect criticism of your code. don't be offended or hurt. learn from it and be a better developer because of it.

-g


s/(\d{2})/chr($1)/ge + print if $_ = '8284703280698276687967';
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Or for one that works from the web, download the fileman you'll get by clicking on the Email link on this forum. It has the ability to tail large files.

Cheers,

Alex

--
Gossamer Threads Inc.
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
In Reply To:
trust me, i'm not subtle about my sarcasm.
/me mops up the floor that soaked with tears because I'm laughing so hard.

You guys don't know the G man in person, but trust me when I say that quote is right on. He ain't subtle in even the most loose definition of the word Smile

[FYI bro, San laughed pretty damn hard too]

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
ahh... how i remember the days of us ripping people new ones...

i don't think many people here have seen some of the classics. are any of the epic tag-team slams still up on ubb?

-g


s/(\d{2})/chr($1)/ge + print if $_ = '8284703280698276687967';
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Nah, I think most things have been pruned down. I'll look one of these days though

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
haha...I remember...hehe!

Actually, I appreciate your humor and GClemmons for that matter...helped me to venture out and learn Perl rather than leeching from the forums...hehe! Wink

Regards,

Eliot Lee
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
"Leeching" from the forum is a good thing in some cases, especially if it is the odd line of code here and there. It is helful to beginners or intermediates as they can see an example of how to apply the code and can then apply it into their own code, thus teaching them a new function.

Obviously copying big chunks is a bit stupid because that doesn't help you to learn.

Share and share alike.
Give to receive. (Like christmas).Wink

I can only think of one LEECHER, of whom we won't mention Wink achoo!! mycg********

Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Paul,

Good points...I should've qualified what I meant by "leeching"...what I meant was more than the simple matter of copying code snippets from the forums...I meant taking ideas without giving credit where it's due and also expecting others to post whole scripts in the forums...that is my definition of "leeching"...

Sorry for the confusion...

Edit: Sorry, Paul...I should stop posting comments like these...this is your Thread afterall and I am sure that you were expecting helpful advice or comments about the script you are writing....

Regards,

Eliot Lee
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Sorry?

What gave you that idea....I don't care who makes posts in my threads. Anyone and everyone is welcome as long as they are not unwelcome Smile


Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
i use..

#!/usr/bin/perl

$log = "/etc/httpd/logs/my.error_log";
$tail = $ENV{'QUERY_STRING'} || 20;
$tail =~ s/\D//;

print "Content-Type: text/plain\n\n";
print "Last $tail Errors:\n";
open (ERROR, "tail -$tail $log | tail -$tail |") || die $!;
while (<ERROR>) { print; }
close ERROR;

by typing http://www.asdf.com/error-log.cgi?50
you can get the last 50 errors

Jerry Su
http://www.jsu07.com
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Cool.

What is the point of the

$tail =~ s/\D//;

bit?

...I mean I know what it does but why do you need it?

Also please could you explain this line:

open (ERROR, "tail -$tail $log | tail -$tail |") || die $!;

So it tails the number of lines in the query string or 20 if there is no query string but what does the red bit do?


Installations:http://www.wiredon.net/gt/
Favicon:http://www.wiredon.net/favicon/

Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
And by typing http://www.asdf.com/error-log.cgi?1000000<</etc/passwd you can get the server's /etc/passwd file.

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
In Reply To:
What is the point of the

$tail =~ s/\D//;
As I demonstrated in my other post, it's an incredibly huge security hole. It should be $tail =~ s/\D//g; or $tail =~ tr/0-9//cd; so that it removes all non-numeric characters, and then the next line should be $tail ||= 20; As shown, without the /g you can do just about anything you want on the system (as the web user) which usually isn't very good.

The red bit is how you can open a command to a filehandle - all the standard output (STDOUT) of the command will be sent to the filehandle (ERROR, in this case). You can also "open" a command with a leading | to indicate that you want anything printed to that filehandle to be provided as STDIN to the program.

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Jason,

Don't know if you know this or not, but your web site is inaccessible via Netscape....hehe!

Regards,

Eliot Lee
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
Bloody netscape... Should be fixed now. I, uh, accidentally left off a </table> tag =)

Worked fine in IE and Mozilla though...

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
I won't re-iterate the importance of testing in Netscape....

If you're interested, read the following Thread:

http://www.gossamer-threads.com/...w=collapsed&sb=5

Regards,

Eliot Lee
Quote Reply
Re: Log reading script (ftp,web,error....etc) ... In reply to
I read that, but given the content of my site I wasn't too concerned ;-)

I agree with you about the netscape testing - I do test my templates with Netscape.

Jason Rhinelander
Gossamer Threads
jason@gossamer-threads.com