Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

trunk gentoo ebuilds and logrotate

 

 

MythTV users RSS feed   Index | Next | Previous | View Threaded


freedenizen at gmail

Sep 21, 2011, 6:10 PM

Post #1 of 14 (1084 views)
Permalink
trunk gentoo ebuilds and logrotate

Hi All,
So I know the gentoo ebuilds for trunk haven't been updated for a
while, and maybe this will get corrected at some point, but in case
someone has already done this I'll ask before going through the work
myself.

Since the change to how logging is handled the /var/log/mythtv on my
BE now contains many hundreds of log files for mythbackend,
mythcommflag, mythpreviewgen, etc. It looks like there aren't any
updated logrotate files floating around anywhere that I have found.
Has anyone modified some to handle this?
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


keemllib at gmail

Sep 21, 2011, 7:05 PM

Post #2 of 14 (1060 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Wed, 2011-09-21 at 18:10 -0700, freedenizen wrote:
...
> Since the change to how logging is handled the /var/log/mythtv on my
> BE now contains many hundreds of log files for mythbackend,
> mythcommflag, mythpreviewgen, etc. It looks like there aren't any
> updated logrotate files floating around anywhere that I have found.
> Has anyone modified some to handle this?
...

Hi;

Does this help?

http://www.mythtv.org/wiki?title=Logging&curid=7289&diff=53235&oldid=53212#Log_file_cleanup

The -delete expression will certainly clean up your logging directory,
and remember that the find command can take the

-exec YourCustomArchive.sh {} \;

expression and you can do whatever you want with the
programname.data.pid.log files.

Then you could use existing logrotate configuration files to
clean up the archive files.

Bill

* Your script could be as simple as:

# Convert the program.date.pid.log into program.log
ARCHIVE=$(echo $1 | sed -n 's/\.20............\.[0-9]*//p')
cat $1 >> $ARCHIVE
rm $1


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


diespambox at gmx

Sep 22, 2011, 1:14 AM

Post #3 of 14 (1061 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

freedenizen <freedenizen [at] gmail> wrote:
> Hi All,
> So I know the gentoo ebuilds for trunk haven't been updated for a
> while, and maybe this will get corrected at some point, but in case

there is a script (mythtv-buildebuild.py), which you can use to create
up to date ebuilds:

https://github.com/MythTV/packaging/tree/master/Gentoo/scripts

works fine for me.

Tobi
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


freedenizen at gmail

Sep 22, 2011, 8:29 AM

Post #4 of 14 (1048 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Wed, Sep 21, 2011 at 7:05 PM, Bill Meek <keemllib [at] gmail> wrote:
> Hi;
>
> Does this help?
>
> http://www.mythtv.org/wiki?title=Logging&curid=7289&diff=53235&oldid=53212#Log_file_cleanup
>
> The -delete expression will certainly clean up your logging directory,
> and remember that the find command can take the
>
>  -exec YourCustomArchive.sh {} \;
>
> expression and you can do whatever you want with the
> programname.data.pid.log files.
>
> Then you could use existing logrotate configuration files to
> clean up the archive files.
>
> Bill
>
> * Your script could be as simple as:
>
>  # Convert the program.date.pid.log into program.log
>  ARCHIVE=$(echo $1 | sed -n 's/\.20............\.[0-9]*//p')
>  cat $1 >> $ARCHIVE
>  rm $1

Ah ya, I actually had created a cronjob:
find /var/log/mythtv -mtime +7 -daystart -execdir rm '{}' \; 2>/dev/null

not elegant but it kept the files down.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


keemllib at gmail

Sep 22, 2011, 3:38 PM

Post #5 of 14 (1046 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Thu, 2011-09-22 at 08:29 -0700, freedenizen wrote:
> On Wed, Sep 21, 2011 at 7:05 PM, Bill Meek <keemllib [at] gmail> wrote:
...
> > Does this help?
> >
Hi;

As noted on IRC last night, my link was poor and should have been:

http://www.mythtv.org/wiki/Logging#Log_file_cleanup

...
> Ah ya, I actually had created a cronjob:
> find /var/log/mythtv -mtime +7 -daystart -execdir rm '{}' \; 2>/dev/null
...

No offense intended. I read the "many hundreds of log files" part and
assumed you hadn't used that solution.

In any event,

The 'find' and '-exec' stuff I added are gone and replaced (by wagnerrp)
with his python script (line 79 needs another trailing ')'.)

os.unlink(os.path.join(self.path, self.filename)

The script expects only the new MythTV log file name formats as in:

application.date.pid.log{.#}{.gz}

and it aborts on file names like mythfrontend.log or my local files
(like LocalShutdown.log etc.)

I'm still working with it, but my python skills are fairly slim.

Bill


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


freedenizen at gmail

Sep 22, 2011, 3:51 PM

Post #6 of 14 (1038 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Thu, Sep 22, 2011 at 3:38 PM, Bill Meek <keemllib [at] gmail> wrote:
<snip>
>> Ah ya, I actually had created a cronjob:
>> find /var/log/mythtv -mtime +7 -daystart -execdir rm '{}' \; 2>/dev/null
> ...
>
> No offense intended. I read the "many hundreds of log files" part and
> assumed you hadn't used that solution.

Actually when I posted I hadn't, but then just decided to do that in
the mean time cause it was a bit of a pain trying to locate the
correct log file.

> In any event,
>
> The 'find' and '-exec' stuff I added are gone and replaced (by wagnerrp)
> with his python script (line 79 needs another trailing ')'.)
>
>  os.unlink(os.path.join(self.path, self.filename)
>
> The script expects only the new MythTV log file name formats as in:
>
>  application.date.pid.log{.#}{.gz}
>
> and it aborts on file names like mythfrontend.log or my local files
> (like LocalShutdown.log etc.)
>
> I'm still working with it, but my python skills are fairly slim.
>
> Bill

Ya, I saw the commit, hadn't tried it out yet though, thanks for the
tips and assistance.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


raymond at wagnerrp

Sep 22, 2011, 5:13 PM

Post #7 of 14 (1039 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On 9/22/2011 18:38, Bill Meek wrote:
> The 'find' and '-exec' stuff I added are gone and replaced (by wagnerrp)
> with his python script (line 79 needs another trailing ')'.)
>
> os.unlink(os.path.join(self.path, self.filename)

Whoops. I tested the logic with the print statement commented out
directly above. I admit, I never actually ran the script completely,
allowing it to delete.

> The script expects only the new MythTV log file name formats as in:
>
> application.date.pid.log{.#}{.gz}
>
> and it aborts on file names like mythfrontend.log or my local files
> (like LocalShutdown.log etc.)

There is now a regular expression filter that it simply ignores those
files that don't match the expected pattern.

As for the ebuilds, I'm putting together an updated version tonight that
will optionally set up log rotation, and a cronjob to run this script.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


keemllib at gmail

Sep 22, 2011, 5:32 PM

Post #8 of 14 (1033 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Thu, 2011-09-22 at 20:13 -0400, Raymond Wagner wrote:
...

Thank you. v0.25pre-3358-gcd002d2 logcleanup.py works fine
now. My glob.glob(...) solution was /almost/ posted too );

Bill


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


gravityhammer at gmail

Apr 17, 2012, 5:31 PM

Post #9 of 14 (885 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Thu, Sep 22, 2011 at 8:32 PM, Bill Meek <keemllib [at] gmail> wrote:
> On Thu, 2011-09-22 at 20:13 -0400, Raymond Wagner wrote:
> ...
>
> Thank you. v0.25pre-3358-gcd002d2 logcleanup.py works fine
> now. My glob.glob(...) solution was /almost/ posted too );
>
> Bill
>

Sorry to dredge up an old topic, but I'm trying to get my logcleanup
script working. If I run it from the command line, I get:

# python /usr/share/mythtv/bin/logcleanup.py
Traceback (most recent call last):
File "/usr/share/mythtv/bin/logcleanup.py", line 133, in <module>
main(opts)
File "/usr/share/mythtv/bin/logcleanup.py", line 95, in main
ls = sorted(LogFile.filter(opts.logpath, os.listdir(opts.logpath)))
File "/usr/share/mythtv/bin/logcleanup.py", line 35, in filter
return [cls(path, f) for f in filelist if cls._re.match(f)]
File "/usr/share/mythtv/bin/logcleanup.py", line 48, in __init__
self.sequence = int(m.group('sequence'))
ValueError: invalid literal for int() with base 10: '.1'

Is there anything that describes the parameters?
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


keemllib at gmail

Apr 17, 2012, 8:16 PM

Post #10 of 14 (882 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On 04/17/2012 07:31 PM, Phil Bridges wrote:
...
> Sorry to dredge up an old topic, but I'm trying to get my logcleanup
> script working. If I run it from the command line, I get:
>
> # python /usr/share/mythtv/bin/logcleanup.py
> Traceback (most recent call last):
> File "/usr/share/mythtv/bin/logcleanup.py", line 133, in<module>
> main(opts)
> File "/usr/share/mythtv/bin/logcleanup.py", line 95, in main
> ls = sorted(LogFile.filter(opts.logpath, os.listdir(opts.logpath)))
> File "/usr/share/mythtv/bin/logcleanup.py", line 35, in filter
> return [cls(path, f) for f in filelist if cls._re.match(f)]
> File "/usr/share/mythtv/bin/logcleanup.py", line 48, in __init__
> self.sequence = int(m.group('sequence'))
> ValueError: invalid literal for int() with base 10: '.1'

Hi;

I never typed: python .../logcleanup.py, just the command itself.
look at the 1st line in the file, that tells it to use python.

> Is there anything that describes the parameters?

logcleanup.py -h, if that's what you mean. To see the default
values, look at the last 10 lines of the file.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


gravityhammer at gmail

Apr 17, 2012, 8:59 PM

Post #11 of 14 (884 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Tue, Apr 17, 2012 at 11:16 PM, Bill Meek <keemllib [at] gmail> wrote:
> On 04/17/2012 07:31 PM, Phil Bridges wrote:
> ...
>
>> Sorry to dredge up an old topic, but I'm trying to get my logcleanup
>> script working.  If I run it from the command line, I get:
>>
>>  # python /usr/share/mythtv/bin/logcleanup.py
>> Traceback (most recent call last):
>>   File "/usr/share/mythtv/bin/logcleanup.py", line 133, in<module>
>>     main(opts)
>>   File "/usr/share/mythtv/bin/logcleanup.py", line 95, in main
>>     ls = sorted(LogFile.filter(opts.logpath, os.listdir(opts.logpath)))
>>   File "/usr/share/mythtv/bin/logcleanup.py", line 35, in filter
>>     return [cls(path, f) for f in filelist if cls._re.match(f)]
>>   File "/usr/share/mythtv/bin/logcleanup.py", line 48, in __init__
>>     self.sequence = int(m.group('sequence'))
>> ValueError: invalid literal for int() with base 10: '.1'
>
>
> Hi;
>
> I never typed: python .../logcleanup.py, just the command itself.
> look at the 1st line in the file, that tells it to use python.
>
>
>> Is there anything that describes the parameters?
>
>
> logcleanup.py -h, if that's what you mean. To see the default
> values, look at the last 10 lines of the file.
>

Thanks for the response. Going to the directory and executing it
gives the same error, even if I use the parameters that have been
dumped into my daily cronjob.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


keemllib at gmail

Apr 17, 2012, 9:38 PM

Post #12 of 14 (882 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On 04/17/2012 10:59 PM, Phil Bridges wrote:
...
>
> Thanks for the response. Going to the directory and executing it
> gives the same error, even if I use the parameters that have been
> dumped into my daily cronjob.

1st, I'm assuming you're on v0.25 (obvious, but gotta mention it.)

I just downloaded a copy from:

http://code.mythtv.org/trac/browser/mythtv/mythtv/contrib/maintenance/logcleanup.py?rev=cd002d2

and it runs A-OK. I'd suggest downloading it (use the plain text
version download.) There are differences compared to the Traceback
you provided in your 1st post.

--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


gravityhammer at gmail

Apr 18, 2012, 6:09 AM

Post #13 of 14 (873 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On Wed, Apr 18, 2012 at 12:38 AM, Bill Meek <keemllib [at] gmail> wrote:
> On 04/17/2012 10:59 PM, Phil Bridges wrote:
> ...
>>
>>
>> Thanks for the response.  Going to the directory and executing it
>> gives the same error, even if I use the parameters that have been
>> dumped into my daily cronjob.
>
>
> 1st, I'm assuming you're on v0.25 (obvious, but gotta mention it.)
>
> I just downloaded a copy from:
>
>  http://code.mythtv.org/trac/browser/mythtv/mythtv/contrib/maintenance/logcleanup.py?rev=cd002d2
>
> and it runs A-OK. I'd suggest downloading it (use the plain text
> version download.) There are differences compared to the Traceback
> you provided in your 1st post.
>
>

Thanks. I'll give that a shot tonight.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


raymond at wagnerrp

Apr 18, 2012, 6:32 AM

Post #14 of 14 (865 views)
Permalink
Re: trunk gentoo ebuilds and logrotate [In reply to]

On 4/18/2012 00:38, Bill Meek wrote:
> On 04/17/2012 10:59 PM, Phil Bridges wrote:
> ...
>>
>> Thanks for the response. Going to the directory and executing it
>> gives the same error, even if I use the parameters that have been
>> dumped into my daily cronjob.
>
> 1st, I'm assuming you're on v0.25 (obvious, but gotta mention it.)
>
> I just downloaded a copy from:
>
>
> http://code.mythtv.org/trac/browser/mythtv/mythtv/contrib/maintenance/logcleanup.py?rev=cd002d2
>
> and it runs A-OK. I'd suggest downloading it (use the plain text
> version download.) There are differences compared to the Traceback
> you provided in your 1st post.
>

Seems the same problem exists in both versions, and is a slight flaw in
the regular expression. The rotated sequence should not include the
preceding period.

_re =
re.compile("(?P<appname>[a-z]*(\.[a-z]+)?).(?P<date>[0-9]{14}).(?P<pid>[0-9]{1,6}).log((?P<sequence>.[0-9]+)\.(?P<compression>[a-zA-Z0-9]+)?)?")
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.