Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Dev

fixes to mythburn.py

 

 

MythTV dev RSS feed   Index | Next | Previous | View Threaded


T.Brackertz at gmx

Sep 27, 2011, 7:08 AM

Post #1 of 11 (757 views)
Permalink
fixes to mythburn.py

Hi,

as I had trouble with bugs in mytharchive I fixed a couple of bugs.

Now I want to contribute the fixes but I'm not sure if I should open several tickets or only one ticket.

(It's the first time I get involved in the development of mythtv and I couldn't find an answer in the FAQs and I found many tickets with patches solving many bugs at once.)

Below there is a description of the changes.

So what's the next step to do?

Thanks, Stefan


-----------------------------------------------------------
Patches in mythburn.py
----------------------
----------------------

typos
-----
all irrelevant



Medium handling:
----------------
former issues:
- no possibility to cancel process
- one has to be fast enough feeding the device, otherwise the process cancels
- some devices need more time to close the tray. They get hard-resetted all the time. No chance to burn a disk, if it is not put in before the script is started
- if you use a disk of wrong type you don't get a chance to put in a correct one. The whole process cancels

changes in BurnDVDISO()


Integration of project-x
------------------------
until now:
After the run of project-x its logfile is parsed (in renameProjectXFiles()) to find out the filenames of the files produced by project-x and to associate them to the streams

issues:
- not very reliable: in many constellations (e.g. recoded .flv-videos) the logfile doesn't fit the expectations. Although there is a fallback mechanism in many cases the script crashes while parsing before reaching the fallback
- the format of the logfile seems to change often if there is a new version of project-x. The parser has to be renewd all the time. (the recent one works for project-x 0.91 but not for older ones)

solution:
- An 0.91 project-x learned a new command line option (-set ExternPanel.appendPidToFileName=1) which causes project-x to use filenames for the output containing the PIDs and SubPIDs in a predictable manner. This should work with upcoming versions, too.
- Therefore no more parsing is necessary.
- Nevertheless there is still a fallback mechanism as in some cases (e.g. recoded mggs) mytharchivehelper gives strange IDs which makes it impossible to find the wanted files by ID
- As the current parser relies on project-x 0.91 as well there is no regression by the fact that at least version 0.91 is necessary now. (The version dependency at the beginning of the sript claiming for project-x 0.90.4 was outdated anyway.)

changes in runProjectX(), renameProjectXFiles() is obsolete now


Divide-by-zero-error when requantising
--------------------------------------
see:
http://code.mythtv.org/trac/ticket/8598#comment:2

issue:
If M2VRequantiser fails for some reason (probably it is not installed) the script crashes with a missleading error message.

reason:
The exit-code of M2VRequantiser is evaluated too late: After then run the size of the produced file is evaluated which gives a divide-by-zero-error. The exit-code of M2VRequantiser is evaluated thereafter, which means never because the script crashes before.

solution:
change the order

changes in runM2VRequantiser


Misleading log-entry
--------------------
issue:
The log-entry produced by getStreamInformation() sometimes mentions the wrong file as the filename is hard-coded

solution:
use the appropriate variable

changes in getStreamInformation()


Inconsistent string-handling in the whole script
------------------------------------------------
issue:
many errors and crashs when sript is used with recordings having special characters in their title or videos with special chars in the filename.

reason:
One part of the strings is unicode (mainly coming from parsing the xml-files), another part is utf-8-encoded (mainly coming from database). They get mixed up and / or get printed to logfile or console unencoded or double-encoded. This results in many errors.

solution:
Consequently use unicode-strings:
- make the database-object use unicode-strings in getDatabaseConnection()
- encode strings to utf-8 before printing to logfile or console in write() and runCommand()
- remove all encode- and decode-statements in the rest of the script


Endless growing work-directory
------------------------------
issue:
In some rare cases in which many DVDs with many small and one big title are created the work-directory theoreticallly grows endless

reason:
Before a new file is written to disk the old file with the same name is deleted but the rest is not.
Imagine the following situation:
Create a DL-disc with 20 titles, all very small, only the last one having 8GB. You get 20 subdirs in the work-directory of which the "20"-directory is very big. All together they have 9GB.
Next step: The same as before but with only 19 titles. Result: Only the first 19 dirs of the last run get deleted, the big "20"-dir stays. But there is a new big "19"-dir. All together they have ~17GB
An so on. You end up with a work directory containing at least 160GB

solution:
delete everything in the "work"-dir at the beginning instead of incrementally deletion. Also shortens the code.

new deleteEverythingInFolder()-function and changes at some other places
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

--
NEU: FreePhone - 0ct/min Handyspartarif mit Geld-zurück-Garantie!
Jetzt informieren: http://www.gmx.net/de/go/freephone
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Sep 28, 2011, 5:35 AM

Post #2 of 11 (734 views)
Permalink
Re: fixes to mythburn.py [In reply to]

Hi: It's my impression that dev interest in MythArchive is almost
extinct. I suggest that you just open a ticket with a unified diff of
your changes and the comments you have already made. It gets tedious to
keep things updated if you don't use something like git, but there
haven't been many changes here recently, and at least interested users
will have access to what you have done.

Ignore this if you get any more conventional or 'official' replies.

John P
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


T.Brackertz at gmx

Oct 1, 2011, 3:17 PM

Post #3 of 11 (720 views)
Permalink
Re: fixes to mythburn.py [In reply to]

Hi,

so I opened one big new ticket for all the issues as you suggested:
http://code.mythtv.org/trac/ticket/10071

I hope there is someone to transfer the changes to git.
I think it's quite a pity if there were no one who cares for mytharchive. Due to the changes in the 0.25-branch mytharchivehelper completely stopped working. At the moment I'm trying to get it back to work. Maybe there is a chance to get a git-account myself which allows me to commit the changes if there is no one else around.

Stefan


-------- Original-Nachricht --------
> Datum: Wed, 28 Sep 2011 13:35:06 +0100
> Von: John Pilkington <J.Pilk [at] tesco>
> An: mythtv-dev [at] mythtv
> Betreff: Re: [mythtv] fixes to mythburn.py

> Hi: It's my impression that dev interest in MythArchive is almost
> extinct. I suggest that you just open a ticket with a unified diff of
> your changes and the comments you have already made. It gets tedious to
> keep things updated if you don't use something like git, but there
> haven't been many changes here recently, and at least interested users
> will have access to what you have done.
>
> Ignore this if you get any more conventional or 'official' replies.
>
> John P
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-dev

--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Oct 3, 2011, 3:03 PM

Post #4 of 11 (692 views)
Permalink
Re: fixes to mythburn.py [In reply to]

On 01/10/11 23:17, T.Brackertz [at] gmx wrote:

> Hi,
>
> so I opened one big new ticket for all the issues as you suggested:
> http://code.mythtv.org/trac/ticket/10071
>
> I hope there is someone to transfer the changes to git.
> I think it's quite a pity if there were no one who cares for
mytharchive. Due to the changes in the 0.25-branch mytharchivehelper
completely stopped working. At the moment I'm trying to get it back to
work. Maybe there is a chance to get a git-account myself which allows
me to commit the changes if there is no one else around.
>
> Stefan
>
Your patch was - as is recommended - against master and I'm running a
modified version of fixes. Applying the patch to the fixes distrib
version failed in chunks that involve mytharchivehelper. I should be
able to backport but I may not be quick - and most of your solutions
are to problems that I either haven't encountered or habitually avoid,
so production (and life) may have to take precedence. But when I want
to go to 0.25 it would be great to have a bug-free version waiting
complete with knobs and whistles. I'll try to check it out.

Cheers,

John P
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


T.Brackertz at gmx

Oct 4, 2011, 7:14 AM

Post #5 of 11 (678 views)
Permalink
Re: fixes to mythburn.py [In reply to]

I attached a path-file for the fixes-branch:
http://code.mythtv.org/trac/ticket/10071

Stefan



-------- Original-Nachricht --------
> Datum: Mon, 03 Oct 2011 23:03:00 +0100
> Von: John Pilkington <J.Pilk [at] tesco>
> An: mythtv-dev [at] mythtv
> Betreff: Re: [mythtv] fixes to mythburn.py

> On 01/10/11 23:17, T.Brackertz [at] gmx wrote:
>
> > Hi,
> >
> > so I opened one big new ticket for all the issues as you suggested:
> > http://code.mythtv.org/trac/ticket/10071
> >
> > I hope there is someone to transfer the changes to git.
> > I think it's quite a pity if there were no one who cares for
> mytharchive. Due to the changes in the 0.25-branch mytharchivehelper
> completely stopped working. At the moment I'm trying to get it back to
> work. Maybe there is a chance to get a git-account myself which allows
> me to commit the changes if there is no one else around.
> >
> > Stefan
> >
> Your patch was - as is recommended - against master and I'm running a
> modified version of fixes. Applying the patch to the fixes distrib
> version failed in chunks that involve mytharchivehelper. I should be
> able to backport but I may not be quick - and most of your solutions
> are to problems that I either haven't encountered or habitually avoid,
> so production (and life) may have to take precedence. But when I want
> to go to 0.25 it would be great to have a bug-free version waiting
> complete with knobs and whistles. I'll try to check it out.
>
> Cheers,
>
> John P
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-dev

--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Oct 5, 2011, 4:37 AM

Post #6 of 11 (666 views)
Permalink
Re: fixes to mythburn.py [In reply to]

On 04/10/11 15:14, T.Brackertz [at] gmx wrote:
>
>
> I attached a path-file for the fixes-branch:
> http://code.mythtv.org/trac/ticket/10071
>
> Stefan

Thanks for this. I applied it to my own patched version with only a few
chunks needing to be edited - but I haven't checked my old mods to see
if they really ought to go through your re-encoding process. But I hit
a snag when I tried to run it.

I run many of the processes in mythburn.py under 'ionice -c3' and pass
that prefix through the frontend MythArchive setup screens; in most
cases I haven't built this into the script because I have a CentOS5 box
that doesn't know about ionice, and maybe other distros would also
complain.

Your script then generates command lines beginning like "ionice c3 java
-jar projectx" which gets 'no such file..'. If I remove the double
quotes the command runs happily from the console. Is there a simple way
to fix this for all the commands, or should I build the prefix into the
script for every appropriate command?

John P
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Oct 5, 2011, 11:05 AM

Post #7 of 11 (664 views)
Permalink
Re: fixes to mythburn.py [In reply to]

On 05/10/11 12:37, John Pilkington wrote:
> On 04/10/11 15:14, T.Brackertz [at] gmx wrote:
>>
>>
>> I attached a path-file for the fixes-branch:
>> http://code.mythtv.org/trac/ticket/10071
>>
>> Stefan
>
> Thanks for this. I applied it to my own patched version with only a few
> chunks needing to be edited - but I haven't checked my old mods to see
> if they really ought to go through your re-encoding process. But I hit
> a snag when I tried to run it.
>
> I run many of the processes in mythburn.py under 'ionice -c3' and pass
> that prefix through the frontend MythArchive setup screens; in most
> cases I haven't built this into the script because I have a CentOS5 box
> that doesn't know about ionice, and maybe other distros would also
> complain.
>
> Your script then generates command lines beginning like "ionice c3 java
> -jar projectx" which gets 'no such file..'. If I remove the double
> quotes the command runs happily from the console. Is there a simple way
> to fix this for all the commands, or should I build the prefix into the
> script for every appropriate command?
>
> John P

Never mind: I did the editing and have successfully created an .iso
file. I'll try a few more complex ones and see how it goes.

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


t.brackertz at gmx

Oct 6, 2011, 4:18 PM

Post #8 of 11 (651 views)
Permalink
Re: fixes to mythburn.py [In reply to]

Yes for sure it does not work if you pass "ionice -c3 java -jar projectx"
through the backend-setup. But I run into trouble as some of my tools are
not in the PATH and I put them into mythtv-setup with full path which in
turn contains spaces. Maybe the command itself and its path shouldn't get
quoted by mythburn.py but by the user in mythtv-setup?
Nevertheless if your java is configured correctly running "projectx"
should work. Maybe we should alter mytharchive in a way that it runs
mythburn.py as a whole with ionice? Do you think that makes sense? I can't
imagine a situation in which ionice could be annoying. Of course
mytharchive should test if ionice is installed first.
I also planned to improve mythburn's use of multicores as meither projectx
nor the reencoding of audio to ac3 uses more than one core on my PC.

But at the moment I'm trying to fix some segfaults in mytharchivehelper.
In 0.25 there occur quite many of them. I think they also happen in 0.24
but are not caught and therefore stay unrecognized in most cases.

Stefan

Am 05.10.2011, 13:37 Uhr, schrieb John Pilkington <J.Pilk [at] tesco>:

> On 04/10/11 15:14, T.Brackertz [at] gmx wrote:
>>
>>
>> I attached a path-file for the fixes-branch:
>> http://code.mythtv.org/trac/ticket/10071
>>
>> Stefan
>
> Thanks for this. I applied it to my own patched version with only a few
> chunks needing to be edited - but I haven't checked my old mods to see
> if they really ought to go through your re-encoding process. But I hit
> a snag when I tried to run it.
>
> I run many of the processes in mythburn.py under 'ionice -c3' and pass
> that prefix through the frontend MythArchive setup screens; in most
> cases I haven't built this into the script because I have a CentOS5 box
> that doesn't know about ionice, and maybe other distros would also
> complain.
>
> Your script then generates command lines beginning like "ionice c3 java
> -jar projectx" which gets 'no such file..'. If I remove the double
> quotes the command runs happily from the console. Is there a simple way
> to fix this for all the commands, or should I build the prefix into the
> script for every appropriate command?
>
> John P
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-dev


_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Oct 7, 2011, 1:04 AM

Post #9 of 11 (644 views)
Permalink
Re: fixes to mythburn.py [In reply to]

On 07/10/11 00:18, Stefan Brackertz wrote:
> Yes for sure it does not work if you pass "ionice -c3 java -jar projectx"
> through the backend-setup. But I run into trouble as some of my tools are
> not in the PATH and I put them into mythtv-setup with full path which in
> turn contains spaces. Maybe the command itself and its path shouldn't get
> quoted by mythburn.py but by the user in mythtv-setup?
> Nevertheless if your java is configured correctly running "projectx"
> should work. Maybe we should alter mytharchive in a way that it runs
> mythburn.py as a whole with ionice? Do you think that makes sense?

Not if you want to actually burn the DVD from it. Anyway, I've now
fixed the calls for the tools that I use. See the ticket for a combined
patch.

I can't
> imagine a situation in which ionice could be annoying. Of course
> mytharchive should test if ionice is installed first.
> I also planned to improve mythburn's use of multicores as meither projectx
> nor the reencoding of audio to ac3 uses more than one core on my PC.
>
> But at the moment I'm trying to fix some segfaults in mytharchivehelper.
> In 0.25 there occur quite many of them. I think they also happen in 0.24
> but are not caught and therefore stay unrecognized in most cases.
>
> Stefan
>
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


t.brackertz at gmx

Oct 7, 2011, 5:41 PM

Post #10 of 11 (630 views)
Permalink
Re: fixes to mythburn.py [In reply to]

>> Maybe we should alter mytharchive in a way that it runs
>> mythburn.py as a whole with ionice? Do you think that makes sense?
>
> Not if you want to actually burn the DVD from it. Anyway, I've now
> fixed the calls for the tools that I use. See the ticket for a combined
> patch.

Hm, what's the problem? I think growisofs uses the
buffer-underrun-protection provided by all drives since 2004 by default.
Maybe one could test if the drive is able to protect underruns to be sure.

Stefan
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev


J.Pilk at tesco

Oct 8, 2011, 2:12 AM

Post #11 of 11 (624 views)
Permalink
Re: fixes to mythburn.py [In reply to]

On 08/10/11 01:41, Stefan Brackertz wrote:
>>> Maybe we should alter mytharchive in a way that it runs
>>> mythburn.py as a whole with ionice? Do you think that makes sense?
>>
>> Not if you want to actually burn the DVD from it. Anyway, I've now
>> fixed the calls for the tools that I use. See the ticket for a combined
>> patch.
>
> Hm, what's the problem? I think growisofs uses the
> buffer-underrun-protection provided by all drives since 2004 by default.
> Maybe one could test if the drive is able to protect underruns to be sure.
>
> Stefan

No doubt it ought to work, but I'd prefer not to rely on it unless
forced to do so. I burn-and-verify from iso using K3b, after a quick
check that the iso has no obvious flaws. Verifying does catch an
occasional failure.

John

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-dev

MythTV dev RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.