Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

automating commercial removal

 

 

MythTV users RSS feed   Index | Next | Previous | View Threaded


tmetro+mythtv-users at gmail

Sep 21, 2007, 1:53 PM

Post #1 of 7 (2019 views)
Permalink
automating commercial removal

In reading the instructions on the wiki on commercial removal:
http://www.mythtv.org/wiki/index.php/Removing_Commercials

I see that in order to remove commercial automatically requires setting
up a script:

http://www.mythtv.org/wiki/index.php/Script_-_RemoveCommercials

As I understand it, the real reason a script is required for this
operation is because the cutlist generated by mythcommflag is not
honored by a transcode job normally launched from the UI.

Is there a good reason why the commflag cutlist isn't copied over to the
main cutlist by default? Or at least have an option to do this.

I realize that most users don't bother with commercial removal, instead
opting for the safer option of simply automatically skipping over them
on playback, but in the event you do want to transcode a show, I would
think that most of the time you'd want the commflag cutlist.

I'm wondering if the RemoveCommercials script could be paired down to a
much smaller script that only handles the copying of the cutlist
(mythcommflag --gencutlist). If so, and jobs run in the right order, it
should be possible to set that up as a universal user job, and then use
the regular facilities for commflagging and transcoding. But maybe
ordering is the problem here...it would need to run after mythcommflag
and before mythtranscode.

-Tom
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


Braindead at diablops

Sep 21, 2007, 1:59 PM

Post #2 of 7 (1974 views)
Permalink
Re: automating commercial removal [In reply to]

On Fri, 21 Sep 2007 16:53:05 -0400
Tom Metro <tmetro+mythtv-users [at] gmail> wrote:

> In reading the instructions on the wiki on commercial removal:
> http://www.mythtv.org/wiki/index.php/Removing_Commercials
>
<snip>
> it should be possible to set that up as a universal user job, and
> then use the regular facilities for commflagging and transcoding. But
> maybe ordering is the problem here...it would need to run after
> mythcommflag and before mythtranscode.

Simply setup a userjob with this:

mythcommflag --chanid %CHANID% --starttime %STARTTIME% --gencutlist

And of course a nice description. That'll load the commflags into the cutlist.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


tmetro+mythtv-users at gmail

Sep 22, 2007, 8:34 AM

Post #3 of 7 (1958 views)
Permalink
Re: automating commercial removal [In reply to]

Braindead wrote:
> Simply setup a userjob with this:
> mythcommflag --chanid %CHANID% --starttime %STARTTIME% --gencutlist

Yeah, that's what I had in mind...providing the user job runs before
transcoding. Elsewhere I saw mention of an option to causing transcoding
to be delayed. If such an option exists, that could address the
potential ordering problem.

But what would be really nice is to simply be able to specify the
--gencutlist option on the command line in the back-end "Commercial
Flagger command" setting. Unfortunately there are a few problems with
that. One is that although not clearly documented, from the source it
seems that specifying --gencutlist causes mythcommflag to copy the
cutlist and exit, so you can't do flagging and copying during the same
run. The other problem is that it isn't clear whether you can specify
command line options in that setting.

Feels like there should be an --alsocopycutlist option, or something
like that.

-Tom

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


cpinkham at bc2va

Sep 22, 2007, 9:54 AM

Post #4 of 7 (1961 views)
Permalink
Re: automating commercial removal [In reply to]

* On Sat Sep 22, 2007 at 11:34:59AM -0400, Tom Metro wrote:
> Braindead wrote:
> > Simply setup a userjob with this:
> > mythcommflag --chanid %CHANID% --starttime %STARTTIME% --gencutlist
>
> Yeah, that's what I had in mind...providing the user job runs before
> transcoding. Elsewhere I saw mention of an option to causing transcoding
> to be delayed. If such an option exists, that could address the
> potential ordering problem.

If you are confident enough in the commercial flagger to always copy the
commercial skip list to the cut list automatically before transcoding,
then here is how you can accomplish that with existing Myth functionality.

1) Make sure Myth is setup to run commercial flagging before transcoding.

2) Make a shell script containing the following (untested) code:

#!/bin/sh
mythcommflag --chanid $1 --starttime $2 -V $3
RESULT=$?

mythcommflag --chanid $1 --starttime $2 -V $3 --gencutlist
exit $?

As an example I'll say that you called this script
/usr/local/bin/mythcommflag_wrapper.sh

3) Go into mythtv-setup and change the "Commercial Flagger command" to be
the following:

/usr/local/bin/mythcommflag_wrapper.sh %CHANID% %STARTTIME% %VERBOSELEVEL%

That will run the wrapper script giving it the chanid, starttime, and
the verbose level that mythbackend or mythjobqueue was run with.
The script should flag the recording as normal, then copy the generated
commecial skip list to the cutlist automatically.

The "Commercial Flagger command" and "Transcoder command" settings are
configured identical to the "User Job #X Command" settings. You have
to specify all options that the called program needs to execute, you
can't just put in "mythcommflag_wrapper.sh" and expect things to work.
I only mention this because some have tried it in the past, expecting
the JobQueue to pass the appropriate options to the called program,
but the JobQueue has know way of knowing what the appropriate options
are so the only thing it does it the mornal %VALUE% substitutions and
then runs the command exactly as given in mythtv-setup.

I think you can even put multiple commands in the "Commercial Flagger
command" setting as long as they are separated by a semi-colon. So you
might want to try the following instead of making a wrapper shell script
(this may wrap in your email program so make sure you get it all):

mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% ; mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% --gencutlist

The JobQueue should replace all instances of the variables in the command,
so %CHANID% should be replaced both times it occurs, etc..

> Feels like there should be an --alsocopycutlist option, or something
> like that.

Since you'd have to modify the "Commercial Flagger command" anyway to add
this command line option and it's so easy to do this anyway by calling
mythcommflag twice, I don't see a big need to add another option.

All that being said, even I don't even trust mythcommflag enough to do
this and transcode automatically and I'm the guy that wrote most of it,
but it's up to you what you want to do. :)

--
Chris Pinkham
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


tmetro+mythtv-users at gmail

Sep 25, 2007, 6:24 PM

Post #5 of 7 (1919 views)
Permalink
Re: automating commercial removal [In reply to]

Chris Pinkham wrote:
> If you are confident enough in the commercial flagger to always copy
> the commercial skip list to the cut list automatically before
> transcoding...

I'm not. As I mentioned in another thread, the flagger routinely messes
up on the last commercial break and includes the last segment of the
show as part of the break. (I understand why this happens. The last
segment is brief, and probably has scene changes of a frequency similar
to commercials.)

But I'm also not transcoding universally. I was planning to only
transcode certain "low value" shows. And shows in which the last segment
only contains credits. (Although I'm considering a hack of having a user
job that deleted the last cut from the cut list.)

Assuming most people don't universally transcode shows, I'd think
copying the flag list to the cut list by default would be fairly
harmless. If nothing else, it saves a keystroke when you go to manually
tweak the cut list before running a manual transcode.


> #!/bin/sh
> mythcommflag --chanid $1 --starttime $2 -V $3
> RESULT=$?
>
> mythcommflag --chanid $1 --starttime $2 -V $3 --gencutlist
> exit $?

What's the purpose of "RESULT=$?" if RESULT is never used?


> I think you can even put multiple commands in the "Commercial Flagger
> command" setting as long as they are separated by a semi-colon. So you
> might want to try the following instead of making a wrapper shell script:
>
> mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% ; mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% --gencutlist

I'd substitute && for ;, but otherwise this is the approach I'd try first.

Running mythcommflag --gencutlist ... as a user job seemed to work fine,
but this is something I'd like to apply universally.


>> Feels like there should be an --alsocopycutlist option, or something
>> like that.
>
> Since you'd have to modify the "Commercial Flagger command" anyway to add
> this command line option and it's so easy to do this anyway by calling
> mythcommflag twice, I don't see a big need to add another option.

True. I guess my ideal would be flipping the behavior and adding a
switch to *not* copy the cut list. Copying then becomes the default, and
for those users that universally transcode, they can add the option to
disable it.

-Tom

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


cpinkham at bc2va

Sep 25, 2007, 7:59 PM

Post #6 of 7 (1898 views)
Permalink
Re: automating commercial removal [In reply to]

* On Tue Sep 25, 2007 at 09:24:27PM -0400, Tom Metro wrote:
> > #!/bin/sh
> > mythcommflag --chanid $1 --starttime $2 -V $3
> > RESULT=$?
> >
> > mythcommflag --chanid $1 --starttime $2 -V $3 --gencutlist
> > exit $?
>
> What's the purpose of "RESULT=$?" if RESULT is never used?

I thought I said that was untested but maybe I forgot to. :)
I meant to "exit $RESULT" so that the JobQueue gets the result of
the first flagger command not the second.

> > mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% ; mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% --gencutlist
>
> I'd substitute && for ;

Agreed but not sure if it will work. mythcommflag returns the number of
commercial breaks detected as an error code, so && may not work.

Errors are negative, 0 is good, and positive results are number of
breaks flagged.

> True. I guess my ideal would be flipping the behavior and adding a
> switch to *not* copy the cut list. Copying then becomes the default, and
> for those users that universally transcode, they can add the option to
> disable it.

A lot of users don't want to auto-skip though, and copying the
skip list to the cutlist would make everyone autoskip.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


craig at goranson

Sep 25, 2007, 9:55 PM

Post #7 of 7 (1898 views)
Permalink
Re: automating commercial removal [In reply to]

I have been running a script very similar to the one that Chris posted.
Like others here I have experiences the last "blurp" of the show getting cut
due to the way that USA and other channels use credits etc at the end of
their shows. I like autotranscoding shows, as I tend to record a lot and
then watch them over mythweb while not at home. So its nice to have the
commercials removed for me. So I have my system setup to automatically cut
out commercials on select programs.

I have modified the entry for the commercial flagger job to run a custom
shell script that handles the normal flagging of the commercials, then
copies the skiplist to the cutlist. For "low value" programs that I don't
care if there is 100% accuracy on the skip list I just schedule the
autotranscode job to run, and have it honor the skip list. For programs
that are high value I don't run the auto transcode, so I can always edit the
skip list for accuracy later. Main difference between mine and Chris's
script is that I have some primitive shell code that strips the last
commercial break from the skip list prior to copying the skiplist to the
cutlist. If there are two or more commercial skips, then I have it ignore
the last skip. If there is one skip, I just ignore the one skip.
Eventually I want to rewrite this in Perl and make it more intelligent and
have it pull out commercial skips that are very short, as those are likely
incorrectly flagged parts in shows. Also would be cool to put together
something for mythweb where you could "preview" just the commercial cuts
(small video clips), then approve the commercial cuts before transcoding.
But don't have that kind of time on my hands right now to tackle something
like that.

When scheduling a job I want to automatically remove commercials, I just
check the "autotranscode" box in the scheduling. Since the commercial
flagging job runs when the program starts recording, it will have completed
prior to the autotranscoding starting, which only starts after the program
is done recording. For programs I don't want to automatically remove
commercials, I just don't check the autotranscode on the schedule. (if
there was a program where you absolutely didn't want a cutlist applied, just
setup a job with "mythcommflag -f -c %CHANID% -s %STARTTIME% --clearcutlist"
and have that job run on that program)


I configured the commflag command in mythtv to be:
mythremovecomm.sh -j %JOBID% -V %VERBOSELEVEL%

Then in the mythremovecomm.sh file I have:
----------------------cut---------------------------
#!/bin/sh
mythcommflag $1 $2 $3 $4
EXITCODE=$?

#####################################################
# Gets Skip list,
####################################################
list1=`mythcommflag --getskiplist $1 $2 |sed -n 's/Commercial Skip List:
//p'`
count=`echo $list1 | grep -c ","`

############################################################
#if 1 or more z'comma" found then there are least two commercials
#less then one comma means one or zero commercial skip
###########################################################
if [ $count -gt 0 ]; then
# strip last commercial from list
list=`echo $list1 | sed -re 's/(.+),.+$/\1/'`
else
exit $EXITCODE
fi
mythcommflag $1 $2 $3 $4 --setcutlist $list
exit $EXITCODE
------------------------------------cut---------------------------

Hope that proves useful, it seems to have been working well for me. But
haven't done any kind of detailed testing.

Craig

----- Original Message -----
From: "Tom Metro" <tmetro+mythtv-users [at] gmail>
To: "Discussion about mythtv" <mythtv-users [at] mythtv>
Sent: Tuesday, September 25, 2007 6:24 PM
Subject: Re: [mythtv-users] automating commercial removal


Chris Pinkham wrote:
> If you are confident enough in the commercial flagger to always copy
> the commercial skip list to the cut list automatically before
> transcoding...

I'm not. As I mentioned in another thread, the flagger routinely messes
up on the last commercial break and includes the last segment of the
show as part of the break. (I understand why this happens. The last
segment is brief, and probably has scene changes of a frequency similar
to commercials.)

But I'm also not transcoding universally. I was planning to only
transcode certain "low value" shows. And shows in which the last segment
only contains credits. (Although I'm considering a hack of having a user
job that deleted the last cut from the cut list.)

Assuming most people don't universally transcode shows, I'd think
copying the flag list to the cut list by default would be fairly
harmless. If nothing else, it saves a keystroke when you go to manually
tweak the cut list before running a manual transcode.


> #!/bin/sh
> mythcommflag --chanid $1 --starttime $2 -V $3
> RESULT=$?
>
> mythcommflag --chanid $1 --starttime $2 -V $3 --gencutlist
> exit $?

What's the purpose of "RESULT=$?" if RESULT is never used?


> I think you can even put multiple commands in the "Commercial Flagger
> command" setting as long as they are separated by a semi-colon. So you
> might want to try the following instead of making a wrapper shell script:
>
> mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% ;
> mythcommflag -c %CHANID% -s %STARTTIME% -V %VERBOSELEVEL% --gencutlist

I'd substitute && for ;, but otherwise this is the approach I'd try first.

Running mythcommflag --gencutlist ... as a user job seemed to work fine,
but this is something I'd like to apply universally.


>> Feels like there should be an --alsocopycutlist option, or something
>> like that.
>
> Since you'd have to modify the "Commercial Flagger command" anyway to add
> this command line option and it's so easy to do this anyway by calling
> mythcommflag twice, I don't see a big need to add another option.

True. I guess my ideal would be flipping the behavior and adding a
switch to *not* copy the cut list. Copying then becomes the default, and
for those users that universally transcode, they can add the option to
disable it.

-Tom

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.