Mailing List Archive: MythTV: Dev

New MPEG2 commercial-cut code ready for testing

Index | Next | Previous | View Threaded

mythtv0368 at phracturedblue

Nov 12, 2005, 6:41 PM

Post #1 of 106 (4899 views)
Permalink

New MPEG2 commercial-cut code ready for testing

Well, after a lot of work rewriting the code from c to c++, and time
spent figuring out how to do commercial cutting, I have the first
version available.

In theory, thei si the same app that I've been positing here, but
migrated to C++/Qt (no mythtv hookup yet), and with the added
commerical-cut code.

while it isn't yet ready to go into myth, the known work isn't too
difficult. The big question is the unkown. (A) while I've tested the
C->C++ migration code reasonably well, there are significant
differences, and more testing is needed. (B) I am not yet convinced
that the cutlist processing method I'm using will actually work, and
it certainly has bugs that need working out.

I had to disable the PTS discontinuity code because it won't work with
the commercial cutting code. That'll definitely need to be fixed at
some point. The commercial cuttng code hasn't seen very much testing
yet, mostly because it requires actually watching the video to see if
it works, and I haven't has access to an A/V output device much
lately.

So here we are:
http://www.pblue.org/myth/mpeg2fix-0.7.tgz

Anyway, the more feedback, the sooner this can get into svn. Not sure
if there'll be time to get everything fixed up and included before
0.19. I guess it all depends on how much testing it gets.

to use the cutlist stuff:

get the cutlist from mysql, and apply on the command line.

select chanid, starttime, cutlist from recorded;
if mysql said:
3000 - 6231
10510 - 15321

you would do something:
./mpeg2fix -i infile.nuv -o test.mog -c "3000 - 6231" -c "10510 - 15321"

you can still use the -t or -f switches, but don't necessarily need them.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

g8ecj at gilks

Nov 12, 2005, 7:42 PM

Post #2 of 106 (4808 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Hi Geoff

> So here we are:
> http://www.pblue.org/myth/mpeg2fix-0.7.tgz
>
> Anyway, the more feedback, the sooner this can get into svn. Not sure
> if there'll be time to get everything fixed up and included before
> 0.19. I guess it all depends on how much testing it gets.
>
> to use the cutlist stuff:
>
> get the cutlist from mysql, and apply on the command line.
>
> select chanid, starttime, cutlist from recorded;
> if mysql said:
> 3000 - 6231
> 10510 - 15321
>
> you would do something:
> ./mpeg2fix -i infile.nuv -o test.mog -c "3000 - 6231" -c "10510 - 15321"
>
> you can still use the -t or -f switches, but don't necessarily need them.
>

I get an immediate segfault - I'm afraid my gdb experience is all embedded
using gdbserver so you'll have to clue me up on how to start it with
thread support and how to get a backtrace.

Cheers

--
Robin Gilks

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 13, 2005, 6:02 AM

Post #3 of 106 (4801 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> I get an immediate segfault - I'm afraid my gdb experience is all embedded
> using gdbserver so you'll have to clue me up on how to start it with
> thread support and how to get a backtrace.
>

Sadly, I'm in the same boat:

$ mpeg2fix -i sync.nuv -o test.mpg
Opening sync.nuv
*** glibc detected *** malloc(): memory corruption (fast): 0x44fa48b8 ***
Aborted

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 13, 2005, 9:15 AM

Post #4 of 106 (4797 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/13/05, Cory Papenfuss wrote:
> Sadly, I'm in the same boat:
>
> $ mpeg2fix -i sync.nuv -o test.mpg
> Opening sync.nuv
> *** glibc detected *** malloc(): memory corruption (fast): 0x44fa48b8 ***
> Aborted

Hmm...try this. Add this line right before the call to
'av_open_input_file' (somewhere around line 493):
inputFC = NULL;

I just remembered that I had a very similar problem 2 years ago, when
working on the first incarnation of this tool. The libav in myth is a
bit different than what is in ffmpeg, and I keep forgetting about
that.

if it doesn't work, I'll need a gdb backtrace.

You could also try the statically built version here:
http://www.pblue.org/myth/mpeg2fix.gz

It is compiled against ffmpeg (which is what I normally work against,
and which doesn't require this fix). But hopefully the above
initialization will work for you.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 13, 2005, 11:19 AM

Post #5 of 106 (4785 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> Hmm...try this. Add this line right before the call to
> 'av_open_input_file' (somewhere around line 493):
> inputFC = NULL;
>
> I just remembered that I had a very similar problem 2 years ago, when
> working on the first incarnation of this tool. The libav in myth is a
> bit different than what is in ffmpeg, and I keep forgetting about
> that.
>
That got it to compile, but the -c doesn't appear to work
correctly on one of my streams. I'm sending something like:
mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7440"
and it hangs indefinately. If I cut to a different point:
mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7400"
it appears to work (although blockiness).

Seems to be related to which frame I cut on, so maybe an I vs. P
vs. B? I don't have myth on this machine, so I'm assuming that cut is the
frame num - frame num to *remove* from the stream

> if it doesn't work, I'll need a gdb backtrace.
>
> You could also try the statically built version here:
> http://www.pblue.org/myth/mpeg2fix.gz
>
> It is compiled against ffmpeg (which is what I normally work against,
> and which doesn't require this fix). But hopefully the above
> initialization will work for you.
>
Same symptoms.

Another stream it appears to work OK though.
-Cory
--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

stuarta at squashedfrog

Nov 13, 2005, 1:12 PM

Post #6 of 106 (4782 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Sun, Nov 13, 2005 at 09:15:49AM -0800, Geoffrey Hausheer wrote:
>
> if it doesn't work, I'll need a gdb backtrace.
>

I've just gone straight for the backtrace.
It looks like there is an invalid AVFormatContext being
passed in, so when it attempts to use it things fall apart.

I'm volunteering for extensive testing of this tool as I want
to be able to get to a point where I do an MPEG - MPEG
transcode to dump all commercials, then convert the result
to a DVD compliant stream.

Stuart

Attachments:

mpeg2fix-bt.txt (30.3 KB)

mythtv0368 at phracturedblue

Nov 13, 2005, 1:39 PM

Post #7 of 106 (4782 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/13/05, Cory Papenfuss papenfuss-at-juneau.me.vt.edu {MythTV}
<ticket_7ae9cde7519064942bb26bba32c78366 [at] phracturedblue> wrote:
> That got it to compile, but the -c doesn't appear to work
> correctly on one of my streams. I'm sending something like:
> mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7440"
> and it hangs indefinately. If I cut to a different point:
> mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7400"
> it appears to work (although blockiness).
I have found a couple of trivial fixes that may address the blockiness
and the hanging (and a segfault). Go here:
http://www.pblue.org/myth/mpeg2fix-0.8.tgz

It includes the other fix I posted that was giving you problems
getting started earlier too.

Basically, what I have been doing to test is to take a stream, choose
two arbitrary cut points and then move them around
incrementing/decrementing one at a time to cycle through an entire GOP
at the front and the end, watching the 'Converting Frame...' message
to see what frame number I'm on.

But yes, you are using it correctly: -c "start_frame - end_frame" (in
absolute frame numbers). Because of telecine, there is no easy way to
predict what time the cuts are supposed to happen, so I the logic got
pretty hairy, and that is what I am most concerned about.

There is one thing I know needs fixing: At the end of a cutlist, the
frame-number should restart at '0', and that doesn't happen yet. I'm
not sure if this will cause playback problems or not, but in general,
it shouldn't. After a cutframe, I do reencode such that the first
frame is always an 'I' frame, and add a sequence and gop header at
that point.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 13, 2005, 1:53 PM

Post #8 of 106 (4778 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/13/05, Stuart Auchterlonie wrote:
> On Sun, Nov 13, 2005 at 09:15:49AM -0800, Geoffrey Hausheer wrote:
> >
> > if it doesn't work, I'll need a gdb backtrace.
> >
>
> I've just gone straight for the backtrace.
> It looks like there is an invalid AVFormatContext being
> passed in, so when it attempts to use it things fall apart.
>
This should be fixed in 0.8

> I'm volunteering for extensive testing of this tool as I want
> to be able to get to a point where I do an MPEG - MPEG
> transcode to dump all commercials, then convert the result
> to a DVD compliant stream.
As I've said in the previous thread, this may not be so easy. If you
start with a DVD compliant stream, it should work pretty well, but
depending on your locale, it can be pretty difficult to get a good
starting place. For instance: In the US, the DVD spec requires that
you have either an AC3 or PCM stream, with an MP2 stream being an
optional additional stream. As far as I know everywhere else in the
world, it is the opposite (MP2 or PCM are required, AC3 is optional).
I am not sure what the constraints on GOP structures are yet (since
this code doesn't insert them except after a cut-point, you need to
ensure the structure of the stream is correct to start with). And it
seem sto all come down to compliance. I have spent a lot of time
trying to generate multiple-player compliant DVDs, and found it to be
quite tricky. Many players are picky about the AC3 thing above
(though many aren't), some don't like encdings from liba52 (or
bbmpeg), but work ok with commercial encoders, I normally need ot
reencode to meet the resolution requirements which gets me the correct
GOP structure for free, but haven't had much luck copying directly
from a PVR250 output to DVD. I guess it all depends on what works for
you.

But I'll be happy to try to fix things as long as I can get good debugging data.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 13, 2005, 1:53 PM

Post #9 of 106 (4783 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

stuarta at squashedfrog

Nov 13, 2005, 2:37 PM

Post #10 of 106 (4807 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Sun, Nov 13, 2005 at 01:53:05PM -0800, Geoffrey Hausheer wrote:
> On 11/13/05, Stuart Auchterlonie wrote:
> > On Sun, Nov 13, 2005 at 09:15:49AM -0800, Geoffrey Hausheer wrote:
> > >
> > > if it doesn't work, I'll need a gdb backtrace.
> > >
> >
> > I've just gone straight for the backtrace.
> > It looks like there is an invalid AVFormatContext being
> > passed in, so when it attempts to use it things fall apart.
> >
> This should be fixed in 0.8

Yep it's fixed. This version runs to completion.
Just testing the output with mplayer. Looking good so far.

>
> > I'm volunteering for extensive testing of this tool as I want
> > to be able to get to a point where I do an MPEG - MPEG
> > transcode to dump all commercials, then convert the result
> > to a DVD compliant stream.
> As I've said in the previous thread, this may not be so easy. If you
> start with a DVD compliant stream, it should work pretty well, but
> depending on your locale, it can be pretty difficult to get a good
> starting place. For instance: In the US, the DVD spec requires that
> you have either an AC3 or PCM stream, with an MP2 stream being an
> optional additional stream. As far as I know everywhere else in the
> world, it is the opposite (MP2 or PCM are required, AC3 is optional).
> I am not sure what the constraints on GOP structures are yet (since
> this code doesn't insert them except after a cut-point, you need to
> ensure the structure of the stream is correct to start with). And it
> seem sto all come down to compliance. I have spent a lot of time
> trying to generate multiple-player compliant DVDs, and found it to be
> quite tricky. Many players are picky about the AC3 thing above
> (though many aren't), some don't like encdings from liba52 (or
> bbmpeg), but work ok with commercial encoders, I normally need ot
> reencode to meet the resolution requirements which gets me the correct
> GOP structure for free, but haven't had much luck copying directly
> from a PVR250 output to DVD. I guess it all depends on what works for
> you.
>
> But I'll be happy to try to fix things as long as I can get good debugging data.
>

My main requirement is to remove the flagged sections of the
data stream, without changing it otherwise. It looks like your
test tool does the trick.

I have a good starting point. I'm starting with an MPEG TS stream
recorded from DVB-T. Here's the relevant part of the output

----
Opening /data/mythtv/1014_20051017222900.mpg
Input #0, mpegts, from '/data/mythtv/1014_20051017222900.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x23a], 25.00 fps: Video: mpeg2video, yuv420p, 544x576, 10000 kb/s Stream #0.1[0x23b](eng): Audio: mp2, 48000 Hz, stereo, 192 kb/s
Stream #0.2[0x23c](eng): Audio: mp2, 48000 Hz, mono, 64 kb/s
Stream #0.3[0x23d](eng): Subtitle: dvbsub
Skipping unsupported codec 3
#0 PTS:3307559283 Delta: 0.000000ms queue: 12
#1 PTS:3307558334 Delta: 10.544444ms queue: 2
#2 PTS:3307557608 Delta: 18.611111ms queue: 3

stuarta at squashedfrog

Nov 13, 2005, 4:44 PM

Post #11 of 106 (4791 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Comments so far.

Output stream is MPEG PS while input stream is MPEG TS.

Analyzing the output file with ffmpeg gives the following
errors.

---
[mpeg2video @ 0x8332208]ac-tex damaged at 0 0
[mpeg2video @ 0x8332208]ac-tex damaged at 0 1
.....
all the way down to
.....
[mpeg2video @ 0x8332208]ac-tex damaged at 0 35
[mpeg2video @ 0x8332208]Warning MVs not available
[mpeg2video @ 0x8332208]concealing 1224 DC, 1224 AC, 1224 MV errors
Input #0, mpeg, from '../../utils/mpeg2fix-0.8/scream.mpg':
Duration: 01:46:15.6, start: 0.298611, bitrate: 2169 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 544x576, 25.00 fps, 10000 kb/s
Stream #0.1[0x1c1]: Audio: mp2, 48000 Hz, mono, 64 kb/s
Stream #0.2[0x1c0]: Audio: mp2, 48000 Hz, stereo, 192 kb/s
---

compared to the original input file.

---
Input #0, mpegts, from '/data/mythtv/1014_20051017222900.mpg':
Duration: 02:15:26.4, start: 36750.112089, bitrate: 2265 kb/s
Stream #0.0[0x23a]: Video: mpeg2video, yuv420p, 544x576, 25.00 fps, 10000 kb/s
Stream #0.1[0x23b](eng): Audio: mp2, 48000 Hz, stereo, 192 kb/s
Stream #0.2[0x23c](eng): Audio: mp2, 48000 Hz, mono, 64 kb/s
Stream #0.3[0x23d](eng): Subtitle: dvbsub
---

Also, I've just noticed that the audio streams have been swapped.
The pids are in the same numerical order, but the better audio
stream is now on #0.2, originally it was #0.1

Stuart

mythtv0368 at phracturedblue

Nov 13, 2005, 4:59 PM

Post #12 of 106 (4796 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/13/05, Stuart Auchterlonie wrote:
> I have a good starting point. I'm starting with an MPEG TS stream
> recorded from DVB-T. Here's the relevant part of the output
>
> ----
> Opening /data/mythtv/1014_20051017222900.mpg
> Input #0, mpegts, from '/data/mythtv/1014_20051017222900.mpg':
> Duration: N/A, bitrate: N/A
> Stream #0.0[0x23a], 25.00 fps: Video: mpeg2video, yuv420p, 544x576, 10000 kb/s
> Stream #0.1[0x23b](eng): Audio: mp2, 48000 Hz, stereo, 192 kb/s
> Stream #0.2[0x23c](eng): Audio: mp2, 48000 Hz, mono, 64 kb/s
> Stream #0.3[0x23d](eng): Subtitle: dvbsub
> Skipping unsupported codec 3
> #0 PTS:3307559283 Delta: 0.000000ms queue: 12
> #1 PTS:3307558334 Delta: 10.544444ms queue: 2
> #2 PTS:3307557608 Delta: 18.611111ms queue: 3

Well, just be aware, these are the basic requirements for DVD as near
as I can find:
http://en.wikipedia.org/wiki/MPEG-2#DVD
SVCD may be closer at:
480x480 or 480x576

So there is no way your DVDs will be fully complaint when using the
above stream. However, if you can record your streams to DVD and
watch them with your player of choice today, the new code shouldn't
break that either.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 14, 2005, 4:13 AM

Post #13 of 106 (4781 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> On 11/13/05, Cory Papenfuss papenfuss-at-juneau.me.vt.edu {MythTV}
> <ticket_7ae9cde7519064942bb26bba32c78366 [at] phracturedblue> wrote:
>> That got it to compile, but the -c doesn't appear to work
>> correctly on one of my streams. I'm sending something like:
>> mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7440"
>> and it hangs indefinately. If I cut to a different point:
>> mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7400"
>> it appears to work (although blockiness).
> I have found a couple of trivial fixes that may address the blockiness
> and the hanging (and a segfault). Go here:
> http://www.pblue.org/myth/mpeg2fix-0.8.tgz
>
The hangs I mentioned appear to be gone now, but the blockiness at
the cuts remains.

> Basically, what I have been doing to test is to take a stream, choose
> two arbitrary cut points and then move them around
> incrementing/decrementing one at a time to cycle through an entire GOP
> at the front and the end, watching the 'Converting Frame...' message
> to see what frame number I'm on.
>
Question about this... shouldn't frame 0 already *be* an I-frame,
or is that not what these signify?

Converting frame 6 to an I-frame ((null))
Converting frame 7 to an I-frame ((null))
Converting frame 8 to an I-frame ((null))
Converting frame 0 to an I-frame ((null))
Converting frame 1 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 3 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible

> But yes, you are using it correctly: -c "start_frame - end_frame" (in
> absolute frame numbers). Because of telecine, there is no easy way to
> predict what time the cuts are supposed to happen, so I the logic got
> pretty hairy, and that is what I am most concerned about.
>
That will be tough to debug and probably just needs more people
to hammer on it to see where it breaks. At least I seem to have a
collection of rough-around-the-edges streams to do some preliminary
breaking... :)

One thing that gets a little bit screwy here is what a frame
number actually means. Since the purpose of this is be able to insert
video frames to maintain sync, it ends up being effectively synced to the
audio PTS, no?

> There is one thing I know needs fixing: At the end of a cutlist, the
> frame-number should restart at '0', and that doesn't happen yet. I'm
> not sure if this will cause playback problems or not, but in general,
> it shouldn't. After a cutframe, I do reencode such that the first
> frame is always an 'I' frame, and add a sequence and gop header at
> that point.
>
My guess is that somewhere in the construction of all that is
what's causing the blockiness. I wish I could find a decent utility to
easily parse through the resulting streams. I'm so far using avidemux to
index and count frames, etc.

I just now managed to find one... trying to back down from frame
#7400 on my stream to find if I could eliminate the blockiness, I got
this:
$mpeg2fix -i sync.nuv -o test.mpg -c "1 - 7398" -c "7450 - 51428"
Opening sync.nuv
Input #0, mpeg, from 'sync.nuv':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 704x480, 5370
kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 192 kb/s
#0 PTS:16022 Delta: 0.000000ms queue: 2
#1 PTS:16022 Delta: 0.000000ms queue: 1
Converting frame 3 to an I-frame ((null))
Converting frame 4 to an I-frame ((null))
Converting frame 5 to an I-frame ((null))
Need to insert 7394 frames > max allowed: 20

... sounds like a heuristic logic error like you may be looking
to find.

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

papenfuss at juneau

Nov 14, 2005, 4:42 AM

Post #14 of 106 (4769 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

>
> My main requirement is to remove the flagged sections of the
> data stream, without changing it otherwise. It looks like your
> test tool does the trick.
>
Trouble is, sometimes the streams need changing. The ivtv
captures from tape are notoriously full of missing video frames which
break most post-processing programs. Granted, if the commercials were cut
out, one could just ignore the timestamp mismatches if the only intent was
to play the resulting video. If the goal is to have a cleaned up
MPEG2->MPEG2 stream as the master copy on the mythbox however, it probably
needs cleaning.

Of course, other things that come more or less for free from this
are the removal of padding streams in captures (a few percent on some
captures from what I've seen), remux into more friendly PUs,
frame-accurate editing, and the possibility of hooking in tcrequant for
minor bitrate reductions on-the-fly.

Lots of potential here once things are cleaned up and easy to work
with. Great work so far... this is the closest I've seen to true lossless
cutting from within mythtv since I realized it wasn't present 18 months
ago.

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

bmayland at leoninedev

Nov 14, 2005, 8:16 AM

Post #15 of 106 (4762 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> I have found a couple of trivial fixes that may address the blockiness
> and the hanging (and a segfault). Go here:
> http://www.pblue.org/myth/mpeg2fix-0.8.tgz
>
I get segfaults right at the beginning of every stream I've tried
(has occurred in 0.4, 0.7, 0.8, haven't tested other versions).
Opening /mnt/store/1015_20051105183000.mpg
Input #0, mpeg, from '/mnt/store/1015_20051105183000.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 640x480,
9000 kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 384 kb/s
#0 PTS:36036 Delta: 0.000000ms queue: 12
#1 PTS:34684 Delta: 15.022222ms queue: 2
Warning, QMAT_SHIFT is larger then 21, overflows possible
Segmentation fault (core dumped)

Backtrace:
#0 0xb7436fc6 in dsputil_init_mmx ()
from /usr/local/lib/libmythavcodec-0.18.so.0
#1 0x083404d0 in ?? ()

Bonus: Shell script to pull cutlist from DB and call mpeg2fix
#!/bin/sh

if [ -z "$1" ] ; then
echo Usage: $0 [filename]
exit 1
fi

IN_FILE=`basename $1`

mysql -s -B -u mythtv --password=mythtv mythconverg > /tmp/cut << EOF
SELECT mark FROM recordedmarkup rm
INNER JOIN recorded rd
ON rm.chanid = rd.chanid AND rm.starttime = rd.starttime
WHERE rd.basename = '$IN_FILE'
AND rm.type IN (4,5)
ORDER BY rm.mark, rm.type
EOF

CUTLIST_R=`cat /tmp/cut | awk '((NR%2)==0){printf "%d ",$1}
((NR%2)==1){printf "-c %d-",$1}'`

./mpeg2fix -i $1 -o test.mpg $CUTLIST_R

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 14, 2005, 7:31 PM

Post #16 of 106 (4769 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/14/05, Bryan Mayland wrote:
> Geoffrey Hausheer wrote:
> > I have found a couple of trivial fixes that may address the blockiness
> > and the hanging (and a segfault). Go here:
> > http://www.pblue.org/myth/mpeg2fix-0.8.tgz
> >
> I get segfaults right at the beginning of every stream I've tried
> (has occurred in 0.4, 0.7, 0.8, haven't tested other versions).
> Opening /mnt/store/1015_20051105183000.mpg
> Input #0, mpeg, from '/mnt/store/1015_20051105183000.mpg':
> Duration: N/A, bitrate: N/A
> Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 640x480,
> 9000 kb/s
> Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 384 kb/s
> #0 PTS:36036 Delta: 0.000000ms queue: 12
> #1 PTS:34684 Delta: 15.022222ms queue: 2
> Warning, QMAT_SHIFT is larger then 21, overflows possible
> Segmentation fault (core dumped)
>
> Backtrace:
> #0 0xb7436fc6 in dsputil_init_mmx ()
> from /usr/local/lib/libmythavcodec-0.18.so.0
> #1 0x083404d0 in ?? ()
>
You'd need to recompile with debug enabled, and might need a
thread-capable gdb. make sure to always use 'thread apply all bt'
when debugging. But this looks like a failure inside of libavcodec,
which makes little sense. My guess is that the problem is happening
in build_frame. running with debug set to '-d 3' would be interesting
you could also set a break point at line 723, and step through until
you find the faulty instruction. using imagemagik to display the
resulting yuv file might be useful too.
You could also try with the static build here:
http://www.pblue.org/myth/mpeg2fix.bz2
This would at least help seperate problems with your env from problems
with the code.

capturing the fault in gdb with that version would help determine if
it is specific to your libav, or your streams.

> Bonus: Shell script to pull cutlist from DB and call mpeg2fix
Just be awate, this tool is still in the debugging stage. You
certainly won't want to replace the orginal files with processed ones
at this point. As soon as I am convinced the code is working pretty
well, it will be added to myth, at which times such scripts won't be
needed.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 14, 2005, 9:29 PM

Post #17 of 106 (4746 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Here is mpeg2fix-0.9
it should fix the blockiness Cory reported, and a frame-corruption
issue I found, and has some other minor fixes, but I don't think it'll
fix anyone else's crashing or infinite-loop problems. Unfortunately,
I'm mostly shooting in the dark on those, as I haven't been able to
reproduce them yet.

http://www.pblue.org/myth/mpeg2fix-0.9.tgz

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 15, 2005, 5:11 AM

Post #18 of 106 (4755 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> Here is mpeg2fix-0.9
> it should fix the blockiness Cory reported, and a frame-corruption
> issue I found, and has some other minor fixes, but I don't think it'll
> fix anyone else's crashing or infinite-loop problems. Unfortunately,
> I'm mostly shooting in the dark on those, as I haven't been able to
> reproduce them yet.
>
> http://www.pblue.org/myth/mpeg2fix-0.9.tgz
>
Blockiness fixed, sorta... I think. Something still seems funny,
though... when I compare the resulting stream frame-by-frame to the
original, I see some differences. It would appear that the first three
frames in the output are I-frames... the first is the same as the
originals' first frame. The second is presumably generated by mpeg2fix.
The third is from the original... 7400 frames down the line.

The second I-frame looks blockier than the B-frame it was
generated from. Also, the third I-frame (which should be the same as the
original 7400) look different and more blocky. Wrong header that gets
inherited? By the next GOP, things have cleared up.

Orig (from the beginning):
PTS: 16022 dts: 13019
pos: 0x0 PIC-HEADER I-Frame #: 0 length: 22994 crc: 15a977ee
pos: 0x0 PIC-HEADER P-Frame #: 3 length: 18116 crc: 7e565498
PTS: 16022
pos: 0x0 PIC-HEADER B-Frame #: 1 length: 11140 crc: e7a64f89
pos: 0x0 PIC-HEADER B-Frame #: 2 length: 10896 crc: 8078dce2
PTS: 24662

Orig (from the 7400th frame onward)
PTS: 22188422
pos: 0x0 PIC-HEADER B-Frame #: 4 length: 11132 crc: 16d7601b
PTS: 22197062
pos: 0x0 PIC-HEADER P-Frame #: 8 length: 18084 crc: c28d9c79
pos: 0x0 PIC-HEADER B-Frame #: 6 length: 11412 crc: d76eb7fb
pos: 0x0 PIC-HEADER B-Frame #: 7 length: 11156 crc: fb0bb330
PTS: 22203542
pos: 0x0 PIC-HEADER P-Frame #: 11 length: 16680 crc: 058ac701
pos: 0x0 PIC-HEADER B-Frame #: 9 length: 11188 crc: 6f413a13

Copy (from the beginning of the file told to cut 0-7400):
PTS: 32400 dts: 29397
PTS: 35403 dts: 32400
pos: 0x0 PIC-HEADER I-Frame #: 0 length: 22994 crc: 15a977ee
PTS: 38406 dts: 35403
pos: 0x0 PIC-HEADER I-Frame #: 7 length: 31878 crc: 00e9c291
PTS: 47415 dts: 38406
pos: 0x0 PIC-HEADER I-Frame #: 8 length: 33701 crc: fa7a8478
PTS: 41409
pos: 0x0 PIC-HEADER P-Frame #: 11 length: 16680 crc: 058ac701
PTS: 44412
pos: 0x0 PIC-HEADER B-Frame #: 9 length: 11188 crc: 6f413a13

The bad heuristic still present:
"Need to insert 7394 frames > max allowed: 20"

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

stuarta at squashedfrog

Nov 15, 2005, 7:20 AM

Post #19 of 106 (4732 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Mon, Nov 14, 2005 at 09:29:22PM -0800, Geoffrey Hausheer wrote:
> Here is mpeg2fix-0.9
> it should fix the blockiness Cory reported, and a frame-corruption
> issue I found, and has some other minor fixes, but I don't think it'll
> fix anyone else's crashing or infinite-loop problems. Unfortunately,
> I'm mostly shooting in the dark on those, as I haven't been able to
> reproduce them yet.
>
> http://www.pblue.org/myth/mpeg2fix-0.9.tgz
>

I now have an infinite loop that wasn't there before.
Changes have been 0.8-0.9 and slightly different cutlist.

Some backtraces from where it's looping
(plus a full backtrace as an attachment)

If you need more info let me know

Stuart

----
(gdb) bt
#0 0x08062288 in seek_chunk (mpeg2dec=0x8084880) at decode.c:127
#1 0x080621a0 in mpeg2_seek_header (mpeg2dec=0x8084880) at decode.c:142
#2 0x080623ec in mpeg2_parse (mpeg2dec=0x8084880) at decode.c:159
#3 0x0804c0c1 in MPEG2fixup::process_video (this=0xbff2c824, vf=0x8130468, decode_pic=1)
at mpeg2fix.cpp:753
#4 0x0804cd77 in MPEG2fixup::decode_to_frame (this=0xbff2c824, frameNum=11) at mpeg2fix.cpp:1169
#5 0x0804cdf2 in MPEG2fixup::convert_to_i (this=0xbff2c824, frameNum=9, numFrames=3, headPos=9)
at mpeg2fix.cpp:1188
#6 0x08050689 in MPEG2fixup::start (this=0xbff2c824) at mpeg2fix.cpp:1458
#7 0x08051a79 in main (argc=15, argv=0xbff2db94) at mpeg2fix.cpp:1775

----
(gdb) bt
#0 0x0804c0c4 in MPEG2fixup::process_video (this=0xbff2c824, vf=0x8130468, decode_pic=1)
at mpeg2fix.cpp:753
#1 0x0804cd77 in MPEG2fixup::decode_to_frame (this=0xbff2c824, frameNum=11) at mpeg2fix.cpp:1169
#2 0x0804cdf2 in MPEG2fixup::convert_to_i (this=0xbff2c824, frameNum=9, numFrames=3, headPos=9)
at mpeg2fix.cpp:1188
#3 0x08050689 in MPEG2fixup::start (this=0xbff2c824) at mpeg2fix.cpp:1458
#4 0x08051a79 in main (argc=15, argv=0xbff2db94) at mpeg2fix.cpp:1775

Attachments:

mpeg2fix.bt-full.txt (32.1 KB)

mythtv0368 at phracturedblue

Nov 15, 2005, 7:45 AM

Post #20 of 106 (4729 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/15/05, Cory Papenfuss wrote:
> Orig (from the beginning):
...

your original looks like this (in presentation order)
0 ... 7 8 9 10 11
I ... B P B B P
I ... I* I* B B P

where the '*' represents a regenerated frame. This was done correctly
(or, more precisely, as expected) If the I* frames aren't generated
correctly, then the rest of the GOP is likely to be hosed, as all
further frames (P and B) are generated from frame 8. This is the
situation in which looking at the orginal B frames, and regenerated I
frames should help. Your streams seem to have custom quantization
matrices, and that may be influencing the code too. I have a clip you
sent me earlier, and I'll play with that and see if I can see what you
see. The 'enc' data still isn't being generated correctly for some
reason, so I can't yet do before-vs-after comparisons.

> The second I-frame looks blockier than the B-frame it was
> generated from. Also, the third I-frame (which should be the same as the
> original 7400) look different and more blocky. Wrong header that gets
> inherited? By the next GOP, things have cleared up.
See the above explanation. I hope it explains what is going on.

> The bad heuristic still present:
> "Need to insert 7394 frames > max allowed: 20"
This is caused by cutting on a frame that has no PTS. I know how to
fix it, but haven't done so yet, as I'm trying to figure out a way to
do it that isn't a hack. The next release should have this fixed,
though.
.

And to answer your other email....
> I would also suggest that keeping a stand-alone version of it
> would be useful. AFAIK, there doesn't exist a good "cleanup" utility (or
> I would be using it). Also, I don't do video processing on my mythbox,
> but rather dump the file to USB HD and haul it into my office where I've got a
> machine with more oomph.... having to have a full-blown mythtv install to
> use the code would be a bit overkill.
Yes this is my plan. I might even be able to get rid of the Qt
requirement (I'm only using it because it has more powerful lists than
C++ does on its own). While the program is being targeted at myth, I
am doing my best to make sure it can be used stand-aolne as well.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 15, 2005, 8:28 AM

Post #21 of 106 (4748 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> your original looks like this (in presentation order)
> 0 ... 7 8 9 10 11
> I ... B P B B P
> I ... I* I* B B P
>
> where the '*' represents a regenerated frame. This was done correctly
> (or, more precisely, as expected) If the I* frames aren't generated
> correctly, then the rest of the GOP is likely to be hosed, as all
> further frames (P and B) are generated from frame 8. This is the
> situation in which looking at the orginal B frames, and regenerated I
> frames should help. Your streams seem to have custom quantization
> matrices, and that may be influencing the code too. I have a clip you
> sent me earlier, and I'll play with that and see if I can see what you
> see. The 'enc' data still isn't being generated correctly for some
> reason, so I can't yet do before-vs-after comparisons.
>
Whoops... I realized that it might be the stream-order vs.
presentation order that was confusing me. Still though...

>> The second I-frame looks blockier than the B-frame it was
>> generated from. Also, the third I-frame (which should be the same as the
>> original 7400) look different and more blocky. Wrong header that gets
>> inherited? By the next GOP, things have cleared up.
> See the above explanation. I hope it explains what is going on.
>
My original thought still seems correct. I finally wrote up a
diagram to step through and see which is which. In yours, #8 which is
a P in the original is different from the generated I* in the copy. I
understand that all subsequent frames after #8 I* in the copy will be
based on #8's data then.

If it's any help, the original has interlacing present, whereas
the copy looks to have less interlacing, but more blocky colors. I can
send a chunk of the original containing the cut if desired.

Also, it might be helpful to hack up mpegparse to include what he
thinks the frame# is (as per mepg2fix's definition). As it is, I'm still
using avidemux to visually determine absolute frame numbers (and I/P/B
type).

Man... between all the logic of GOPs, open/closed, PTS
discrepancies, stream order AND frame order, it's a bit of a mind-job to
sort it all out.

>> The bad heuristic still present:
>> "Need to insert 7394 frames > max allowed: 20"
> This is caused by cutting on a frame that has no PTS. I know how to
> fix it, but haven't done so yet, as I'm trying to figure out a way to
> do it that isn't a hack. The next release should have this fixed,
> though.
> .
>
Nifty.

> Yes this is my plan. I might even be able to get rid of the Qt
> requirement (I'm only using it because it has more powerful lists than
> C++ does on its own). While the program is being targeted at myth, I
> am doing my best to make sure it can be used stand-aolne as well.
>
Yay. :)

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 15, 2005, 8:46 AM

Post #22 of 106 (4731 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/15/05, Stuart Auchterlonie wrote:
> On Mon, Nov 14, 2005 at 09:29:22PM -0800, Geoffrey Hausheer wrote:
> > Here is mpeg2fix-0.9
> I now have an infinite loop that wasn't there before.
> Changes have been 0.8-0.9 and slightly different cutlist.
>
> Some backtraces from where it's looping
> (plus a full backtrace as an attachment)
>
> If you need more info let me know
Well, I know why you are getting the infinite loop, but am not sure
how that can happen, and don't know what to do about it. Basically,
the decoder was unable to find an entire frame, and is waiting for
additional data. If the frame is truely broken, it is impossible to
do a frame conversion, and I'm not sure how to proceed.

Can you add the following patch, and send me the resulting abort.dat
file along with the error message?
also running 'print *vf' from gdb after breaking (make sure you are in
process video) and send me that might be useful too.
------------
--- mpeg2fix.cpp.old 2005-11-14 21:26:38.000000000 -0800
+++ mpeg2fix.cpp 2005-11-15 08:35:41.000000000 -0800
@@ -782,6 +782,10 @@
fprintf(stderr, "Warning: partial frame found!\n");
return 0;
}
+ } else if (state == STATE_BUFFER) {
+ write_data("abort.dat", vf->pkt.data, vf->pkt.size);
+ fprintf(stderr, "Failed to decode frame. Position was:
%d\n", last_pos);
+ assert(state != STATE_BUFFER);
}

last_pos = (vf->pkt.size - mpeg2_getpos(dec)) - 4;
--------------------
Thanks,
.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 15, 2005, 9:33 AM

Post #23 of 106 (4733 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>Here is mpeg2fix-0.9
>it should fix the blockiness Cory reported, and a frame-corruption
>issue I found, and has some other minor fixes, but I don't think it'll
>fix anyone else's crashing or infinite-loop problems. Unfortunately,
>I'm mostly shooting in the dark on those, as I haven't been able to
>reproduce them yet.
>
>http://www.pblue.org/myth/mpeg2fix-0.9.tgz
>
>.Geoff
>_______________________________________________
>mythtv-dev mailing list
>mythtv-dev [at] mythtv
>http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
>
>
Does this have to be compiled against bleeding edge SVN? I am staying
with 7738 until LiveTV gets sorted, and when I try to compile this I get
the following error:

root [at] mytht:~/test/mpeg2fix-0.9# pushd replex; make ; popd
~/test/mpeg2fix-0.9/replex ~/test/mpeg2fix-0.9
Makefile:56: .depend: No such file or directory
cc -M avi.c element.c mpg_common.c pes.c replex.c ringbuffer.c ts.c
multiplex.c -I..> .depend
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. element.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. pes.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. mpg_common.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. ts.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. ringbuffer.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. avi.c
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. multiplex.c
ar -rcs libreplex.a element.o pes.o mpg_common.o ts.o ringbuffer.o avi.o
multiplex.o
cc -c -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -I.. replex.c
cc -o replex replex.o -L. -lreplex
~/test/mpeg2fix-0.9
root [at] mytht:~/test/mpeg2fix-0.9# make
g++ -g -O0 -fno-inline -fno-default-inline -c -I./libavformat
-I./libavcodec -I./libavutil -I./libmpeg2 -I./replex -I/usr/include/qt3
mpeg2fix.cpp
cc -g -O0 -fno-inline -fno-default-inline -c -I./libavformat
-I./libavcodec -I./libavutil -I./libmpeg2 -I./replex -I/usr/include/qt3
helper.c
cc1: warning: "-fno-default-inline" is valid for C++ but not for C/ObjC
g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
-L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
-lpthread -lqt-mt -lm
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCLoadQMatrix'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCSyncSurface'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCFlushSurface'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCBeginSurface'
collect2: ld returned 1 exit status
make: *** [mpeg2fix] Error 1

Thoughts? I'd like to help test this out, as I could use the functionality.

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

stuarta at squashedfrog

Nov 15, 2005, 9:43 AM

Post #24 of 106 (4732 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Tue, Nov 15, 2005 at 12:33:34PM -0500, Tom Lichti wrote:
> g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
> helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
> -L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
> -lpthread -lqt-mt -lm
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCLoadQMatrix'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCSyncSurface'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCFlushSurface'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCBeginSurface'
> collect2: ld returned 1 exit status
> make: *** [mpeg2fix] Error 1
>
> Thoughts? I'd like to help test this out, as I could use the functionality.
>

Looks like your myth build uses XvMC, so you will need to add
the XvMC libs to the link line.

stuarta at squashedfrog

Nov 15, 2005, 9:51 AM

Post #25 of 106 (4734 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Tue, Nov 15, 2005 at 08:46:58AM -0800, Geoffrey Hausheer wrote:
> Well, I know why you are getting the infinite loop, but am not sure
> how that can happen, and don't know what to do about it. Basically,
> the decoder was unable to find an entire frame, and is waiting for
> additional data. If the frame is truely broken, it is impossible to
> do a frame conversion, and I'm not sure how to proceed.
>
> Can you add the following patch, and send me the resulting abort.dat
> file along with the error message?
> also running 'print *vf' from gdb after breaking (make sure you are in
> process video) and send me that might be useful too.

Interesting. I tried using the cutpoints from previous runs where
things worked, and it runs to completion again. Looks like where
the cutpoints are affects this. My two command lines...

Working:
nice -10 ./mpeg2fix -i /data/mythtv/1014_20051017222900.mpg -o test2.mpg -c "0 - 7960" -c "40761 - 48204" -c "70560 - 78037" -c "141288 - 148735" -c "189744 - 203159"

Loops:
nice -10 ./mpeg2fix -i /data/mythtv/1014_20051017222900.mpg -o test.mpg -c "0 - 7960" -c "40750 - 48204" -c "70560 - 78037" -c "141288 - 148735" -c "189744 - 203159"

It looks like that first cutpoint change (from 40761 to 40750) is what does
it.

Output when it stops
Failed to decode frame. Position was: 0
mpeg2fix: mpeg2fix.cpp:788: bool MPEG2fixup::process_video(MPEG2frame*, int): Assertion `state != STATE_BUFFER' failed.
Aborted

I'll send the rest of the logs direct as they are two big for the list.

Stuart

tom at redpepperracing

Nov 15, 2005, 9:58 AM

Post #26 of 106 (2828 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Stuart Auchterlonie wrote:

>On Tue, Nov 15, 2005 at 12:33:34PM -0500, Tom Lichti wrote:
>
>
>>g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
>>helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
>>-L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
>>-lpthread -lqt-mt -lm
>>/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>`XvMCLoadQMatrix'
>>/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>`XvMCSyncSurface'
>>/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>`XvMCFlushSurface'
>>/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>`XvMCBeginSurface'
>>collect2: ld returned 1 exit status
>>make: *** [mpeg2fix] Error 1
>>
>>Thoughts? I'd like to help test this out, as I could use the functionality.
>>
>>
>>
>
>Looks like your myth build uses XvMC, so you will need to add
>the XvMC libs to the link line.
>
>
Makes sense, but what/where should I be linking to? libavcodec seems to
be the culprit, but I see the xvmc stuff in that directory, not sure
where to go from here.

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 15, 2005, 10:04 AM

Post #27 of 106 (2833 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Tom Lichti wrote:
> Makes sense, but what/where should I be linking to? libavcodec seems
> to be the culprit, but I see the xvmc stuff in that directory, not
> sure where to go from here.
In the Makefile I added another LIBS line:
LIBS += -lXvMCW

This assumes you have XvMC wrappers though. Check your ./configure
output from mythtv and it will say which -l is used for XvMC I believe.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 15, 2005, 10:40 AM

Post #28 of 106 (2828 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Bryan Mayland wrote:

> Tom Lichti wrote:
>
>> Makes sense, but what/where should I be linking to? libavcodec seems
>> to be the culprit, but I see the xvmc stuff in that directory, not
>> sure where to go from here.
>
> In the Makefile I added another LIBS line:
> LIBS += -lXvMCW
>
> This assumes you have XvMC wrappers though. Check your ./configure
> output from mythtv and it will say which -l is used for XvMC I believe.

Well, I tried that, and I tried just -lXvMC, neither worked. ./configure
shows: XvMC libs -lXvMCW

Any more thoughts?

Thanks
Tom

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 15, 2005, 11:10 AM

Post #29 of 106 (2832 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> I don't think it'll
> fix anyone else's crashing or infinite-loop problems. Unfortunately,
> I'm mostly shooting in the dark on those, as I haven't been able to
> reproduce them yet.
>
Do you have an SSE/2 enabled processor? The problem in my case
seems to be that the outbuf malloc in MPEG2fixup::build_frame() is
returning an 8-byte aligned pointer. I believe this blows up on
libavcodec when it tries to do an SSE2 DCT.
Theres a similar possible problem in MPEG2frame() ctor and
MPEG2frame::ensure_size().
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 15, 2005, 11:16 AM

Post #30 of 106 (2828 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Tom Lichti wrote:
> Well, I tried that, and I tried just -lXvMC, neither worked.
> ./configure shows: XvMC libs -lXvMCW
>
Well that's unusual. Is it not on a standard lib path maybe? Did
you try adding -L/path/to/where/libXvMCW.a is?
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 15, 2005, 11:24 AM

Post #31 of 106 (2828 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Bryan Mayland wrote:

> Tom Lichti wrote:
>
>> Well, I tried that, and I tried just -lXvMC, neither worked.
>> ./configure shows: XvMC libs -lXvMCW
>>
> Well that's unusual. Is it not on a standard lib path maybe? Did
> you try adding -L/path/to/where/libXvMCW.a is?

It's in /usr/X11R6/lib which doesn't seem out of place. I have that path
in /etc/ld/so.conf, I added

LIBS += -L/usr/X11R6/lib

to the Makefile, still no joy.

Tom

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 15, 2005, 11:29 AM

Post #32 of 106 (2840 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/15/05, Bryan Mayland wrote:
> Geoffrey Hausheer wrote:
> > I don't think it'll
> > fix anyone else's crashing or infinite-loop problems. Unfortunately,
> > I'm mostly shooting in the dark on those, as I haven't been able to
> > reproduce them yet.
> >
> Do you have an SSE/2 enabled processor? The problem in my case
> seems to be that the outbuf malloc in MPEG2fixup::build_frame() is
> returning an 8-byte aligned pointer. I believe this blows up on
> libavcodec when it tries to do an SSE2 DCT.
> Theres a similar possible problem in MPEG2frame() ctor and
> MPEG2frame::ensure_size().
Quite possible. While my processor is certainly SSE2 capable, it is
quite likely that I'm not using any SSE2 instructions in my
libavcodec. I'll look into aligning it for the next release.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 15, 2005, 1:06 PM

Post #33 of 106 (2837 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> You'd need to recompile with debug enabled, and might need a
> thread-capable gdb. make sure to always use 'thread apply all bt'
> when debugging. But this looks like a failure inside of libavcodec,
> which makes little sense. My guess is that the problem is happening
> in build_frame.
With debug enabled it faults in the avcodec_encode_video() function
(specifically in the dct_quantize_SSE2 method). Here's a backtrace from
the thread:
(gdb) bt
#0 0xb740f246 in dct_quantize_SSE2 (s=0xb71e7783, block=0x8229450,
n=135775936, qscale=0, overflow=0x2) at mpegvideo_mmx_template.c:152
#1 0x08229450 in ?? ()
#2 0xb71e7783 in encode_mb (s=0x8229450, motion_x=0, motion_y=0)
at mpegvideo.c:4243
#3 0xb71ecb9d in encode_thread (c=0x84555f0, arg=0x8229450)
at mpegvideo.c:5146
#4 0xb71ceaa4 in avcodec_default_execute (c=0x84555f0,
func=0xb71ea590 <encode_thread>, arg=0x8229514, ret=0x0, count=1)
at utils.c:420
#5 0xb71de764 in MPV_encode_picture (avctx=0x84555f0, buf=0xb5d9d008 "",
buf_size=-128, data=0x8140f60) at mpegvideo.c:5457
#6 0xb71d022a in avcodec_encode_video (avctx=0x84555f0,
buf=0xffffff80 <Address 0xffffff80 out of bounds>, buf_size=614400,
pict=0x8140f60) at utils.c:868
#7 0x0804c838 in MPEG2fixup::build_frame (this=0xbfa71084, pkt=0xbfa70c2c,
fname=0x0) at mpeg2fix.cpp:929
#8 0x0804ce67 in MPEG2fixup::convert_to_i (this=0xbfa71084, frameNum=0,
numFrames=2, headPos=0) at mpeg2fix.cpp:1194
#9 0x08050699 in MPEG2fixup::start (this=0xbfa71084) at mpeg2fix.cpp:1458
#10 0x08051a89 in main (argc=7, argv=0xbfa72404) at mpeg2fix.cpp:1775

It is interesting to not that the buf paramter to avcodec_encode_video()
is bad in the bt, but was valid when it was passed. Since it appears to
be an alignment issue, I changed the passed Context to disable all
MMX/SSE routines. I got this output, no segfaults, and a working mpg
file it seems:
Opening /mnt/store/1015_20051108183000.mpg
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags: mmx mmxext sse sse2
Input #0, mpeg, from '/mnt/store/1015_20051108183000.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 640x480,
6000 kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 384 kb/s
#0 PTS:36039 Delta: 0.000000ms queue: 11
#1 PTS:34685 Delta: 15.044444ms queue: 2
Mux rate: 6.49 Mbit/s
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 0 to an I-frame ((null))
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 1 to an I-frame ((null))
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 9 to an I-frame ((null))
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 10 to an I-frame ((null))
[mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 11 to an I-frame ((null))

> Just be awate, this tool is still in the debugging stage. You
> certainly won't want to replace the orginal files with processed ones
> at this point. As soon as I am convinced the code is working pretty
> well, it will be added to myth, at which times such scripts won't be
> needed.
>
Yeah the shell script is just nice to have so I don't have to figure
out what the cutpoints are for every file I want to test with.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 15, 2005, 1:51 PM

Post #34 of 106 (2833 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/15/05, Bryan Mayland wrote:
> Geoffrey Hausheer wrote:
> With debug enabled it faults in the avcodec_encode_video() function
> (specifically in the dct_quantize_SSE2 method). Here's a backtrace from
> the thread:
> (gdb) bt
...
> #6 0xb71d022a in avcodec_encode_video (avctx=0x84555f0,
> buf=0xffffff80 <Address 0xffffff80 out of bounds>, buf_size=614400,
> pict=0x8140f60) at utils.c:868
> #7 0x0804c838 in MPEG2fixup::build_frame (this=0xbfa71084, pkt=0xbfa70c2c,
> fname=0x0) at mpeg2fix.cpp:929
> #8 0x0804ce67 in MPEG2fixup::convert_to_i (this=0xbfa71084, frameNum=0,
> numFrames=2, headPos=0) at mpeg2fix.cpp:1194
> #9 0x08050699 in MPEG2fixup::start (this=0xbfa71084) at mpeg2fix.cpp:1458
> #10 0x08051a89 in main (argc=7, argv=0xbfa72404) at mpeg2fix.cpp:1775
>
I don't understand why buf == -128 that isn't an allowed output from
malloc. Have you tried replacing the malloc with a memalign(16,
outbuf_size)? there may be some stack corruption that is causing buf
to be invalid after the call to avccodec_encode_video
The same may go for intra_matrix.
It could be aligned (I think) via:
uint16_t intra_matrix[64] ATTR_ALIGNED(16);

> Opening /mnt/store/1015_20051108183000.mpg
> [mpeg2video @ 0xb7454f04]libavcodec: CPU flags: mmx mmxext sse sse2
> Input #0, mpeg, from '/mnt/store/1015_20051108183000.mpg':
> Duration: N/A, bitrate: N/A
> Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 640x480,
> 6000 kb/s
> Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 384 kb/s
> #0 PTS:36039 Delta: 0.000000ms queue: 11
> #1 PTS:34685 Delta: 15.044444ms queue: 2
> Mux rate: 6.49 Mbit/s
> [mpeg2video @ 0xb7454f04]libavcodec: CPU flags:
> Warning, QMAT_SHIFT is larger then 21, overflows possible
> Converting frame 0 to an I-frame ((null))
This looks normal (though I still have no idea what those QMAT_SHIFT
messages mean)

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 15, 2005, 2:44 PM

Post #35 of 106 (2836 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Bryan Mayland wrote:
> With debug enabled it faults in the avcodec_encode_video() function
> (specifically in the dct_quantize_SSE2 method). Here's a backtrace
> from the thread:
> (gdb) bt
> #0 0xb740f246 in dct_quantize_SSE2 (s=0xb71e7783, block=0x8229450,
> n=135775936, qscale=0, overflow=0x2) at mpegvideo_mmx_template.c:152
Which you may be interested to know is fixed following this ticket
from last week:
http://cvs.mythtv.org/trac/ticket/626
This is fixed by svn rev 7837. I didn't notice because I'm pinned to
like 7738. Anyone experiencing segfaults may want to svn update -rHEAD
mythtv/libs/libavcodec/i386/mpegvideo_mmx_template.c to see if it fixes
their problem.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 15, 2005, 8:12 PM

Post #36 of 106 (2842 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Here's another one:
http://www.pblue.org/myth/mpeg2fix-0.10.tgz

Should fix:
Cory's problem with an abort that looked like:
Need to insert 139 frames > max allowed: 20

Bryan's problem with an infinite loop (or a failed assert)

Sort of fixes:
Cory's blockiness. Basically I just forced the quality to be at level
2. Makes it better, but doesn't necessarily fix the underlying
problem. There may be other contols that can be sent to the encoder.
you can play with the line that says 'c->qmax =' to try different
values, lower is better (and bigger) with '1' being the smallest
allowed. qmax must be >= qmin. largest allowed size is 31

Possibly fixes:
Robin's hang. Assuming it is the same issue Bryan was having.

The crash in avcdec_encode, if it was due to an alignment issue (I
have applied the relevant alignment requested earlier)

Let me know how it goes.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

g8ecj at gilks

Nov 16, 2005, 2:01 AM

Post #37 of 106 (2818 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> Here's another one:
> http://www.pblue.org/myth/mpeg2fix-0.10.tgz
>
> Should fix:
> Cory's problem with an abort that looked like:
> Need to insert 139 frames > max allowed: 20
>
> Bryan's problem with an infinite loop (or a failed assert)
>
> Sort of fixes:
> Cory's blockiness. Basically I just forced the quality to be at level
2. Makes it better, but doesn't necessarily fix the underlying
> problem. There may be other contols that can be sent to the encoder.
you can play with the line that says 'c->qmax =' to try different
values, lower is better (and bigger) with '1' being the smallest
allowed. qmax must be >= qmin. largest allowed size is 31
>
>
> Possibly fixes:
> Robin's hang. Assuming it is the same issue Bryan was having.
>
> The crash in avcdec_encode, if it was due to an alignment issue (I have
applied the relevant alignment requested earlier)
>
> Let me know how it goes.
>

Works for me now - the infinte loop appears to have gone (I've tried
different cut point to check this!!)

This is the output from the processing of a 1 hour show with 3 breaks
being edited out. I still get the odd QMAT_SHIFT warnings (whatever they
mean).

One question Geoff - is the whitespace required around the dash in the
cutpoint frame number range? Guess I'll have to try it :-)

./mpeg2fix -i /mnt/store/2056_20051101113000.mpg -o /mnt/tmp/xxx.mpg -c
"28378 - 30050" -c "60121 - 61919" -c "87697 - 89795" -f mpeg2
Opening /mnt/store/2056_20051101113000.mpg
Input #0, mpeg, from '/mnt/store/2056_20051101113000.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 25.00 fps: Video: mpeg2video, yuv420p, 480x480, 8000
kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 256 kb/s
#0 PTS:36037 Delta: 0.000000ms queue: 21
#1 PTS:35649 Delta: 4.311111ms queue: 2
Mux rate: 8.39 Mbit/s
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 9 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 10 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 11 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 0 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 1 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 9 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 10 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 11 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 0 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 1 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 9 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 10 to an I-frame ((null))
Warning, QMAT_SHIFT is larger then 21, overflows possible
Converting frame 11 to an I-frame ((null))

--
Robin Gilks

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 16, 2005, 5:33 AM

Post #38 of 106 (2814 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> Have you tried replacing the malloc with a memalign(16,
> outbuf_size)?
Yeah I had tried that yesterday and it was still giving me the segv,
so I started digging into libavcodec since it seemed to be the problem.
About 30 minutes into it, I went digging in the svn log for the
mpegvideo_mmx_template.c thinking I was seeing a regression, then saw
the changes in HEAD. So it turned out to be not mpeg2fix related at all.

mpeg2fix-0.10 works great, no infinite loops, no segfaults. Just
QMAT_SHIFTs. I'll check the quality on the resultant cuts later today
for blockiness or other artifacts. Thanks for all your work so far!
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 5:52 AM

Post #39 of 106 (2811 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Adam Egger wrote:
> On 11/16/05, Geoffrey Hausheer <mythtv0368 [at] phracturedblue> wrote:
>
>> Sorry for the rapid fire relases, but here is 0.11:
>> http://www.pblue.org/myth/mpeg2fix-0.11.tgz
>>
>> It includes:
>> Correct renumbering of frames after a cut-point
>> Do not insert 1st frame if cutlist starts at 0
>> Minor code cleanups
>>
I still can't compile it. Do I need bleeding edge SVN for this? I'm
still on 7738, but I could get a side-copy of SVN, to try. Off we go... :)

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 6:08 AM

Post #40 of 106 (2825 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Adam Egger wrote:
> On 11/16/05, Tom Lichti <tom [at] redpepperracing> wrote:
>
>
>> I still can't compile it. Do I need bleeding edge SVN for this? I'm
>> still on 7738, but I could get a side-copy of SVN, to try. Off we go... :)
>>
>
> Just send the error messages you get.
I did that yesterday. For brevity:

root [at] mytht:~/test/mpeg2fix-0.11# make
g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
-L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
-lpthread -lqt-mt -lm
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCLoadQMatrix'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCSyncSurface'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCFlushSurface'
/usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
`XvMCBeginSurface'
collect2: ld returned 1 exit status
make: *** [mpeg2fix] Error 1

> Did you put the correct path to
> the svn myth in MAKELINKS?
>
Yes:

root [at] mytht:~/test/mpeg2fix-0.11# ls -l
lrwxrwxrwx 1 root root 40 Nov 16 09:51 libavcodec ->
/var/lib/mythcvs/mythtv/libs//libavcodec
lrwxrwxrwx 1 root root 41 Nov 16 09:51 libavformat ->
/var/lib/mythcvs/mythtv/libs//libavformat
lrwxrwxrwx 1 root root 39 Nov 16 09:51 libavutil ->
/var/lib/mythcvs/mythtv/libs//libavutil
lrwxrwxrwx 1 root root 42 Nov 16 09:51 libmpeg2 ->
/var/lib/mythcvs/mythtv/libs//libmythmpeg2

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 16, 2005, 6:16 AM

Post #41 of 106 (2812 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> mpeg2fix-0.10 works great, no infinite loops, no segfaults. Just
> QMAT_SHIFTs. I'll check the quality on the resultant cuts later today for
> blockiness or other artifacts. Thanks for all your work so far!

I checked 0.10, and you were right... the fix for the error with
"Need to insert 7394 frames > max allowed: 20" is no longer there.

The "blockiness" is still present, though. Not really blockiness
as it is frame in != frame out. The .yuv files look like the original.
The I-frame inserted looks like it's encoded differently... like without
interlacing, and coarser quantization. I tried changing the c->qmax = 1
but had not discernable difference.

Still have the
"Warning, QMAT_SHIFT is larger then 21, overflows possible" as well.

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 16, 2005, 6:18 AM

Post #42 of 106 (2808 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Sorry for the rapid fire relases, but here is 0.11:
http://www.pblue.org/myth/mpeg2fix-0.11.tgz

It includes:
Correct renumbering of frames after a cut-point
Do not insert 1st frame if cutlist starts at 0
Minor code cleanups

The only thing that I am sure is broken is what will happen on a PTS
wrap. That will actually be kinda painful to fix (and difficult to
verify in the general case), but it is my next task. I also need to
reenable the PTS discrepency code at some point, but I'm thinking I
may wait on that, as it really needs to be rethought out.

Anyhow, unless any additional bugs pop up, 0.12 (the PTS wrap thing)
will likely be the last release before I roll it into myth.

There are still many unsupported features, but that stuff can be added
in the future and won't gate most users.

By the way, I've looked into the QMAT_SHIFT thing, and it could be a
real issue in some cases, but I have no idea what to do about it.
What is happening is that the quantization coefficients from the
original stream are too large (for libavcodec), resulting in a
possible overflow during encoding. This can happen if a stream uses a
non-standard quantization matrix (which many do). As there is no
option to change the coefficients, there's not much else I can do
about it. However, if the reported number is 21, there may not be a
problem, as that indicates a signed overflow, which may recover ok,
depending on how the code is written (I need to look into it some
more). Also, if you use SSE, you may be ok, as the shift is only 16
in that case, which should never cause an issue.

Again, thanks for all the help testing and debugging.
.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 6:22 AM

Post #43 of 106 (2810 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Cory Papenfuss wrote:
> The "blockiness" is still present, though. Not really blockiness
> as it is frame in != frame out. The .yuv files look like the original.
> The I-frame inserted looks like it's encoded differently... like without
> interlacing, and coarser quantization. I tried changing the c->qmax = 1
> but had not discernable difference.
I can see it too, I'm just not sure what to do about it. Either I am
not setting up the encoder corrctly, or it can't process your streams
correctly, but in either case, I'm not sure how I can make it any
better. I'd recomend playing with ffmpeg. If you can find settings
that make the image look good there, I can incorporate them into the
encoder.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

vijay.s.gill at gmail

Nov 16, 2005, 6:43 AM

Post #44 of 106 (2808 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Hi,

Sorry for barging in, but there is a project called mpeg2cut2

http://www.geocities.com/rocketjet4/

I have used this for quite some time on windows. And the source code
is available too. See if some part of it can be taken out to make it
command line and linux compatible. I am just giving an idea.

Regards from

Vijay Gill
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv2005 at bdam

Nov 16, 2005, 6:47 AM

Post #45 of 106 (2819 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Geoffrey Hausheer <mythtv0368 [at] phracturedblue> wrote:
> Sorry for the rapid fire relases, but here is 0.11:
> http://www.pblue.org/myth/mpeg2fix-0.11.tgz
>
> It includes:
> Correct renumbering of frames after a cut-point
> Do not insert 1st frame if cutlist starts at 0
> Minor code cleanups

I unfortunately still can't use it. It still crashes here with 0.11
after a few seconds. Do you need more information?

./mpeg2fix -i /big/myth/record/File\ -\ 2005-11-06\,\ 10-19\ AM.mpg -o
./temp.mpg
Input #0, mpegts, from '/big/myth/record/File - 2005-11-06, 10-19 AM.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x131], 25.00 fps: Video: mpeg2video, yuv420p, 720x576, 15000 kb/s
Stream #0.1[0x132](deu): Audio: mp2, 48000 Hz, stereo, 192 kb/s
Stream #0.2[0x138](deu): Audio: ac3, 48000 Hz, 5:1, 448 kb/s
#0 PTS:3782606226 Delta: 0.000000ms queue: 14
#1 PTS:3782605700 Delta: 5.844444ms queue: 4
#2 PTS:3782605833 Delta: 4.366667ms queue: 2
Mux rate: 15.90 Mbit/s

STATE: 4 (833e301) W: 720 H: 576 P: 1080000 4:24:29.7 T:2 F: 2
STATE: 4 (83a1980) W: 720 H: 576 P: 1080000 T:0 F: 2
STATE: 4 (853307b) W: 720 H: 576 P: 1080000 T:1 F: 2
STATE: 4 (84e881a) W: 720 H: 576 P: 1080000 T:5 F: 2
STATE: 4 (8244f1b) W: 720 H: 576 P: 1080000 T:3 F: 2
STATE: 4 (b5e21021) W: 720 H: 576 P: 1080000 T:4 F: 2
STATE: 4 (8400c8b) W: 720 H: 576 P: 1080000 T:8 F: 2
STATE: 4 (851802b) W: 720 H: 576 P: 1080000 T:6 F: 2
STATE: 4 (83c3883) W: 720 H: 576 P: 1080000 T:7 F: 2
STATE: 4 (8290b0b) W: 720 H: 576 P: 1080000 T:11 F: 2
STATE: 4 (84986c8) W: 720 H: 576 P: 1080000 T:9 F: 2
STATE: 4 (818cb01) W: 720 H: 576 P: 1080000 T:10 F: 2
STATE: 4 (836c84a) W: 720 H: 576 P: 1080000 4:24:29.19 T:2 F: 2
STATE: 4 (8542ba3) W: 720 H: 576 P: 1080000 T:0 F: 2
STATE: 4 (8250d1a) W: 720 H: 576 P: 1080000 T:1 F: 2
STATE: 4 (82eaab0) W: 720 H: 576 P: 1080000 T:5 F: 2
STATE: 4 (850b62b) W: 720 H: 576 P: 1080000 T:3 F: 2
STATE: 4 (84290a1) W: 720 H: 576 P: 1080000 T:4 F: 2
STATE: 4 (83ad672) W: 720 H: 576 P: 1080000 T:8 F: 2
STATE: 4 (832fcf8) W: 720 H: 576 P: 1080000 T:6 F: 2
STATE: 4 (837be8b) W: 720 H: 576 P: 1080000 T:7 F: 2
STATE: 4 (829b8c8) W: 720 H: 576 P: 1080000 T:11 F: 2
STATE: 4 (846eba3) W: 720 H: 576 P: 1080000 T:9 F: 2
STATE: 4 (83f4eb1) W: 720 H: 576 P: 1080000 T:10 F: 2
STATE: 4 (b5e21082) W: 720 H: 576 P: 1080000 4:24:30.6 T:2 F: 2
STATE: 0 (8374595) W: 720 H: 576 P: 1080000 4:24:29.19 T:2 F: 2
STATE: 0 (8543db2) W: 720 H: 576 P: 1080000 4:24:29.19 T:0 F: 2
STATE: 0 (825214a) W: 720 H: 576 P: 1080000 4:24:29.19 T:1 F: 2
STATE: 0 (82ee515) W: 720 H: 576 P: 1080000 4:24:29.19 T:5 F: 2
STATE: 0 (850cb92) W: 720 H: 576 P: 1080000 4:24:29.19 T:3 F: 2
STATE: 0 (842a4ea) W: 720 H: 576 P: 1080000 4:24:29.19 T:4 F: 2
STATE: 0 (83b1255) W: 720 H: 576 P: 1080000 4:24:29.19 T:8 F: 2
STATE: 0 (83310b2) W: 720 H: 576 P: 1080000 4:24:29.19 T:6 F: 2
STATE: 0 (837d652) W: 720 H: 576 P: 1080000 4:24:29.19 T:7 F: 2
STATE: 0 (829f3ad) W: 720 H: 576 P: 1080000 4:24:29.19 T:11 F: 2
STATE: 0 (846fef2) W: 720 H: 576 P: 1080000 4:24:29.19 T:9 F: 2
*** glibc detected *** double free or corruption (out): 0x083a1470 ***
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 6:51 AM

Post #46 of 106 (2810 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> On 11/16/05, Tom Lichti wrote:
>
>> root [at] mytht:~/test/mpeg2fix-0.11# make
>> g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
>> helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
>> -L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
>> -lpthread -lqt-mt -lm
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCLoadQMatrix'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCSyncSurface'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCFlushSurface'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCBeginSurface'
>>
> I can't help you with these (except to say that they'll go away once
> the code is part of myth). you need to add -lXv (or something) to the
> LIBS line in Makefile. Since I don't use Xv, I don't know which
> library you need.
>
>
Well, I compiled and installed a separate copy of latest SVN, but it is
still looking at the older libmythavcodec. I installed to
/var/lib/mythcvs/latest/install, but the error shows it going to my
regular /usr/local install path.

I'm going to recompile my old version without XvMC and see what happens...

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv2005 at bdam

Nov 16, 2005, 6:56 AM

Post #47 of 106 (2812 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Tom Lichti <tom [at] redpepperracing> wrote:

> I still can't compile it. Do I need bleeding edge SVN for this? I'm
> still on 7738, but I could get a side-copy of SVN, to try. Off we go... :)

Just send the error messages you get. Did you put the correct path to
the svn myth in MAKELINKS?
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 7:32 AM

Post #48 of 106 (2827 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Adam Egger wrote:
> I unfortunately still can't use it. It still crashes here with 0.11
> after a few seconds. Do you need more information?
>
> ./mpeg2fix -i /big/myth/record/File\ -\ 2005-11-06\,\ 10-19\ AM.mpg -o
> ./temp.mpg
> Input #0, mpegts, from '/big/myth/record/File - 2005-11-06, 10-19 AM.mpg':
> Duration: N/A, bitrate: N/A
> Stream #0.0[0x131], 25.00 fps: Video: mpeg2video, yuv420p, 720x576, 15000 kb/s
> Stream #0.1[0x132](deu): Audio: mp2, 48000 Hz, stereo, 192 kb/s
> Stream #0.2[0x138](deu): Audio: ac3, 48000 Hz, 5:1, 448 kb/s
> #0 PTS:3782606226 Delta: 0.000000ms queue: 14
> #1 PTS:3782605700 Delta: 5.844444ms queue: 4
> #2 PTS:3782605833 Delta: 4.366667ms queue: 2
> Mux rate: 15.90 Mbit/s
>
> STATE: 4 (833e301) W: 720 H: 576 P: 1080000 4:24:29.7 T:2 F: 2
...
Yes, this isn't useful. Not sure how you are getting STATE messages
without seeting the debug switch. If you actually set dbg_level
inside the code, it is telling ithat you are never getting into the
PTS detection code. You should be seeing lines saying 'VID ....'
Especially since multiple I frames have passed.

But I'd need a gdb trace to actually make any progress on this (or
alternatively the first 2 megs of your video).

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 7:36 AM

Post #49 of 106 (2817 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Tom Lichti wrote:
> root [at] mytht:~/test/mpeg2fix-0.11# make
> g++ -g -O0 -fno-inline -fno-default-inline -o mpeg2fix mpeg2fix.o
> helper.o replex/libreplex.a -L./libavformat -L./libavcodec -L./libavutil
> -L./libmpeg2 -L./replex -lmythavformat-0.18 -lmythmpeg2-0.18 -lreplex
> -lpthread -lqt-mt -lm
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCLoadQMatrix'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCSyncSurface'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCFlushSurface'
> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
> `XvMCBeginSurface'
I can't help you with these (except to say that they'll go away once
the code is part of myth). you need to add -lXv (or something) to the
LIBS line in Makefile. Since I don't use Xv, I don't know which
library you need.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 7:41 AM

Post #50 of 106 (2813 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Vijay Gill wrote:
> Hi,
>
> Sorry for barging in, but there is a project called mpeg2cut2
>
> http://www.geocities.com/rocketjet4/
>
> I have used this for quite some time on windows. And the source code
> is available too. See if some part of it can be taken out to make it
> command line and linux compatible. I am just giving an idea.
>
Thanks, but this is not very useful. It appears to be a GOP level
clipping program. It is not frame accurate. Feel free to port it to
linux, but gopchop already does a lot of what this seems to do. The
goals of the mpeg2fix (what will be mythtranscode) are different, in
that it is being designed to (a) turn a non-compliant stream into a
compliant one (to some extent), and (b) do frame-accurate cutting.

Everyone needs to scratch their own itch. Mine is mostly related to
playing all of my broken streams on my RokuHD, and the cutting code is
just a reasonably minor addition to make everyone else happy.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 7:42 AM

Post #51 of 106 (2548 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Bryan Mayland wrote:
> Geoffrey Hausheer wrote:
>>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>> `XvMCLoadQMatrix'
>>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>> `XvMCSyncSurface'
>>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>> `XvMCFlushSurface'
>>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>>> `XvMCBeginSurface'
>>>
>> I can't help you with these (except to say that they'll go away once
>> the code is part of myth). you need to add -lXv (or something) to the
>> LIBS line in Makefile. Since I don't use Xv, I don't know which
>> library you need.
>>
> Actually, it is LIBS += -lXvMCW (generally). Some people still
> can't get it to link even with the lib reference, but I don't know why
> because I just adding the lib works for me.
>
Tried that as well, didn't help. I've just rebuilt myth without XvMC,
and mpeg2fix compiled! Yay.

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 8:04 AM

Post #52 of 106 (2534 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> On 11/16/05, Tom Lichti wrote:
>
>> Well, I compiled and installed a separate copy of latest SVN, but it is
>> still looking at the older libmythavcodec. I installed to
>> /var/lib/mythcvs/latest/install, but the error shows it going to my
>> regular /usr/local install path.
>>
>> I'm going to recompile my old version without XvMC and see what happens...
>>
>>
> do you have LD_LIBRARY_PATH set? ld uses it during linking to find
> the relevant .so files. If it is set, clear it. if it isn't set, set
> it to point at the libs you want to use. That should fix ld using the
> wrong libs, though perhaps not the actual issue. You need to keep it
> set when running mpeg2fix, though, otehrwise it'll revert back to the
> default search path.
>
>
Okay, cool. I'll try that if what I'm doing doesn't work.

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 16, 2005, 8:14 AM

Post #53 of 106 (2534 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCLoadQMatrix'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCSyncSurface'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCFlushSurface'
>> /usr/local/lib/libmythavcodec-0.18.so.0: undefined reference to
>> `XvMCBeginSurface'
>>
> I can't help you with these (except to say that they'll go away once
> the code is part of myth). you need to add -lXv (or something) to the
> LIBS line in Makefile. Since I don't use Xv, I don't know which
> library you need.
>
Actually, it is LIBS += -lXvMCW (generally). Some people still
can't get it to link even with the lib reference, but I don't know why
because I just adding the lib works for me.

What's the right way to view the cnv*.yuv files? I've tried
yuvtoppm 640 480 < cnv0.yuv | ppmtopng > cnv0.png, but I get a green,
red, and black frame (of the proper size though). I'm assuming it is
because the yuv isn't laid out in the way yuvtoppm is expecting, but
can't find an imagemagic tool that will convert it.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 16, 2005, 8:20 AM

Post #54 of 106 (2532 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Wed, 16 Nov 2005, Geoffrey Hausheer wrote:

> On 11/16/05, Cory Papenfuss wrote:
>> The "blockiness" is still present, though. Not really blockiness
>> as it is frame in != frame out. The .yuv files look like the original.
>> The I-frame inserted looks like it's encoded differently... like without
>> interlacing, and coarser quantization. I tried changing the c->qmax = 1
>> but had not discernable difference.
> I can see it too, I'm just not sure what to do about it. Either I am
> not setting up the encoder corrctly, or it can't process your streams
> correctly, but in either case, I'm not sure how I can make it any
> better. I'd recomend playing with ffmpeg. If you can find settings
> that make the image look good there, I can incorporate them into the
> encoder.
>
> .Geoff
>
I have looked visually carefully at the .yuv, .yuv.enc, and output
mpeg with ffplay and I believe that the decode and encode are the same.
Once it's frame-frozen at the same generated I-frame in the output stream,
it looks different, however. Especially regarding the interlacing. Is it
possible that top/bottom fields got swapped once it got muxed in the
stream?

Another strange one. Doing basically nothing with a stream except
cleaning. It hangs towards the end at the same place (unbuffered
output at debug 4)

$mpeg2fix -d 4 -i cory_grad_luke_conf_alton.nuv -o test2.mpg
<the big snip>
VID: B #:12 nb: 2 pts: 609414887 dts: 609414887 pos: 8acd000
Id:0: 1:52:51.276 V:62669 3871 A:674123 8799
PTS discrepency: 609421373 != 609422669 on B-Type (14)
PTS discrepency: 609424621 != 609425672 on B-Type (15)
PTS discrepency: 609427864 != 609428675 on P-Type (16)
VID: P #:16 nb: 2 pts: 609427864 dts: 609419666 pos: 86a32b8
Id:0: 1:52:51.420 V:39117 3827 A:674123 8799
VID: B #:14 nb: 2 pts: 609421373 dts: 609421373 pos: 89213d8
Id:0: 1:52:51.348 V:11797 3783 A:674123 8799

When I ran it through gdb --args , 'run', it finished!

$ du -sk cory_grad_luke_conf_alton.nuv
4621744 cory_grad_luke_conf_alton.nuv
$ du -sk test2.mpg
3945220 test2.mpg

That's as far as it gets.

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 16, 2005, 8:24 AM

Post #55 of 106 (2527 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Tom Lichti wrote:
> Well, I compiled and installed a separate copy of latest SVN, but it is
> still looking at the older libmythavcodec. I installed to
> /var/lib/mythcvs/latest/install, but the error shows it going to my
> regular /usr/local install path.
>
> I'm going to recompile my old version without XvMC and see what happens...
>
do you have LD_LIBRARY_PATH set? ld uses it during linking to find
the relevant .so files. If it is set, clear it. if it isn't set, set
it to point at the libs you want to use. That should fix ld using the
wrong libs, though perhaps not the actual issue. You need to keep it
set when running mpeg2fix, though, otehrwise it'll revert back to the
default search path.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 8:25 AM

Post #56 of 106 (2532 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Bryan Mayland wrote:
> What's the right way to view the cnv*.yuv files? I've tried
> yuvtoppm 640 480 < cnv0.yuv | ppmtopng > cnv0.png, but I get a green,
> red, and black frame (of the proper size though). I'm assuming it is
> because the yuv isn't laid out in the way yuvtoppm is expecting, but
> can't find an imagemagic tool that will convert it.

I use imagemagik.

display -size 640x480 cnv0.yuv

you can see the .enc files by doing somthing like:
cat cnv0.yuv.enc
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 16, 2005, 8:27 AM

Post #57 of 106 (2540 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> What's the right way to view the cnv*.yuv files? I've tried yuvtoppm 640
> 480 < cnv0.yuv | ppmtopng > cnv0.png, but I get a green, red, and black frame
> (of the proper size though). I'm assuming it is because the yuv isn't laid
> out in the way yuvtoppm is expecting, but can't find an imagemagic tool that
> will convert it.

Zen-master-mpegger Geoff suggested this for me awhile back:
display -size 720x480 cnv0.yuv

cat cnv0.yuv.enc cnv0.yuv.enc > out.m2v
ffplay out.m2v

... if you just do one frame you only see black... apparently it
gets buffered.

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 16, 2005, 8:29 AM

Post #58 of 106 (2548 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Geoffrey Hausheer wrote
> I use imagemagik.
>
> display -size 640x480 cnv0.yuv
>
> you can see the .enc files by doing somthing like:
> cat cnv0.yuv.enc
>
oops...hit the wrong key.

cat cnv0.yuv.enc cnv0.yuv.enc cnv0.yuv.enc > cnv0.m2v
ffplay cnv0.m2v
(mplayer may work as well). This isn't gauranteed to work, but should
in most cases. In 0.12 I'll include a decoded yuv of the output as
well.

Just to be clear:
the cnv0.yuv is the frame that is being re-encoded. the cnv0.yuv.enc
is the re-encoded-frame (it'll be an 'I' frame with a sequence
header).

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv2005 at bdam

Nov 16, 2005, 8:30 AM

Post #59 of 106 (2542 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Geoffrey Hausheer <mythtv0368 [at] phracturedblue> wrote:
> On 11/16/05, Adam Egger wrote:
> > I unfortunately still can't use it. It still crashes here with 0.11
> > after a few seconds. Do you need more information?
> >
> > ./mpeg2fix -i /big/myth/record/File\ -\ 2005-11-06\,\ 10-19\ AM.mpg -o
> > ./temp.mpg
> > Input #0, mpegts, from '/big/myth/record/File - 2005-11-06, 10-19 AM.mpg':
> > Duration: N/A, bitrate: N/A
> > Stream #0.0[0x131], 25.00 fps: Video: mpeg2video, yuv420p, 720x576, 15000 kb/s
> > Stream #0.1[0x132](deu): Audio: mp2, 48000 Hz, stereo, 192 kb/s
> > Stream #0.2[0x138](deu): Audio: ac3, 48000 Hz, 5:1, 448 kb/s
> > #0 PTS:3782606226 Delta: 0.000000ms queue: 14
> > #1 PTS:3782605700 Delta: 5.844444ms queue: 4
> > #2 PTS:3782605833 Delta: 4.366667ms queue: 2
> > Mux rate: 15.90 Mbit/s
> >
> > STATE: 4 (833e301) W: 720 H: 576 P: 1080000 4:24:29.7 T:2 F: 2
> ...
> Yes, this isn't useful. Not sure how you are getting STATE messages
> without seeting the debug switch. If you actually set dbg_level
> inside the code, it is telling ithat you are never getting into the
> PTS detection code. You should be seeing lines saying 'VID ....'
> Especially since multiple I frames have passed.
>
> But I'd need a gdb trace to actually make any progress on this (or
> alternatively the first 2 megs of your video).

Sorry I apparently removed the used -d4 option in my last posting.
That's what gbd shows me now:

STATE: 4 (8244f1b) W: 720 H: 576 P: 1080000 T:3 F: 2

STATE: 4 (b5e5a021) W: 720 H: 576 P: 1080000 T:4 F: 2

STATE: 4 (8400c8b) W: 720 H: 576 P: 1080000 T:8 F: 2

STATE: 4 (851802b) W: 720 H: 576 P: 1080000 T:6 F: 2

STATE: 4 (83c3883) W: 720 H: 576 P: 1080000 T:7 F: 2

STATE: 4 (8290b0b) W: 720 H: 576 P: 1080000 T:11 F: 2

STATE: 4 (84986c8) W: 720 H: 576 P: 1080000 T:9 F: 2

STATE: 4 (818cb01) W: 720 H: 576 P: 1080000 T:10 F: 2

STATE: 4 (836c84a) W: 720 H: 576 P: 1080000 4:24:29.19 T:2 F: 2

STATE: 4 (8542ba3) W: 720 H: 576 P: 1080000 T:0 F: 2

STATE: 4 (8250d1a) W: 720 H: 576 P: 1080000 T:1 F: 2

STATE: 4 (82eaab0) W: 720 H: 576 P: 1080000 T:5 F: 2

STATE: 4 (850b62b) W: 720 H: 576 P: 1080000 T:3 F: 2

STATE: 4 (84290a1) W: 720 H: 576 P: 1080000 T:4 F: 2

STATE: 4 (83ad672) W: 720 H: 576 P: 1080000 T:8 F: 2

STATE: 4 (832fcf8) W: 720 H: 576 P: 1080000 T:6 F: 2

STATE: 4 (837be8b) W: 720 H: 576 P: 1080000 T:7 F: 2

STATE: 4 (829b8c8) W: 720 H: 576 P: 1080000 T:11 F: 2

STATE: 4 (846eba3) W: 720 H: 576 P: 1080000 T:9 F: 2

STATE: 4 (83f4eb1) W: 720 H: 576 P: 1080000 T:10 F: 2

STATE: 4 (b5e5a082) W: 720 H: 576 P: 1080000 4:24:30.6 T:2 F: 2

STATE: 0 (8374595) W: 720 H: 576 P: 1080000 4:24:29.19 T:2 F: 2

STATE: 0 (8543db2) W: 720 H: 576 P: 1080000 4:24:29.19 T:0 F: 2

STATE: 0 (825214a) W: 720 H: 576 P: 1080000 4:24:29.19 T:1 F: 2

STATE: 0 (82ee515) W: 720 H: 576 P: 1080000 4:24:29.19 T:5 F: 2

STATE: 0 (850cb92) W: 720 H: 576 P: 1080000 4:24:29.19 T:3 F: 2

STATE: 0 (842a4ea) W: 720 H: 576 P: 1080000 4:24:29.19 T:4 F: 2

STATE: 0 (83b1255) W: 720 H: 576 P: 1080000 4:24:29.19 T:8 F: 2

STATE: 0 (83310b2) W: 720 H: 576 P: 1080000 4:24:29.19 T:6 F: 2

STATE: 0 (837d652) W: 720 H: 576 P: 1080000 4:24:29.19 T:7 F: 2

STATE: 0 (829f3ad) W: 720 H: 576 P: 1080000 4:24:29.19 T:11 F: 2

STATE: 0 (846fef2) W: 720 H: 576 P: 1080000 4:24:29.19 T:9 F: 2

*** glibc detected *** double free or corruption (out): 0x083a1470 ***

Program received signal SIGABRT, Aborted.
[Switching to Thread -1225319584 (LWP 28235)]
0xb75047a7 in raise () from /lib/tls/libc.so.6
(gdb) bt
#0 0xb75047a7 in raise () from /lib/tls/libc.so.6
#1 0xb750604b in abort () from /lib/tls/libc.so.6
#2 0xb753b005 in __fsetlocking () from /lib/tls/libc.so.6
#3 0xb7541657 in malloc_usable_size () from /lib/tls/libc.so.6
#4 0xb7541af2 in free () from /lib/tls/libc.so.6
#5 0x0804c9ea in MPEG2fixup::build_frame (this=0xbfea3bc0,
pkt=0xbfea378c, fname=0xbfea37c0 "cnv0.yuv")
at mpeg2fix.cpp:956
#6 0x0804d105 in MPEG2fixup::convert_to_i (this=0xbfea3bc0,
frameNum=9, numFrames=3, headPos=9) at mpeg2fix.cpp:1222
#7 0x080507d0 in MPEG2fixup::start (this=0xbfea3bc0) at mpeg2fix.cpp:1494
#8 0x08051bfd in main (argc=14, argv=0xbfea4f34) at mpeg2fix.cpp:1806
(gdb) thread apply all bt

Thread 2 (Thread -1227773008 (LWP 28241)):
#0 0xb7f06b91 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib/tls/libpthread.so.0
#1 0x0804bf0a in MPEG2replex::start (this=0xbfea3bc0) at mpeg2fix.cpp:425
#2 0x0804c0aa in MPEG2fixup::replex_start (data=0xbfea3bc0) at mpeg2fix.cpp:391
#3 0xb7f04cfd in start_thread () from /lib/tls/libpthread.so.0
#4 0xb75ac13e in clone () from /lib/tls/libc.so.6

Thread 1 (Thread -1225319584 (LWP 28235)):
#0 0xb75047a7 in raise () from /lib/tls/libc.so.6
#1 0xb750604b in abort () from /lib/tls/libc.so.6
#2 0xb753b005 in __fsetlocking () from /lib/tls/libc.so.6
#3 0xb7541657 in malloc_usable_size () from /lib/tls/libc.so.6
#4 0xb7541af2 in free () from /lib/tls/libc.so.6
#5 0x0804c9ea in MPEG2fixup::build_frame (this=0xbfea3bc0,
pkt=0xbfea378c, fname=0xbfea37c0 "cnv0.yuv")
at mpeg2fix.cpp:956
#6 0x0804d105 in MPEG2fixup::convert_to_i (this=0xbfea3bc0,
frameNum=9, numFrames=3, headPos=9) at mpeg2fix.cpp:1222
#7 0x080507d0 in MPEG2fixup::start (this=0xbfea3bc0) at mpeg2fix.cpp:1494
#8 0x08051bfd in main (argc=14, argv=0xbfea4f34) at mpeg2fix.cpp:1806
(gdb)
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

paulrwheeler at gmail

Nov 16, 2005, 9:20 AM

Post #60 of 106 (2529 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey,

Did you ever have any luck with getting the subtitles back in? Will
this be too difficult/time consuming to do? Are you planning to look
into it at all?

Paul

P.S. I should be able to test the code again at the weekend to see if
have any probs with it.

On 11/16/05, Tom Lichti <tom [at] redpepperracing> wrote:
> Geoffrey Hausheer wrote:
> > On 11/16/05, Tom Lichti wrote:
> >
> >> Well, I compiled and installed a separate copy of latest SVN, but it is
> >> still looking at the older libmythavcodec. I installed to
> >> /var/lib/mythcvs/latest/install, but the error shows it going to my
> >> regular /usr/local install path.
> >>
> >> I'm going to recompile my old version without XvMC and see what happens...
> >>
> >>
> > do you have LD_LIBRARY_PATH set? ld uses it during linking to find
> > the relevant .so files. If it is set, clear it. if it isn't set, set
> > it to point at the libs you want to use. That should fix ld using the
> > wrong libs, though perhaps not the actual issue. You need to keep it
> > set when running mpeg2fix, though, otehrwise it'll revert back to the
> > default search path.
> >
> >
> Okay, cool. I'll try that if what I'm doing doesn't work.
>
> Thanks
> Tom
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
>
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 10:12 AM

Post #61 of 106 (2534 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Adam Egger wrote:
>
> STATE: 4 (b5e5a082) W: 720 H: 576 P: 1080000 4:24:30.6 T:2 F: 2
> STATE: 0 (8374595) W: 720 H: 576 P: 1080000 4:24:29.19 T:2 F: 2
> STATE: 0 (8543db2) W: 720 H: 576 P: 1080000 4:24:29.19 T:0 F: 2
> STATE: 0 (825214a) W: 720 H: 576 P: 1080000 4:24:29.19 T:1 F: 2
> STATE: 0 (82ee515) W: 720 H: 576 P: 1080000 4:24:29.19 T:5 F: 2
> STATE: 0 (850cb92) W: 720 H: 576 P: 1080000 4:24:29.19 T:3 F: 2
> STATE: 0 (842a4ea) W: 720 H: 576 P: 1080000 4:24:29.19 T:4 F: 2
> STATE: 0 (83b1255) W: 720 H: 576 P: 1080000 4:24:29.19 T:8 F: 2
> STATE: 0 (83310b2) W: 720 H: 576 P: 1080000 4:24:29.19 T:6 F: 2
> STATE: 0 (837d652) W: 720 H: 576 P: 1080000 4:24:29.19 T:7 F: 2
> STATE: 0 (829f3ad) W: 720 H: 576 P: 1080000 4:24:29.19 T:11 F: 2
> STATE: 0 (846fef2) W: 720 H: 576 P: 1080000 4:24:29.19 T:9 F: 2
>
> *** glibc detected *** double free or corruption (out): 0x083a1470 ***
>
> Program received signal SIGABRT, Aborted.
> [Switching to Thread -1225319584 (LWP 28235)]
> 0xb75047a7 in raise () from /lib/tls/libc.so.6
> (gdb) bt
> #0 0xb75047a7 in raise () from /lib/tls/libc.so.6
> #1 0xb750604b in abort () from /lib/tls/libc.so.6
> #2 0xb753b005 in __fsetlocking () from /lib/tls/libc.so.6
> #3 0xb7541657 in malloc_usable_size () from /lib/tls/libc.so.6
> #4 0xb7541af2 in free () from /lib/tls/libc.so.6
> #5 0x0804c9ea in MPEG2fixup::build_frame (this=0xbfea3bc0,
> pkt=0xbfea378c, fname=0xbfea37c0 "cnv0.yuv")
> at mpeg2fix.cpp:956
> #6 0x0804d105 in MPEG2fixup::convert_to_i (this=0xbfea3bc0,
> frameNum=9, numFrames=3, headPos=9) at mpeg2fix.cpp:1222
> #7 0x080507d0 in MPEG2fixup::start (this=0xbfea3bc0) at mpeg2fix.cpp:1494
> #8 0x08051bfd in main (argc=14, argv=0xbfea4f34) at mpeg2fix.cpp:1806

This is odd. 1st I really need to see your full command line, as you
appear to be applying a commercial-cut here, but I don't know what you
actually chose.

2nd, while this debug is more interesting, it isn't very enlightening.
There is really no way that I can see to get a double free on the
codecContext (the free atline 956), which implies, perhaps, a stack
overrun. I wonder if this is a segfault from libavcodec similar to
what Bryan saw. What version of SVN are you running?
You could try setting a break point at line 939 (at the
avcodec_encode_video call) and doing something like this before and
after the call:
print c
print outbuf
print picture
print out_size
print outbuf_size
print *c
print *picture

if any of the 1st 3 are different (the 0x????????, not any data
afterwards), that is definitely stack corruption. The 6th and 7th
commands will dump out a lot of text. I'm not particularly interested
in the values, but you should spot-check that there are no whacky
numbers.
after that step through and ensure that it still dies on the 'free(c)'

You could also try disabling all SSE, SSE2, etc by adding something
like this at line
938 (before the encode_video call):
c->dsp_mask = 0xffff;
recompiling and rerun.

Sorry I can't be of more help than that, but maybe something above
will help us make some progress.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 10:15 AM

Post #62 of 106 (2529 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Paul Wheeler wrote:
> Geoffrey,
>
> Did you ever have any luck with getting the subtitles back in? Will
> this be too difficult/time consuming to do? Are you planning to look
> into it at all?
>
No this hasn't happened, and probably won't until after the code is
merged with mythtranscode. It shouldn't be TOO difficult, as there
are only minor changes in my code to handle it. The real problem will
be supportingit in libreplex (no support at all right now), and from
what I understand, there are several different subtitle encoding
methods, which might make supporting it there trickier. I also don't
know if I have access to the relevant documentation on how they are
coded, which would certainly make it a lot harder to implement. I'll
have to dig around and see what I have.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 10:32 AM

Post #63 of 106 (2530 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> On 11/16/05, Tom Lichti wrote:
>
>> Well, I compiled and installed a separate copy of latest SVN, but it is
>> still looking at the older libmythavcodec. I installed to
>> /var/lib/mythcvs/latest/install, but the error shows it going to my
>> regular /usr/local install path.
>>
>> I'm going to recompile my old version without XvMC and see what happens...
>>
>>
> do you have LD_LIBRARY_PATH set? ld uses it during linking to find
> the relevant .so files. If it is set, clear it. if it isn't set, set
> it to point at the libs you want to use. That should fix ld using the
> wrong libs, though perhaps not the actual issue. You need to keep it
> set when running mpeg2fix, though, otehrwise it'll revert back to the
> default search path.
>
That seems to have done it. I just recompiled latest SVN without XvMC
support, and set the LD_LIBRARY_PATH, and it compiled and cropped the
first file I tried. I'll try and remember to try and play it tonight,
see how it looks. I bet I have a recording that will throw some
curveballs at the program, if you want me to test it some more... :)

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 12:50 PM

Post #64 of 106 (2535 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> On 11/16/05, Tom Lichti wrote:
>
>> I'll try and remember to try and play it tonight,
>> see how it looks. I bet I have a recording that will throw some
>> curveballs at the program, if you want me to test it some more... :)
>>
>
> Well testing is what we need more than anything else (and watching the
> results especially). I know of 3 issues at the moment: Adam's (which
> I am waiting on more debug info for, and otherwise can't reproduce),
> Cory's display issue which I can reproduce, but don't know how to fix
> (though the feedback he gave today may give a direction to go in), and
> also his hang at the end which appears to be some sort of race (very
> worrisome).
>
> If I can track down the 3rd, I probably won't gate the merge with the
> other 2 (unless Adam comes back with some more interesting info). So
> crashes, hangs, bad visuals, mismatched audio, etc. If you see any of
> this, please let me know.
>
> .Geoff
>
Will do. I managed to break my exisiting myth setup, but I think I've
got that under control again... :)

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 1:32 PM

Post #65 of 106 (2529 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Tom Lichti wrote:
> I'll try and remember to try and play it tonight,
> see how it looks. I bet I have a recording that will throw some
> curveballs at the program, if you want me to test it some more... :)

Well testing is what we need more than anything else (and watching the
results especially). I know of 3 issues at the moment: Adam's (which
I am waiting on more debug info for, and otherwise can't reproduce),
Cory's display issue which I can reproduce, but don't know how to fix
(though the feedback he gave today may give a direction to go in), and
also his hang at the end which appears to be some sort of race (very
worrisome).

If I can track down the 3rd, I probably won't gate the merge with the
other 2 (unless Adam comes back with some more interesting info). So
crashes, hangs, bad visuals, mismatched audio, etc. If you see any of
this, please let me know.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv2005 at bdam

Nov 16, 2005, 1:46 PM

Post #66 of 106 (2525 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Geoffrey Hausheer <mythtv0368 [at] phracturedblue> wrote:

> This is odd. 1st I really need to see your full command line, as you
> appear to be applying a commercial-cut here, but I don't know what you
> actually chose.

It doesn't make a different if I use -c or not. It always crashes:
./mpeg2fix -i /big/myth/record/Stromberg\ -\ 2005-11-06\,\ 10-19\
AM.mpg -o ./Stromberg.mpg -c "0 - 4065" -c "35067 - 47846" -c "55701 -
62226" -d 4

It crashes with all files I've tried so far. So it's not just this one.

> 2nd, while this debug is more interesting, it isn't very enlightening.
> There is really no way that I can see to get a double free on the
> codecContext (the free atline 956), which implies, perhaps, a stack
> overrun. I wonder if this is a segfault from libavcodec similar to
> what Bryan saw. What version of SVN are you running?
> You could try setting a break point at line 939 (at the
> avcodec_encode_video call) and doing something like this before and
> after the call:
> print c
> print outbuf
> print picture
> print out_size
> print outbuf_size
> print *c
> print *picture

Haven't tried this so far. I'm running the most current SVN compiled
with --enable-proc-opt.

> if any of the 1st 3 are different (the 0x????????, not any data
> afterwards), that is definitely stack corruption. The 6th and 7th
> commands will dump out a lot of text. I'm not particularly interested
> in the values, but you should spot-check that there are no whacky
> numbers.
> after that step through and ensure that it still dies on the 'free(c)'
>
> You could also try disabling all SSE, SSE2, etc by adding something
> like this at line
> 938 (before the encode_video call):
> c->dsp_mask = 0xffff;
> recompiling and rerun.

No difference, still crashing. I'm using a 64bit CPU (but a 32bit OS-
Debian sid). Are you sometimes in #mythtv-users/#mythtv? It could be
easier to find the solution via IRC

Adam aka beavis
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 16, 2005, 5:40 PM

Post #67 of 106 (2524 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> Well testing is what we need more than anything else (and watching the
> results especially). I know of 3 issues at the moment: Adam's (which
> I am waiting on more debug info for, and otherwise can't reproduce),
> Cory's display issue which I can reproduce, but don't know how to fix
> (though the feedback he gave today may give a direction to go in), and
> also his hang at the end which appears to be some sort of race (very
> worrisome).
>
I hate to say it, but I finally got a chance to look frame-by-frame
at my output after cuts and I'm seeing a lot of errors. I'm using a
MPEG2 PS captured from a PVR-250.
-- The areas around the cutpoints have blocks of incorrect color (err1)
-- Some frames are out of order. Like I've got err1, then a correct I
from after the cut, followed by err1 again.
-- After that, I get garbled output like err3 until the next I (which is
like err1+motion from after the cut).
What can I get you that will help you diagnose the problem? Would you
like the first 20MB or so from the file to play with?
http://capnbry.net/~bmayland/fi/pvr150/err1.jpg
http://capnbry.net/~bmayland/fi/pvr150/err2.jpg

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 6:40 PM

Post #68 of 106 (2538 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Bryan Mayland wrote:
> I hate to say it, but I finally got a chance to look frame-by-frame
> at my output after cuts and I'm seeing a lot of errors. I'm using a
> MPEG2 PS captured from a PVR-250.
> -- The areas around the cutpoints have blocks of incorrect color (err1)
> -- Some frames are out of order. Like I've got err1, then a correct I
> from after the cut, followed by err1 again.
> -- After that, I get garbled output like err3 until the next I (which is
> like err1+motion from after the cut).
I'd want the .yuv and corresponding .enc files after the cut (as I
said earlier, use imagemagik's 'display' to ensure you got the one you
want). I expect that the .yuv file will look ok (none of those
blocks), but if not, that is a bad sign right there.

you get them by using '-d 3' and you should send me 100 lines or so
before the corresponding 'Converting ...' line (and probably 100lines
after too for good measure). Also let me know if you are seeing those
QMAT_SHIFT messages.

If after all of that, I can't figure out what is going on, I could use
the 1st bit of your video, with a relevant cut point to duplicate the
issue. Or if you prefer, you could do both, and save some back and
forth e-mail.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 16, 2005, 7:17 PM

Post #69 of 106 (2542 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> I'd want the .yuv and corresponding .enc files after the cut (as I
> said earlier, use imagemagik's 'display' to ensure you got the one you
> want). I expect that the .yuv file will look ok (none of those
> blocks), but if not, that is a bad sign right there.
>
For simplicity sake, I only did one cut point in the file close to
the beginning.
http://capnbry.net/~bmayland/fi/pvr150/bmayland.mpeg2fix.tar.gz
In there are the yuv and enc files. One yuv looks like it has some bad
blocks in it (the bottom should not be green), one looks good (from
after the cut). The mp2f_cut.2 is a -d2 output from the whole 1.5GB
file, the mp2f_cut.3.cut is the first 2000 lines of -d3.
> you get them by using '-d 3' and you should send me 100 lines or so
> before the corresponding 'Converting ...' line (and probably 100lines
> after too for good measure). Also let me know if you are seeing those
> QMAT_SHIFT messages.
>
Yes, I'm getting QMAT_SHIFT warnings on these.
> If after all of that, I can't figure out what is going on, I could use
> the 1st bit of your video, with a relevant cut point to duplicate the
> issue. Or if you prefer, you could do both, and save some back and
> forth e-mail.
>
I've put the file up too, the first 20MB contains the area I cut in
this example. I've also tried cutting in a lot of other places in the
file just to test, with similar results. You may want to start this
downloading while you look at the other file, since this pipe is only
1/2 a T1. The end of the file might get messy since I just cut it with
a dd.
http://capnbry.net/~bmayland/fi/pvr150/testinsmall.mpg

I appreciate you looking at these, and feel free to email me
directly if you need anything else.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 16, 2005, 8:01 PM

Post #70 of 106 (2528 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>On 11/16/05, Tom Lichti wrote:
>
>
>>I'll try and remember to try and play it tonight,
>>see how it looks. I bet I have a recording that will throw some
>>curveballs at the program, if you want me to test it some more... :)
>>
>>
>
>Well testing is what we need more than anything else (and watching the
>results especially). I know of 3 issues at the moment: Adam's (which
>I am waiting on more debug info for, and otherwise can't reproduce),
>Cory's display issue which I can reproduce, but don't know how to fix
>(though the feedback he gave today may give a direction to go in), and
>also his hang at the end which appears to be some sort of race (very
>worrisome).
>
>If I can track down the 3rd, I probably won't gate the merge with the
>other 2 (unless Adam comes back with some more interesting info). So
>crashes, hangs, bad visuals, mismatched audio, etc. If you see any of
>this, please let me know.
>
>
Okay, I viewed the test file that I did this afternoon, and it's great,
but I am getting the blockiness after a cut that Cory was seeing. This
is from a PVR-250, PS format I believe.

On the file that I figured would be a problem, it bails after about 10MB
of data (out of 600MB total). I knew it would have problems with this
file. I have 2 PVR-250's, and I think one of them is bad, because it
generates blocky MPEG files, and the Mythfrontend spits out a lot of
error messages while playing. Do you want any specific output or
debugging with this, or should I just forget it, since I know the file
is bad?

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 16, 2005, 9:27 PM

Post #71 of 106 (2514 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Just for everyone's info,
While debugging Cory's problem, I discovered a completely different
(and unrelated) bug that will require significant rework (and very
likely will result in slowing down the processor). Basically,
libmpeg2 doesn't work reliably when you have 2 independant decoders
going. This may necessitate switching back to libavcodec as the
decoder.
The second issue is that the current method of rebuilding frames will
give complete garbage if you select the B-frames 0 or 1 after a
sequence header. Fixing this means processing the previous GOP, which
will complicate things quite a bit. The solution may be to fully
decode every frame in order (rather than just decoding headers until a
re-encode is needed). That would drastically slow down the
processing. This second bug may explain Bryan's corruption, or it may
not.

Anyhow, what this means is that while I am still interested in issues
that crop up, I won't be doing any more debugging until I've resolved
these new problems. It also may be a couple of days before I've got a
new version to be tested.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 17, 2005, 9:11 AM

Post #72 of 106 (2506 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/16/05, Geoffrey Hausheer wrote:
> Anyhow, what this means is that while I am still interested in issues
> that crop up, I won't be doing any more debugging until I've resolved
> these new problems. It also may be a couple of days before I've got a
> new version to be tested.
Well, nothing so dramatic in the end:
http://www.pblue.org/myth/mpeg2fix-0.12.tgz

This only fixes the problems I mentioned in my previous mail, along
with Adam's problem. There should be no performance impact to this
version.

Cory's hang is probably not fixed here. Bryan's corruption may or may
not be fixed (I haven't been able to duplicate his results with this
version so far. Bryan, if you can still generate the corruption, send
me the command you are using on the file you sent me). Cory's
'roughness' is not fixed.

Anyone else who has experienced video corruption issues, this version
may help (and may not)

Using '-d 2' produces several files around cutpoints:
cnv#.yuv : the original frame to reencode
cnv#.yuv.enc: the mpeg2-encoded frame (after re-encoding)
cnv#.yuv.enc.yuv: a decoded version of the above.

If you see any discrepency between cnv#.yuv and cnv#.yuv.enc.yuv then
the encoding got borked. If cnv#.yuv doesn't look ok, the decoding
got borked. If bth look ok, but playing the mpeg2 is hosed,
well...we'd need to do more debugging.

remember, use imagemagik to display frames:
display -size widthxheight cnv#.yuv

also, using -c format is : -c "start - end"
the quotes, spaces and hyphen are all needed.

If you want some help debugging, I'll hang around #mythtv (nick:
PhracturedBlue) when I can.

Thanks for your patience.
.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 17, 2005, 10:12 AM

Post #73 of 106 (2515 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>On 11/16/05, Geoffrey Hausheer wrote:
>
>
>>Anyhow, what this means is that while I am still interested in issues
>>that crop up, I won't be doing any more debugging until I've resolved
>>these new problems. It also may be a couple of days before I've got a
>>new version to be tested.
>>
>>
>Well, nothing so dramatic in the end:
>http://www.pblue.org/myth/mpeg2fix-0.12.tgz
>
>This only fixes the problems I mentioned in my previous mail, along
>with Adam's problem. There should be no performance impact to this
>version.
>
>Cory's hang is probably not fixed here. Bryan's corruption may or may
>not be fixed (I haven't been able to duplicate his results with this
>version so far. Bryan, if you can still generate the corruption, send
>me the command you are using on the file you sent me). Cory's
>'roughness' is not fixed.
>
>Anyone else who has experienced video corruption issues, this version
>may help (and may not)
>
>
>
I tried it on my mpeg of death, and it still fails at the same spot.
What info do you want, if any?

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 17, 2005, 11:43 AM

Post #74 of 106 (2512 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>On 11/17/05, Tom Lichti wrote:
>
>
>>I tried it on my mpeg of death, and it still fails at the same spot.
>>What info do you want, if any?
>>
>>
>When you say 'fails' what do you mean? Are you getting a hang or a
>crash, or an assertion failure?
>
>
It just stops processing. A normal run shows this:

root [at] mytht:/myth/tv/mpeg2fix-0.11# ./mpeg2fix -i
../3046_20051112050000.mpg -o test.mpg
Opening ../3046_20051112050000.mpg
Input #0, mpeg, from '../3046_20051112050000.mpg':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 480x480,
3000 kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 192 kb/s
#0 PTS:36037 Delta: 0.000000ms queue: 17
#1 PTS:34363 Delta: 18.600000ms queue: 2
Mux rate: 3.24 Mbit/s
Warning: partial frame found!
Warning, QMAT_SHIFT is larger then 21, overflows possible
Inserting 1 I-Frames after frame #2 (null)
root [at] mytht:/myth/tv/mpeg2fix-0.11#

And only creates an 11MB file from the original 600+MB.

>run with '-d 3' and send me the last 30 or so lines.
>
>
Id:0: 0:00:40.838 V:2262519 8667 A:447703 7831
VID: B #:3 nb: 2 pts: 3669497 dts: 3669497 pos: 8205780
Id:0: 0:00:40.772 V:2254951 8623 A:447703 7831
VID: B #:4 nb: 2 pts: 3672500 dts: 3672500 pos: 8213d50
Id:0: 0:00:40.805 V:2251119 8579 A:447703 7831
PTS discrepency: 3678506 != 3678531 on B-Type (6)
PTS discrepency: 3681509 != 3681534 on B-Type (7)
PTS discrepency: 3684512 != 3684537 on P-Type (8)
VID: P #:8 nb: 2 pts: 3684512 dts: 3675528 pos: 824e190
Id:0: 0:00:40.939 V:2247135 8535 A:447703 7831
VID: B #:6 nb: 2 pts: 3678506 dts: 3678506 pos: 8151fa8
Id:0: 0:00:40.872 V:2238739 8491 A:447703 7831
VID: B #:7 nb: 2 pts: 3681509 dts: 3681509 pos: 8140420
Id:0: 0:00:40.905 V:2234499 8447 A:447703 7831
PTS discrepency: 3687515 != 3687540 on B-Type (9)
PTS discrepency: 3690518 != 3690543 on B-Type (10)
PTS discrepency: 3693521 != 3693546 on P-Type (11)
VID: P #:11 nb: 2 pts: 3693521 dts: 3684537 pos: 82369a8
Id:0: 0:00:41.039 V:2229939 8403 A:447703 7831
VID: B #:9 nb: 2 pts: 3687515 dts: 3687515 pos: 8243e18
Id:0: 0:00:40.972 V:2221643 8359 A:447703 7831
VID: B #:10 nb: 2 pts: 3690518 dts: 3690518 pos: 81cca88
Id:0: 0:00:41.005 V:2217371 8315 A:447703 7831
PTS discrepency: 3696524 != 3696549 on B-Type (12)
PTS discrepency: 3699527 != 3699552 on B-Type (13)
PTS discrepency: 3702530 != 3702555 on P-Type (14)
VID: P #:14 nb: 2 pts: 3702530 dts: 3693546 pos: 81280a8
Id:0: 0:00:41.139 V:2212919 8271 A:447703 7831
VID: B #:12 nb: 2 pts: 3696524 dts: 3696524 pos: 81beb18
Id:0: 0:00:41.072 V:2203067 8227 A:447703 7831
VID: B #:13 nb: 2 pts: 3699527 dts: 3699527 pos: 825f6b8
Id:0: 0:00:41.105 V:2198231 8183 A:447703 7831
Warning: partial frame found!
PTS discrepency: 3711539 != 3705558 on I-Type (2)
Warning, QMAT_SHIFT is larger then 21, overflows possible
Inserting 1 I-Frames after frame #2 ins0.yuv
VID: I #:2 nb: 2 pts: 3705558 dts: 3702555 pos: 81fb946
Id:0: 0:00:41.172 V:2193159 8139 A:447703 7831
VID: I #:3 nb: 2 pts: 3708561 dts: 3705558 pos: 82f4290
Id:0: 0:00:41.206 V:2173010 8095 A:447703 7831
VID: I #:4 nb: 2 pts: 3711564 dts: 3708561 pos: 832f738
Id:0: 0:00:41.239 V:2061417 8051 A:447703 7831

>run through gdb, and get me a back trace.
>
>
Not sure how to do that, exactly...

>after that, it may be useful to use mpegparse on it to find out what
>
>
I see it in the directory, how do I build it? I know I sound like an
idiot, but I'm not a developer, just someone trying to help...

>is going on, or get me a saample of the video that I can play with.
>
>
That shouldn't be a problem.

>there will be videos that we just can't handle, but if myth can play
>it, we should be able to fix it.
>
>
Myth plays it, but there are lots of artifacts, and error messages on
the console.

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 17, 2005, 12:18 PM

Post #75 of 106 (2504 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Well, I finally figured out what was causing the corruption in Bryan's
streams (and likely in Cory's too). Turns out that interlacing wasn't
being enabled properly.
Add the following line around line 950 (right after where c->qmin is set):
c->flags |= CODEC_FLAG_INTERLACED_DCT;

That is just a hack. It actually needs to be properly detected, but
I'd like validation that it actually fixes the problem people have had
with ugly images.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 17, 2005, 12:24 PM

Post #76 of 106 (2205 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/17/05, Tom Lichti wrote:
> I tried it on my mpeg of death, and it still fails at the same spot.
> What info do you want, if any?
When you say 'fails' what do you mean? Are you getting a hang or a
crash, or an assertion failure?

run with '-d 3' and send me the last 30 or so lines.
run through gdb, and get me a back trace.

after that, it may be useful to use mpegparse on it to find out what
is going on, or get me a saample of the video that I can play with.
there will be videos that we just can't handle, but if myth can play
it, we should be able to fix it.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

stuarta at squashedfrog

Nov 17, 2005, 12:25 PM

Post #77 of 106 (2201 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Wed, Nov 16, 2005 at 10:15:48AM -0800, Geoffrey Hausheer wrote:
> On 11/16/05, Paul Wheeler wrote:
> > Geoffrey,
> >
> > Did you ever have any luck with getting the subtitles back in? Will
> > this be too difficult/time consuming to do? Are you planning to look
> > into it at all?
> >
> No this hasn't happened, and probably won't until after the code is
> merged with mythtranscode. It shouldn't be TOO difficult, as there
> are only minor changes in my code to handle it. The real problem will
> be supportingit in libreplex (no support at all right now), and from
> what I understand, there are several different subtitle encoding
> methods, which might make supporting it there trickier. I also don't
> know if I have access to the relevant documentation on how they are
> coded, which would certainly make it a lot harder to implement. I'll
> have to dig around and see what I have.
>

The utilities behind mythburn seem to be able to pull out the
subtitle stream and make them useful as a DVD subtitle, so it
has been done for some situations.

As for the documentation, much of the DVB subtitle stuff was
decoded with the libs found in ffmpeg/libavcodec etc, so it's
in there somewhere.

Stuart

papenfuss at juneau

Nov 17, 2005, 1:05 PM

Post #78 of 106 (2211 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Thu, 17 Nov 2005, Geoffrey Hausheer wrote:

> Well, I finally figured out what was causing the corruption in Bryan's
> streams (and likely in Cory's too). Turns out that interlacing wasn't
> being enabled properly.
> Add the following line around line 950 (right after where c->qmin is set):
> c->flags |= CODEC_FLAG_INTERLACED_DCT;
>
> That is just a hack. It actually needs to be properly detected, but
> I'd like validation that it actually fixes the problem people have had
> with ugly images.
>
Appears to be the same as before. From the sounds of it, however,
other people have a different issue than I do. I don't have green blocks
or any such corruption... just a strangely interpolated/interlaced sorta
"quantization" of the streamed encoded frame.

Also, 0.12 still hangs at size:
$ du -sk test2.mpg
3945220 test2.mpg

Also, this change broke mpegparse on my streams.
diff mpegparse.c ../mpeg2fix-0.11/mpegparse.c
57c57
< #define WRITESTREAMS
---
> //#define WRITESTREAMS

in the following way:

$ mpegparse cory_grad_luke_conf_alton.nuv
Reading 'cory_grad_luke_conf_alton.nuv'
Couldn't open audout file
mpegparse: mpegparse.c:769: main: Assertion `0' failed.
Aborted

Cheers... :)

-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

bmayland at leoninedev

Nov 17, 2005, 5:51 PM

Post #79 of 106 (2201 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>Add the following line around line 950 (right after where c->qmin is set):
> c->flags |= CODEC_FLAG_INTERLACED_DCT;
>
>That is just a hack. It actually needs to be properly detected, but
>I'd like validation that it actually fixes the problem people have had
>with ugly images.
>
>
I tried adding this interlaced_dct line with 0.12 and the same cut
from before (300-390). Frame 297 is ok, 298 (I) block artifacts, 299
good-looking I frame from after the cut, 300 is frame 298 again, then
301 starts B frames that look bad and doesn't clear until close to the
next I at 315. This is viewing frame by frame with VirtualDubMod.
Could it be a problem with its stream vs presentation order?
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 18, 2005, 11:50 PM

Post #80 of 106 (2172 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Here we go again:
http://www.pblue.org/myth/mpeg2fix-0.13.tgz

This fixes a deadlock in replex on streams with multiple audio
streams. i'm not 100% sure of the fix, but I think it is ok. This
should resolve the problem Adam was having on IRC.

It also should remove all deadlock conditions. The code should
instead abort with an assert if it detects a deadlock (which at least
tells the user what is going on, and may help debug)

The PTS wrapping code was added, though it hasn't been tested too thoroughly.

I added code which attempts to preserve the interlaced/progresive info
when rebuilding frames. libavcodec doesn't give enough control to do
this right, so I hope the hack I put in works. Probably needs some
user testing.

So that is it. I now have no features I plan to add before this code
goes into myth. I will likely begin working on the interface to
mythtranscode, and commit the changes when I'm done.

New bugs may have cropped, up, there are probably still lots of issues
(I haven't fully resolved either Bryan's or Cory's issues, and there
isn't anything in here which would do so), but any issues that I can't
debug before the final merge I'll just deal with as bugs through the
tracker.

Things which could be tackled at some point (after the merge)
add subtitle support
add ability to rebuild a bad frame from the previous good frame
(possibly interpolate if good data is available). This is the only
way I can fix Tom's corrupted stream problem, and it would be really
cool, but will require significant effort.
support ATSC output (as opposed to converting to PS)
enforce DVD compliance (add borders or clip to resize to a given
dimension without reencoding, and build new I-frames as needed to meet
the GOP requirements).

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 19, 2005, 7:39 AM

Post #81 of 106 (2149 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> It doesn't sound like an avcodec issue, but you can never be sure.
> Please compare my video to yours, and see if they are different. If
> they are the same, well...I can't explain it as it plays great for me,
> so they'll be different. The question will be 'why'
> One way to answer is to run through gdb with '-d 2' and set a
> breakpoint in write_yuv. the first call will be the original frame,
> the second, the reconstructed one. doing a print
> *info->display_picture in both cases and compare the 'flags' value.
> it should be 9 or 10 the first time write_yuv is called, and always be
> '9' on the second call.
>
The flags looked right. They were indicating B->I B->I P->I, etc.
The cnv?.yuv files look great, the corresponding cnv?.yuv.enc and
cnv?.yuv.enc.yuv are garbage.

Since I was pulling my hair out on this, and not being all that
familiar with the avcodec API, I decided to try something on a whim:
disabling simd optimizations using
c->dsp_flags = 0xffff;
Bingo, everything looks great. I then tried enabling each optimization
individually. If I disable MMX, everything crisp. MMX optimizations
on, garbage. This is on a Pentium 4 2.4B, mmx sse sse2.

So points me to yet another libavcodec problem, this time with MMX
opts. Everything in mpeg2fix works great as long as I keep mmx disabled
for now. I've tried it on 5 PS streams and cutpoints all up and down
the file and every cut looks good. Really nice work!
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

papenfuss at juneau

Nov 19, 2005, 7:54 AM

Post #82 of 106 (2160 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

> Here we go again:
> http://www.pblue.org/myth/mpeg2fix-0.13.tgz
>
> This fixes a deadlock in replex on streams with multiple audio
> streams. i'm not 100% sure of the fix, but I think it is ok. This
> should resolve the problem Adam was having on IRC.
>
> It also should remove all deadlock conditions. The code should
> instead abort with an assert if it detects a deadlock (which at least
> tells the user what is going on, and may help debug)
>
Well, that seems true enough:
$ mpeg2fix -d4 -i cory_grad_luke_conf_alton.nuv -o test2.mpg > log4.txt
Input #0, mpeg, from 'cory_grad_luke_conf_alton.nuv':
Duration: N/A, bitrate: N/A
Stream #0.0[0x1e0], 29.97 fps: Video: mpeg2video, yuv420p, 704x480, 6000
kb/s
Stream #0.1[0x1c0]: Audio: mp2, 48000 Hz, stereo, 192 kb/s
Warning, QMAT_SHIFT is larger then 21, overflows possible
Mux rate: 6.29 Mbit/s
Deadlock detected. One buffer is full when
the other is empty! Aborting
mpeg2fix: mpeg2fix.cpp:627: void MPEG2fixup::add_frame(MPEG2frame*):
Assertion `ok' failed.
Aborted

Didn't see much useful out of end of the log4:
PTS discrepency: 609664731 != 609662909 on B-Type (13)
PTS discrepency: 609667975 != 609665912 on B-Type (14)
PTS discrepency: 609671216 != 609668915 on P-Type (15)
VID: P #:15 nb: 2 pts: 609671216 dts: 609659906 pos: 879b840
Id:0: 1:52:54.124 V:83 MP2: 199 AC3:
VID: B #:13 nb: 2 pts: 609664731 dts: 609664731 pos: 876f3e8
Id:0: 1:52:54.052 V:82 MP2: 199 AC3:
VID: B #:14 nb: 2 pts: 609667975 dts: 609667975 pos: 88a27c8
Id:0: 1:52:54.088 V:81 MP2: 199 AC3:

> The PTS wrapping code was added, though it hasn't been tested too thoroughly.
>
This *should* be a very unusual situation though, right? Probably
most likely from constructed sources like dvb or hdtv. ivtv captures
probably just happily spool the PTS along.

> I added code which attempts to preserve the interlaced/progresive info
> when rebuilding frames. libavcodec doesn't give enough control to do
> this right, so I hope the hack I put in works. Probably needs some
> user testing.
>
Unfortunately, my same issue is there. Not sure how to debug this
any further... everything seems good. The decoded/encoded/redecoded debug
frames all look great.

> So that is it. I now have no features I plan to add before this
code > goes into myth. I will likely begin working on the interface to
> mythtranscode, and commit the changes when I'm done.
>
> New bugs may have cropped, up, there are probably still lots of issues
> (I haven't fully resolved either Bryan's or Cory's issues, and there
> isn't anything in here which would do so), but any issues that I can't
> debug before the final merge I'll just deal with as bugs through the
> tracker.
>
I'm sure there will be a whole slew of them once it gets a larger
test base. Seems like everyone who's added their own "stream of death"
(I like that, BTW) has uncovered yet another obscure bug.

> Things which could be tackled at some point (after the merge)
> add subtitle support
> add ability to rebuild a bad frame from the previous good frame
> (possibly interpolate if good data is available). This is the only
> way I can fix Tom's corrupted stream problem, and it would be really
> cool, but will require significant effort.
That'd be pretty spiffy.

> support ATSC output (as opposed to converting to PS)
Does it currently require PS or will any TS work too? I don't have TS
sources, but just curious.

> enforce DVD compliance (add borders or clip to resize to a given
> dimension without reencoding, and build new I-frames as needed to meet
> the GOP requirements).
>
I didn't even think that could be done. It thought a frame was a
frame and you were stuck with the size unless you transcoded.

Cheers,
-Cory

--

*************************************************************************
* Cory Papenfuss *
* Electrical Engineering candidate Ph.D. graduate student *
* Virginia Polytechnic Institute and State University *
*************************************************************************

mythtv0368 at phracturedblue

Nov 19, 2005, 8:23 AM

Post #83 of 106 (2162 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/19/05, Cory Papenfuss wrote:
> > It also should remove all deadlock conditions. The code should
> > instead abort with an assert if it detects a deadlock (which at least
> > tells the user what is going on, and may help debug)
> >
> Well, that seems true enough:
> Didn't see much useful out of end of the log4:
> PTS discrepency: 609664731 != 609662909 on B-Type (13)
> PTS discrepency: 609667975 != 609665912 on B-Type (14)
> PTS discrepency: 609671216 != 609668915 on P-Type (15)
> VID: P #:15 nb: 2 pts: 609671216 dts: 609659906 pos: 879b840
> Id:0: 1:52:54.124 V:83 MP2: 199 AC3:
> VID: B #:13 nb: 2 pts: 609664731 dts: 609664731 pos: 876f3e8
> Id:0: 1:52:54.052 V:82 MP2: 199 AC3:
> VID: B #:14 nb: 2 pts: 609667975 dts: 609667975 pos: 88a27c8
> Id:0: 1:52:54.088 V:81 MP2: 199 AC3:
Run with 'd 3' (and just let the output go to stdout). then send me
the previous 100 lines (or preferably more).
the Id lines are now much more useful:
Id:# (0 is video, 1 is audio)
V:# is the number of free slots in the queue
V:#, MP:#, and AC3:# are the number of free slots in the respective queue.

When one get sto 0 and the other is at 199, the deadlock breaker kicks in.
With that info, along with the time it should be able to find what is going on.

In the above, it is already obvious something is wrong. your video
queue is 60% full, but the MP2 queue is empty. This means that either
there haven't been any audio frames for a long time, or the audio is
significantly behind the video from a PTS perspective.

I need enough log that I can see enough Id:0 and Id:1 lines to figure
out what is wrong.

.Geoff

> > support ATSC output (as opposed to converting to PS)
> Does it currently require PS or will any TS work too? I don't have TS
> sources, but just curious.
If I write the code, then you can convert PS to TS, so it won't matter
what your input format is.

>
> > enforce DVD compliance (add borders or clip to resize to a given
> > dimension without reencoding, and build new I-frames as needed to meet
> > the GOP requirements).
> >
> I didn't even think that could be done. It thought a frame was a
> frame and you were stuck with the size unless you transcoded.
Right, I'd have to re-encode a frame in the middle of the GOP to break
the GOP down into valid number of frames. But as for changing the
resolution, that can be done without re-encoding as long as you are
either rmeoving blocks or adding a border. you can't resample, of
course. I've never actually looked into how it works, but there are
some utilities out there that do it.
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 19, 2005, 8:46 AM

Post #84 of 106 (2159 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/19/05, Cory Papenfuss wrote:
> > The PTS wrapping code was added, though it hasn't been tested too thoroughly.
> >
> This *should* be a very unusual situation though, right? Probably
> most likely from constructed sources like dvb or hdtv. ivtv captures
> probably just happily spool the PTS along.
Actually, not so much.
PTS is mod 2^33, and are in units of 90,000/sec
Which means you get an overflow every 26.5 hours
Now the way the wrapping code work, we can only handle recordings less
than 13hours long (this might be fixable in the future, but I don't
think it is an unreasonable limit)

I think the PVR cards, will reset the PTS to something near 0 when
they start recording, but DVB and ATSC can have arbitrary start
points, so you should see a wrap in one out of every 26 1-hour
recordings.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 19, 2005, 5:20 PM

Post #85 of 106 (2158 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Yep, here's another one...hoepfully the last:
http://www.pblue.org/myth/mpeg2fix-0.14.tgz

With luck, this will fix all the problems Cory, Adam, Tom, and Bryan
have been having :)

Changes include: disable all optimizations when encoding. This really
won't impact performance, and fixes Bryan's issues

prevent broken PTS values from hosing the stream. if a single bad PTS
was found, it would cause replex to get out of sync, and force the
deadlock breaker to kick in. Or if the bad PTS was on an audio frame,
memory usage would explode until the oom killer kicked in. Now, we
just blow-away bad PTS values, and replace them with a valid one. I
also added a max_cap on number of queued frames so the run-away memory
usage won't happen.

Allow broken frames. If a broken video frame came up, the code used
to abort. Now it will just continue on, producing potentially bad
frames. I'd like to be able to fix them, but for now these broken
frames will just be discarded. Hopefully the streams will be
watchable, despite having artifacts. I've watched a few, and while
they don't look pretty, they seem to work ok.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 20, 2005, 3:42 PM

Post #86 of 106 (2137 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Well the new transcoding stuff is in SVN now. It is likely horribly
broken, as a lot of changes were needed for the merge. Besides
updating some of the logging, there are no additional bug fixes from
0.15. I had to remove all the 'assert' lines, so for those of you who
were seeing them, hoepfully the code still errors out correctly.

The code can still be used standalone as well:
http://www.pblue.org/myth/mpeg2fix-0.15.tgz
The only real difference is the Makefile (and the command line
inerface is completely different)

It is not enabled for automatic transcoding for the moment but you
should be able to use '-m' on the command line to have it kick in.

There are still a bunch of things I should have done before the commit
but didn't (clean up the memory on shutdown, remname functions to be
Muth compliant, etc), but those are mostly cosmetic, and I didn't want
to inject any additional noise during the merge. I'll get those
things cleaned up in the next couple days.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

spin667 at mchsi

Nov 21, 2005, 9:26 AM

Post #87 of 106 (2117 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Sun, Nov 20, 2005 at 03:42:46PM -0800, Geoffrey Hausheer wrote:
> Well the new transcoding stuff is in SVN now. It is likely horribly
> broken, as a lot of changes were needed for the merge. Besides
> updating some of the logging, there are no additional bug fixes from
> 0.15. I had to remove all the 'assert' lines, so for those of you who
> were seeing them, hoepfully the code still errors out correctly.
>
> The code can still be used standalone as well:
> http://www.pblue.org/myth/mpeg2fix-0.15.tgz
> The only real difference is the Makefile (and the command line
> inerface is completely different)
>

I've tried a variety of versions of mpeg2fix. 0.14 is the first one
that worked for me. Previous versions shows some corruption (green
blocky flashes) right after the cuts. 0.14 worked perfectly as far as
I can tell, but produced 600k+ lines of output with no debugging
enabled on a 1.5 hour show. Mostly they messages were:

Found invalid audio PTS (off by 0)
and
Found invalid PTS (off by 0) at frame 0

It's also incredibly slow. To the best of my knowledge there is
nothing wrong with my recordings. (They're all made with a PVR-250
and a recent ivtv.) Are these messages indicative of a problem or
just 0.14 being overly verbose?

When I try 0.l5 it's much faster, less verbose but ends with a
reported deadlock. I'm not at home to check out the recording created
by 0.15 but the file looks to be a tiny bit shorter. Both 0.14 and
0.15 report the following a number of times:

Warning, QMAT_SHIFT is larger then 21, overflows possible

I'll attach a snippet of the output from 0.14 and the whole run from
0.15. If you want me to run 0.15 with debug turned on, let me know.

Attachments:

mpeg2fix-0.14.out (4.17 KB)

mpeg2fix-0.15.out (10.4 KB)

mythtv0368 at phracturedblue

Nov 21, 2005, 11:05 AM

Post #88 of 106 (2097 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Aran Cox wrote:
> On Sun, Nov 20, 2005 at 03:42:46PM -0800, Geoffrey Hausheer wrote:
> I've tried a variety of versions of mpeg2fix. 0.14 is the first one
> that worked for me. Previous versions shows some corruption (green
> blocky flashes) right after the cuts. 0.14 worked perfectly as far as
> I can tell, but produced 600k+ lines of output with no debugging
> enabled on a 1.5 hour show. Mostly they messages were:
>
>
> Found invalid audio PTS (off by 0)
> and
> Found invalid PTS (off by 0) at frame 0
All versions since I added the PTS wrap code have had assorted issues.
I'm working on it, but I don't expect many recordings to go through
ok at this point. The message you are seeing in 0.14 was fixed in
0.15, but the frame insertion logic got broken several versions ago,
and I'm still trying to work out the corner cases.
I checked in some more fixes into svn which should help, though I'm
still not convinced it is right.

The mpeg2fix version is here:
http://www.pblue.org/myth/mpeg2fix-0.16.tgz

This may be the last version of mpeg2fix I release. I'm going to
include the relevant makefile and build instructions in myth's svn, so
I can keep the code all in one place. You'll be able tocheck out
mythtranscode, and build mpeg2fix independantly from myth using the
alternate makefile

Note. the debug switch is now a flag, instead of a level. To get
debug enabled, use '-d 65535'

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 21, 2005, 12:13 PM

Post #89 of 106 (2096 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> All versions since I added the PTS wrap code have had assorted issues.
> I'm working on it, but I don't expect many recordings to go through
> ok at this point. The message you are seeing in 0.14 was fixed in
> 0.15, but the frame insertion logic got broken several versions ago,
> and I'm still trying to work out the corner cases.
> I checked in some more fixes into svn which should help, though I'm
> still not convinced it is right.
>
> The mpeg2fix version is here:
> http://www.pblue.org/myth/mpeg2fix-0.16.tgz
>
>
Just tried this, and now it gets about 60% through my 'stream of death'
before it fails with a deadlock.

I wouldn't put too much more time into my particular issue, as this file
is REALLY bad, I'm surprised it gets as far as it does. I'm just
compiling SVN myth right now, since LiveTV seems to be fixed.

Thanks for all the hard work with this, it will certainly help me out,
with my non-corrupt files at any rate.

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 21, 2005, 12:28 PM

Post #90 of 106 (2093 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> All versions since I added the PTS wrap code have had assorted issues.
>
I was just coming here to let you know this....
> I checked in some more fixes into svn which should help, though I'm
> still not convinced it is right.
>
...until I saw this. [7964] fixed the "Deadlock detected!" I was
getting.

One thing I find unusual about the mythtranscode version is that when
you specify to "--honorcutlist" or -l, it expects you to pass the
cutlist on the command line if you specify an input file instead of a
chanid and starttime. This behavior seems unintuitive (although I can
see why someone would want to pass a cutlist on the command line).

I would think that passing a cutlist on the command line should override
the use of the one in the database, but the cutlist should be loaded
from the DB is --honorcutlist is used and no cutlist is passed on the
command line. Maybe I'm thinking about it the wrong way, but that would be:
Index: main.cpp
===================================================================
--- main.cpp (revision 7965)
+++ main.cpp (working copy)
@@ -395,7 +395,7 @@
int exitcode = TRANSCODE_EXIT_OK;
if ((result == REENCODE_MPEG2TRANS) || mpeg2)
{
- if (useCutlist && !found_infile)
+ if (useCutlist && deleteMap.isEmpty())
pginfo->GetCutList(deleteMap);

MPEG2fixup *m2f = new MPEG2fixup(infile.ascii(), outfile.ascii(),

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 21, 2005, 12:44 PM

Post #91 of 106 (2113 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Bryan Mayland wrote:
> Geoffrey Hausheer wrote:
> > All versions since I added the PTS wrap code have had assorted issues.
> >
> I was just coming here to let you know this....
> > I checked in some more fixes into svn which should help, though I'm
> > still not convinced it is right.
> >
> ...until I saw this. [7964] fixed the "Deadlock detected!" I was
> getting.

Just to be clear..it is working ok for you so far?

>
> One thing I find unusual about the mythtranscode version is that when
> you specify to "--honorcutlist" or -l, it expects you to pass the
> cutlist on the command line if you specify an input file instead of a
> chanid and starttime. This behavior seems unintuitive (although I can
> see why someone would want to pass a cutlist on the command line).
Yes it is unituitive (Im not sure whether I wrote that or not. If so,
it would have been long ago), but see below...

> - if (useCutlist && !found_infile)
> + if (useCutlist && deleteMap.isEmpty())
> pginfo->GetCutList(deleteMap);
You can't do this. If a file is specified, that means that there is
no pginfo, so no way to look up the cutlist.

Basically, we find programs based on the chanid/start-time fields. If
they aren't specified, we are in standalone mode, and don't know how
the program is associated with the db (if it even is)

so there are 2 ways to work:
a) you specify a input file, you are in standalone mode. if you want
a cutlist, you need to specify it on the cmd line.

b) you are in db mode. you specify the starttime and chanid. If you
want a cutlist it will be loaded from the dB (you can't apply one on
the cmdline)

there is also a subset of (b) which is
c) you are running from the jobqueue. Myth will handle replacing the
old file with the new one.

be aware, that there is a hole between (b) and (c). If you are NOT
running from the job-queue, your position-map will still be updated
when mythtranscode runs. This means if you try to watch teh recording
after trasncoding but before replacing the old version with the
transcoded one, you'll get very strange response.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

torbjorn.jansson at mbox200

Nov 21, 2005, 12:58 PM

Post #92 of 106 (2098 views)
Permalink

RE: New MPEG2 commercial-cut code ready for testing [In reply to]

mythtv-dev-bounces [at] mythtv <> wrote:
>> - if (useCutlist && !found_infile)
>> + if (useCutlist && deleteMap.isEmpty())
>> pginfo->GetCutList(deleteMap);
> You can't do this. If a file is specified, that means that there is
> no pginfo, so no way to look up the cutlist.
>

Why not try to find the filename by looking at the basename field in the
recorded table?

_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 21, 2005, 1:05 PM

Post #93 of 106 (2111 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Torbj�rn Jansson wrote:

> Why not try to find the filename by looking at the basename field in the
> recorded table?
I didn't say you can't, but 'don't want to'. If you want to use the
db, then use the right commands. There is no reason to support a
reverse lookup. mythtranscode is not meant to be used on the command
line, except by people who know what they are doing. And i need the
capability to use it without the db.

The interface could certainly be cleaned up to make everyone happy,
but I think the ROI is very low. Better to spend my time fixing bugs
and adding features.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bmayland at leoninedev

Nov 21, 2005, 1:30 PM

Post #94 of 106 (2111 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
>> ...until I saw this. [7964] fixed the "Deadlock detected!" I was
>> getting.
>>
> Just to be clear..it is working ok for you so far?
>
Yes, using r7964 it seems to be working as it should (cutting
properly and not displaying the deadlock message at the end), but
running from the command line. I have not had a chance to try it
automatically as a job.
>> - if (useCutlist && !found_infile)
>> + if (useCutlist && deleteMap.isEmpty())
>> pginfo->GetCutList(deleteMap);
>>
> You can't do this. If a file is specified, that means that there is
> no pginfo, so no way to look up the cutlist.
>
Oh you're right. It should say:

if (useCutlist && pginfo && deleteMap.isEmpty())

The code above it uses the passed file basename to lookup the pginfo
based on splitting the filename by regex. Of course, this will break if
the filename pattern changes but it does work now.
> be aware, that there is a hole between (b) and (c). If you are NOT
> running from the job-queue, your position-map will still be updated
> when mythtranscode runs. This means if you try to watch teh recording
> after trasncoding but before replacing the old version with the
> transcoded one, you'll get very strange response.
>
Ah I didn't notice that. I actually did a comflag --rebuild after,
then replaced the file myself when I tested.
> The interface could certainly be cleaned up to make everyone happy,
> but I think the ROI is very low. Better to spend my time fixing bugs
> and adding features.
You are absolutely right. It does already look up the pginfo using
the basename though... :)
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 21, 2005, 3:29 PM

Post #95 of 106 (2087 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Tom Lichti wrote:
> Just tried this, and now it gets about 60% through my 'stream of death'
> before it fails with a deadlock.
>
Are you getting the 'Deadlock detected' message, and then a
termination? I could try to debug this if you were to run using '-d
65535' and zip up and send me the entire log.

Ideally, there shouldn't be any conditions in which those failures
occur, as it should be possible to work on any stream that myth can
play. In reality, that probably isn't so easy, but I'm still
interested in the cases it doesn't work.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 21, 2005, 5:10 PM

Post #96 of 106 (2086 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>On 11/21/05, Tom Lichti wrote:
>
>
>>Just tried this, and now it gets about 60% through my 'stream of death'
>>before it fails with a deadlock.
>>
>>
>>
>Are you getting the 'Deadlock detected' message, and then a
>termination? I could try to debug this if you were to run using '-d
>65535' and zip up and send me the entire log.
>
>
Yeup, I can do that. And yes, it does exactly that.

>Ideally, there shouldn't be any conditions in which those failures
>occur, as it should be possible to work on any stream that myth can
>play. In reality, that probably isn't so easy, but I'm still
>interested in the cases it doesn't work.
>
>
If you are up for it, so am I. When my SVN compile and install is
finished I'll get that for you.

Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 21, 2005, 8:13 PM

Post #97 of 106 (2086 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:

>On 11/21/05, Tom Lichti wrote:
>
>
>>Just tried this, and now it gets about 60% through my 'stream of death'
>>before it fails with a deadlock.
>>
>>
>>
>Are you getting the 'Deadlock detected' message, and then a
>termination? I could try to debug this if you were to run using '-d
>65535' and zip up and send me the entire log.
>
>
Done. The logfile is here:
http://www.redpepperracing.com/images/deadlock.log.gz

It is about 1.6MB. I didn't capture the messages to the console (is that
STDOUT or STDERR, I can never remember) If you want that let me know and
I'll redo it.

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Nov 21, 2005, 8:48 PM

Post #98 of 106 (2086 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Tom Lichti wrote:
> It is about 1.6MB. I didn't capture the messages to the console (is that
> STDOUT or STDERR, I can never remember) If you want that let me know and
> I'll redo it.
Nope, this is useful. The problem you are having is that you are
missing some audio frames. The code is trying to correct for bad pts,
and the result is that the video is backing up waiting for the audio.
I need to make the correction algorithm less agressive, but getting it
to work without causing other issues will be tricky. This is one of
the known limitations (the code behaves badly with missing audio
frames). I'll look into fixing it when I get a chance.

Thanks,
.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

tom at redpepperracing

Nov 22, 2005, 5:40 AM

Post #99 of 106 (2090 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

Geoffrey Hausheer wrote:
> On 11/21/05, Tom Lichti wrote:
>
>> It is about 1.6MB. I didn't capture the messages to the console (is that
>> STDOUT or STDERR, I can never remember) If you want that let me know and
>> I'll redo it.
>>
> Nope, this is useful. The problem you are having is that you are
> missing some audio frames. The code is trying to correct for bad pts,
> and the result is that the video is backing up waiting for the audio.
> I need to make the correction algorithm less agressive, but getting it
> to work without causing other issues will be tricky. This is one of
> the known limitations (the code behaves badly with missing audio
> frames). I'll look into fixing it when I get a chance.
>
Okay, cool. Glad I could help.

On a side note, I noticed that I am recording everything in PS mode,
should I be using TS? Would that gain me anything?

Thanks
Tom
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

gkruse at gmail

Nov 22, 2005, 10:04 AM

Post #100 of 106 (2069 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

If I try to transcode a recording after I have already transcoded it
once (trying to figure out commercial removing) using the new mpeg 2
transcoder, I get the following error:

2005-11-22 10:00:15.604 Transcoding aborted, need MPEG-2.

As far as I know, the file is still mpeg 2
Let me know what I need to provide.

Geoff

On Nov 22, 2005, at 5:40 AM, Tom Lichti wrote:

> Geoffrey Hausheer wrote:
>> On 11/21/05, Tom Lichti wrote:
>>
>>> It is about 1.6MB. I didn't capture the messages to the console
>>> (is that
>>> STDOUT or STDERR, I can never remember) If you want that let me
>>> know and
>>> I'll redo it.
>>>
>> Nope, this is useful. The problem you are having is that you are
>> missing some audio frames. The code is trying to correct for bad
>> pts,
>> and the result is that the video is backing up waiting for the
>> audio. I need to make the correction algorithm less agressive, but
>> getting it
>> to work without causing other issues will be tricky. This is one of
>> the known limitations (the code behaves badly with missing audio
>> frames). I'll look into fixing it when I get a chance.
>>
> Okay, cool. Glad I could help.
>
> On a side note, I noticed that I am recording everything in PS
> mode, should I be using TS? Would that gain me anything?
>
> Thanks
> Tom
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

Attachments:

smime.p7s (2.31 KB)

gkruse at gmail

Nov 22, 2005, 12:03 PM

Post #101 of 106 (1296 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

I also noticed that the transcoded files are generated with funny
permissions:

Before transcoding:
-rw-r--r-- 1 geoff geoff Nov 20 20:37 1012_20051120210000.mpg

After transcoding:
-rwx------ 1 geoff geoff Nov 22 11:07 1012_20051120210000.mpg

This makes it so when I try to download the recording with mythweb, I
get a forbidden error. All the other recordings which are generated
and commflagged by the backend have the permissions of the first
recording I listed.

Geoff
On Nov 22, 2005, at 5:40 AM, Tom Lichti wrote:

> Geoffrey Hausheer wrote:
>> On 11/21/05, Tom Lichti wrote:
>>
>>> It is about 1.6MB. I didn't capture the messages to the console
>>> (is that
>>> STDOUT or STDERR, I can never remember) If you want that let me
>>> know and
>>> I'll redo it.
>>>
>> Nope, this is useful. The problem you are having is that you are
>> missing some audio frames. The code is trying to correct for bad
>> pts,
>> and the result is that the video is backing up waiting for the
>> audio. I need to make the correction algorithm less agressive, but
>> getting it
>> to work without causing other issues will be tricky. This is one of
>> the known limitations (the code behaves badly with missing audio
>> frames). I'll look into fixing it when I get a chance.
>>
> Okay, cool. Glad I could help.
>
> On a side note, I noticed that I am recording everything in PS
> mode, should I be using TS? Would that gain me anything?
>
> Thanks
> Tom
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

Attachments:

smime.p7s (2.31 KB)

mythtv0368 at phracturedblue

Nov 22, 2005, 8:17 PM

Post #102 of 106 (1283 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/22/05, Geoffrey Kruse wrote:
> If I try to transcode a recording after I have already transcoded it
> once (trying to figure out commercial removing) using the new mpeg 2
> transcoder, I get the following error:
>
> 2005-11-22 10:00:15.604 Transcoding aborted, need MPEG-2.
>
> As far as I know, the file is still mpeg 2
> Let me know what I need to provide.
As I said, it isnt' enabled by default yet. You need to either
transcode by hand form the command line, or you need to enable it by
adding the appropriate lines in the recordingprofiles tables

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

gkruse at gmail

Nov 22, 2005, 8:27 PM

Post #103 of 106 (1286 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On Nov 22, 2005, at 8:17 PM, Geoffrey Hausheer wrote:

> On 11/22/05, Geoffrey Kruse wrote:
>> If I try to transcode a recording after I have already transcoded it
>> once (trying to figure out commercial removing) using the new mpeg 2
>> transcoder, I get the following error:
>>
>> 2005-11-22 10:00:15.604 Transcoding aborted, need MPEG-2.
>>
>> As far as I know, the file is still mpeg 2
>> Let me know what I need to provide.
> As I said, it isnt' enabled by default yet. You need to either
> transcode by hand form the command line, or you need to enable it by
> adding the appropriate lines in the recordingprofiles tables
>
I had the old mpg2 code enabled with special tables and it seems to
transcode just fine, it just spits out these warnings.
> .Geoff
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

Attachments:

smime.p7s (2.31 KB)

mythtv0368 at phracturedblue

Nov 23, 2005, 5:56 AM

Post #104 of 106 (1271 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/22/05, Geoffrey Kruse wrote:
> I had the old mpg2 code enabled with special tables and it seems to
> transcode just fine, it just spits out these warnings.

That makes more sense with respect to your other message. I'll fix
the permissions problem now.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

bobnvic at gmail

Dec 2, 2005, 12:33 PM

Post #105 of 106 (1217 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 11/21/05, Geoffrey Hausheer <mythtv0368 [at] phracturedblue> wrote:
snip
> The mpeg2fix version is here:
> http://www.pblue.org/myth/mpeg2fix-0.16.tgz
>
> This may be the last version of mpeg2fix I release. I'm going to
> include the relevant makefile and build instructions in myth's svn, so
> I can keep the code all in one place. You'll be able tocheck out
> mythtranscode, and build mpeg2fix independantly from myth using the
> alternate makefile

I wanted to compile mpeg2fix for standalone so that I could check/fix
some recordings of mine, however I couldn't find the relevant makefile
and build instructions in mythtv/programs/mythtranscode. I'm running
svn 8049. Could you tell me where I need to look?

Thanks,
Bob C
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

mythtv0368 at phracturedblue

Dec 2, 2005, 2:00 PM

Post #106 of 106 (1196 views)
Permalink

Re: New MPEG2 commercial-cut code ready for testing [In reply to]

On 12/2/05, Bob Cottingham wrote:
> I wanted to compile mpeg2fix for standalone so that I could check/fix
> some recordings of mine, however I couldn't find the relevant makefile
> and build instructions in mythtv/programs/mythtranscode. I'm running
> svn 8049. Could you tell me where I need to look?
>
I haven't had a chance to add it yet (been on vacation for most of 2
weeks). I will be making some chnages this weekend (mostly cleanup
stuff), and will add the relevant stuff then. For now, just grab the
last mpeg2fix standalone I posted, and copy over the mpeg2fix.cpp and
.h files, and you should be good to go.

.Geoff
_______________________________________________
mythtv-dev mailing list
mythtv-dev [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads