Mailing List Archive: MythTV: Users

IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH?

Index | Next | Previous | View Threaded

blammo.doh at gmail

Jan 8, 2006, 2:13 PM

Post #1 of 8 (499 views)
Permalink

IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH?

This is getting old:

2006-01-08 14:53:20.087 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:53:21.216 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:53:21.586 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:53:21.666 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:53:29.968 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:53:30.194 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:54:08.315 IOBOUND - blocking in ThreadedFileWriter::Write()
2006-01-08 14:54:49.007 IOBOUND - blocking in ThreadedFileWriter::Write()

I've read all the threads I can find on this topic, with no good
fixes. The only one that seems applicable is the 2M/8M/32M buffer
change for HD content. Before I introduce CVS into my otherwise
completely stable env, I'd like to make sure there's nothing else I
can do.

-- only does it while recording HD content
-- does it more on CBS than other stations
-- seems to have no correlation to cpu load, disk IO, etc.

backend hardware:
Athlon 2400
512M PC3200
NVidia chipset motherboard (NF7-SG)
3ware 9500S-12 w/256M cache, write caching enabled (box is on a UPS)
8x160gb SATA drives
(2) Air2PC HD cards
(1) PVR-250

software:
FC4
myth from yum (0.18.1.20050523-1)

It's doing it as we speak, watching the NFL wildcard game. Just
finished recording/watching the one on fox, not a blip. Now the one on
CBS, I'm averaging a report every 4-5 seconds during the game, not at
all during the commercials. It has the visual effect of garbage /
stutter / etc on the dedicated frontend I'm watching.

I routinely record 3 programs at the same time, and as long as one of
them isn't high-bit-rate CBS (csi miami for example) there's zero
problems.

vmstat -d
sda 13634885 33356 2575506290 508342252 12288617 1775750 1620207680
879929132 0 96712

procinfo -n10 -d
Memory: Total Used Free Shared Buffers
Mem: 0 -33 33 0 0
Swap: 0 0 0

Bootup: Tue Dec 20 15:31:33 2005 Load average: 1.60 1.81 1.88 1/177 382

user : 0:00:00.46 1.8% page in : 0
nice : 0:00:00.00 0.0% page out: 0
system: 0:00:01.74 7.0% swap in : 0
idle : 0:00:07.81 31.2% swap out: 0
uptime: 18d 23:37:19.00 context : 239864

irq 0: 2500 timer irq 9: 0 acpi
irq 1: 0 i8042 irq 10: 27624 ivtv0, eth0
irq 2: 0 cascade [4] irq 11: 4930 Technisat/B2C2 FlexC
irq 3: 623 3w-9xxx irq 12: 0 i8042
irq 7: 867 Technisat/B2C2 FlexC irq 14: 3 ide0
irq 8: 0 rtc

The only tuner recording anything right now is the IRQ11 device. The
eth0 traffic is coming from my watching it while it's recording.

I'm getting what I would consider (based on where I've come from)
great results from the RAID array:

[root [at] backend ~]# bonnie++ -d /raid/temp -u0 -g0 -n0 -s 1024
Using uid:0, gid:0.
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
backend1 1G 30144 85 41942 23 25496 14 35914 90 116390 38 560.8 4
backend1,1G,30144,85,41942,23,25496,14,35914,90,116390,38,560.8,4,,,,,,,,,,,,,

Ahh, one last note, I'm running elevator=deadline.

what the hell is going on?
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

kuphal at dls

Jan 8, 2006, 4:43 PM

Post #2 of 8 (494 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

Blammo wrote:

>This is getting old:
>
>2006-01-08 14:53:20.087 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:53:21.216 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:53:21.586 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:53:21.666 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:53:29.968 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:53:30.194 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:54:08.315 IOBOUND - blocking in ThreadedFileWriter::Write()
>2006-01-08 14:54:49.007 IOBOUND - blocking in ThreadedFileWriter::Write()
>
>
>I've read all the threads I can find on this topic, with no good
>fixes. The only one that seems applicable is the 2M/8M/32M buffer
>change for HD content. Before I introduce CVS into my otherwise
>completely stable env, I'd like to make sure there's nothing else I
>can do.
>
>
>
<snip>

>what the hell is going on?
>
>
From everything I can remember about IOBOUND, it means your hardware
(probably disk) isn't able to keep up. What filesystem are you using?
I'm assuming that CBS is 1080i while the other channels are 720p? I'm
guessing also that the commercials are not at 1080i either. Seems to me
that indicates strongly that there is something going on in the I/O to
disk. Also, are you running real-time commflagging on these
recordings? If so, does this behavior change if you do not run that job?

I think this thread is the most relevant to your situation:
http://www.opensubscriber.com/message/mythtv-dev [at] mythtv/1682169.html

and includes suggestions about increasing the buffer size as well as
elevator=cfg and the filesystem in question.

Kevin
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mwahal at gmail

Jan 8, 2006, 6:11 PM

Post #3 of 8 (489 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

elevator=cfq didnt help in my setup. I never got around to moving my
recordings to jfs file system. So, I'm just using 16MB buffer and
seldom had iobound problem. I added another hard drive and formatted
it as xfs file system. But the recording is still on ext3.

On 1/8/06, Kevin Kuphal <kuphal [at] dls> wrote:
> Blammo wrote:
>
> >This is getting old:
> >
> >2006-01-08 14:53:20.087 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:53:21.216 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:53:21.586 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:53:21.666 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:53:29.968 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:53:30.194 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:54:08.315 IOBOUND - blocking in ThreadedFileWriter::Write()
> >2006-01-08 14:54:49.007 IOBOUND - blocking in ThreadedFileWriter::Write()
> >
> >
> >I've read all the threads I can find on this topic, with no good
> >fixes. The only one that seems applicable is the 2M/8M/32M buffer
> >change for HD content. Before I introduce CVS into my otherwise
> >completely stable env, I'd like to make sure there's nothing else I
> >can do.
> >
> >
> >
> <snip>
>
> >what the hell is going on?
> >
> >
> From everything I can remember about IOBOUND, it means your hardware
> (probably disk) isn't able to keep up. What filesystem are you using?
> I'm assuming that CBS is 1080i while the other channels are 720p? I'm
> guessing also that the commercials are not at 1080i either. Seems to me
> that indicates strongly that there is something going on in the I/O to
> disk. Also, are you running real-time commflagging on these
> recordings? If so, does this behavior change if you do not run that job?
>
> I think this thread is the most relevant to your situation:
> http://www.opensubscriber.com/message/mythtv-dev [at] mythtv/1682169.html
>
> and includes suggestions about increasing the buffer size as well as
> elevator=cfg and the filesystem in question.
>
> Kevin
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

blammo.doh at gmail

Jan 8, 2006, 7:52 PM

Post #4 of 8 (489 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

On 1/8/06, Mudit Wahal <mwahal [at] gmail> wrote:
> elevator=cfq didnt help in my setup. I never got around to moving my
> recordings to jfs file system. So, I'm just using 16MB buffer and
> seldom had iobound problem. I added another hard drive and formatted
> it as xfs file system. But the recording is still on ext3.

I'm already running JFS, and elevator=deadline, which seemed to have
the best overall performance for HD playback during recording.

It's been a while since this the thread mentioning buffer sizes was
started. Is there any way to modify this outside of a recompile?

FWIW, I don't have this problem under ANY other circumstances. I
routinely (not during this recording or occurance) copy multi-gig
files to/from this box via SMB. There are (4) other machines in this
myth cluster that do transcode / commflag / playback (all reads I
realize) via NFS, and don't have this problem. I can copy files from
one drive to the raid array, and back, no such problem occurs.

This appears to be a mythbackend problem, not an OS/hardware problem,
best that I can tell.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

kuphal at dls

Jan 8, 2006, 7:54 PM

Post #5 of 8 (487 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

Blammo wrote:

>On 1/8/06, Mudit Wahal <mwahal [at] gmail> wrote:
>
>
>>elevator=cfq didnt help in my setup. I never got around to moving my
>>recordings to jfs file system. So, I'm just using 16MB buffer and
>>seldom had iobound problem. I added another hard drive and formatted
>>it as xfs file system. But the recording is still on ext3.
>>
>>
>
>I'm already running JFS, and elevator=deadline, which seemed to have
>the best overall performance for HD playback during recording.
>
>It's been a while since this the thread mentioning buffer sizes was
>started. Is there any way to modify this outside of a recompile?
>
>
No. Are you performing comflagging on this recording while recording?
Have you tried disabling that to see if it changes anything?

Kevin
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mwahal at gmail

Jan 8, 2006, 8:37 PM

Post #6 of 8 (479 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

On 1/8/06, Kevin Kuphal <kuphal [at] dls> wrote:
> Blammo wrote:
>
> >On 1/8/06, Mudit Wahal <mwahal [at] gmail> wrote:
> >
> >
> >>elevator=cfq didnt help in my setup. I never got around to moving my
> >>recordings to jfs file system. So, I'm just using 16MB buffer and
> >>seldom had iobound problem. I added another hard drive and formatted
> >>it as xfs file system. But the recording is still on ext3.
> >>
> >>
> >
> >I'm already running JFS, and elevator=deadline, which seemed to have
> >the best overall performance for HD playback during recording.
> >
> >It's been a while since this the thread mentioning buffer sizes was
> >started. Is there any way to modify this outside of a recompile?
> >
> >
> No. Are you performing comflagging on this recording while recording?
> Have you tried disabling that to see if it changes anything?
>
> Kevin

I mostly encountered it when I was doing real time comflagging along
with recording one or two shows. Once I disabled realtime comflagging
by modifying mythcommflag script, as it'll start comflagging when the
recoreding completed. But I may have started one/two more recordings.
So, the mythcommflag program will just log the programs to be
comflagged. Then a cron job will run real comflagg program when there
is no recording happening on the box and no recording is scheduled to
happen for next couple of hours. After the comflag is done, it'll set
an alarm to awake before the next recording and goto sleep :-)
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

blammo.doh at gmail

Jan 8, 2006, 10:35 PM

Post #7 of 8 (482 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

On 1/8/06, Kevin Kuphal <kuphal [at] dls> wrote:
> Blammo wrote:
>
> >On 1/8/06, Mudit Wahal <mwahal [at] gmail> wrote:
> >
> >
> >>elevator=cfq didnt help in my setup. I never got around to moving my
> >>recordings to jfs file system. So, I'm just using 16MB buffer and
> >>seldom had iobound problem. I added another hard drive and formatted
> >>it as xfs file system. But the recording is still on ext3.
> >>
> >>
> >
> >I'm already running JFS, and elevator=deadline, which seemed to have
> >the best overall performance for HD playback during recording.
> >
> >It's been a while since this the thread mentioning buffer sizes was
> >started. Is there any way to modify this outside of a recompile?
> >
> >
> No. Are you performing comflagging on this recording while recording?
> Have you tried disabling that to see if it changes anything?

in this case, today, there were two activities taking place:

1. recording HD from CBS
2. watching the same HD recording on a dedicated frontend

even when I stopped activity #2, I still was getting the error
messages, which then show up as "glitches" in the recording.

nothing else. No commflag, no transcode, no nfs activity, no smb
activity. simply mythbackend saving a HD recording.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mwahal at gmail

Jan 9, 2006, 10:31 AM

Post #8 of 8 (480 views)
Permalink

Re: IOBOUND - blocking in ThreadedFileWriter::Write() --- WTH? [In reply to]

run top and see how much iowait do you have in the system.

On 1/8/06, Blammo <blammo.doh [at] gmail> wrote:
> On 1/8/06, Kevin Kuphal <kuphal [at] dls> wrote:
> > Blammo wrote:
> >
> > >On 1/8/06, Mudit Wahal <mwahal [at] gmail> wrote:
> > >
> > >
> > >>elevator=cfq didnt help in my setup. I never got around to moving my
> > >>recordings to jfs file system. So, I'm just using 16MB buffer and
> > >>seldom had iobound problem. I added another hard drive and formatted
> > >>it as xfs file system. But the recording is still on ext3.
> > >>
> > >>
> > >
> > >I'm already running JFS, and elevator=deadline, which seemed to have
> > >the best overall performance for HD playback during recording.
> > >
> > >It's been a while since this the thread mentioning buffer sizes was
> > >started. Is there any way to modify this outside of a recompile?
> > >
> > >
> > No. Are you performing comflagging on this recording while recording?
> > Have you tried disabling that to see if it changes anything?
>
> in this case, today, there were two activities taking place:
>
> 1. recording HD from CBS
> 2. watching the same HD recording on a dedicated frontend
>
> even when I stopped activity #2, I still was getting the error
> messages, which then show up as "glitches" in the recording.
>
> nothing else. No commflag, no transcode, no nfs activity, no smb
> activity. simply mythbackend saving a HD recording.
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads