Mailing List Archive: MythTV: Users

Re: Planning Tuners and Disk I/O for a MythTVBackend System

Index | Next | Previous | View Threaded

mythtv at ncc1701

Feb 7, 2010, 7:37 PM

Post #1 of 22 (1976 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System

>>That leads me to believe that the practical number of physical tuners
for a PVR system can
>> be limited by disk IO throughput, and should be an important
consideration when designing,
>> and using the system.

>No. You can probably put a dozen digital tuners in a system before you
might have to
>worry about exceeding the throughput of a single modern hard drive.

Absolutely.. but..

Recording speed with 12 tuners requires 12 concurrent streams to be
written to the disk. The
bottleneck becomes the seek times and rotational latency.

If noise and cost wasn't a problem you would go for 15K RPM drives just
to ensure latency
and seek times are minimized.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

bkamen at benjammin

Feb 7, 2010, 8:00 PM

Post #2 of 22 (1941 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On 2/7/2010 9:37 PM, MythTV wrote:
>
>>> That leads me to believe that the practical number of physical tuners
> for a PVR system can
>>> be limited by disk IO throughput, and should be an important
> consideration when designing,
>>> and using the system.
>
>> No. You can probably put a dozen digital tuners in a system before you
> might have to
>> worry about exceeding the throughput of a single modern hard drive.
>
> Absolutely.. but..
>
> Recording speed with 12 tuners requires 12 concurrent streams to be
> written to the disk. The
> bottleneck becomes the seek times and rotational latency.
>
> If noise and cost wasn't a problem you would go for 15K RPM drives just
> to ensure latency
> and seek times are minimized.

Could always start striping drives in a raid 0 kinda deal. :D

-Ben
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

raymond at wagnerrp

Feb 7, 2010, 11:37 PM

Post #3 of 22 (1935 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On 2/7/2010 22:37, MythTV wrote:
> Recording speed with 12 tuners requires 12 concurrent streams to be
> written to the disk. The
> bottleneck becomes the seek times and rotational latency.
>

The innermost track of a 1.5TB 5400RPM drive is going to push 50MB/s
sequentially. With some safe estimates of 30 seeks/sec, and a 15ms seek
latency, almost half the time would be spent seeking, dropping
throughput to 25MB/s (ignoring onboard cache and NCQ). Those twelve
recordings each at a respectable 16mbps will use 24MB/s. That puts it
in the realm of possible, but probably not without problems. Of course
if you own 12 tuners, you should have no problem getting multiple hard
drives to record to.

On 2/7/2010 23:00, Ben Kamen wrote:
> Could always start striping drives in a raid 0 kinda deal. :D

No. Striping causes the same exact problem. You still have all the
disks seeking in unison, you just end up with significantly higher
throughput. If you've got proper hardware RAID with write-back cache,
it may be able to re-order the writes to be more sequential. The whole
purpose of Storage Groups added in 0.21 was to allow you to add multiple
independent drives to MythTV, and allow it to properly load balance
between them.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

beckett.jim at gmail

Feb 8, 2010, 2:42 AM

Post #4 of 22 (1921 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Raymond Wagner wrote:
> The innermost track of a 1.5TB 5400RPM drive is going to push 50MB/s
> sequentially. With some safe estimates of 30 seeks/sec, and a 15ms
> seek latency, almost half the time would be spent seeking, dropping
> throughput to 25MB/s (ignoring onboard cache and NCQ). Those twelve
> recordings each at a respectable 16mbps will use 24MB/s. That puts it
> in the realm of possible, but probably not without problems. Of
> course if you own 12 tuners, you should have no problem getting
> multiple hard drives to record to.
>
>
> ... Striping causes the same exact problem. You still have all the
> disks seeking in unison, you just end up with significantly higher
> throughput. If you've got proper hardware RAID with write-back cache,
> it may be able to re-order the writes to be more sequential. The
> whole purpose of Storage Groups added in 0.21 was to allow you to add
> multiple independent drives to MythTV, and allow it to properly load
> balance between them.
>
I am appreciative of all the responses that I have received on this
topic. (Although, I'm extremely embarrassed that the initial set of
'facts' I was using were so horribly incorrect.)

This discussion has illustrated some things about my current storage
system that are probably just flat wrong, and that I should fix, before
I even think about the requirements for a new system down the road.

First the background, and then a load of questions at the end...

One mistake is that all of my partitions are ext3, and I should probably
change the recording partitions to xfs, or another file system that
handles large files better.

(I haven't noticed any issues with the current ext3 file systems I have
in place, but that could be due to inattention and dumb luck, as much as
anything. I think I should fix this for good measure, if nothing else.)

Another mistake I think I have made was using my USB2 500GB drive for an
additional recording storage area.

I think that this recordings storage area should probably be moved to
one of the internal SATA drives, and any music, pictures, and video
storage groups, should probably move to the USB disk.

My thinking is that the primary use on the USB disk should be read
mostly, and would only see occasional write activity on it.

The last mistake I see is that the way I have these disks partitioned is
a mess (IMO):

Filesystem Size Used Avail Use% Mounted on

/dev/sda1 55G 2.7G 49G 6% /
/dev/sda3 138G 188M 131G 1% /livetv
/dev/sda4 267G 154G 100G 61% /recordings

/dev/sdb1 3.8G 97M 3.5G 3% /pictures
/dev/sdb2 376G 169G 189G 48% /videos
/dev/sdb5 9.4G 2.9G 6.1G 33% /music
/dev/sdb6 68G 822M 64G 2% /datashare

/dev/sdc1 459G 325G 111G 75% /recordings2

So in order to clean up my storage issues, I think I would probably want
to have it set up like this, and used balanced IO in MythTV for recordings:

/dev/sda (first internal disk) - 500.1GB SATA ~73 MB/s disk reads

sda1, 60GB, mount point /, Ext3
sda2, 400GB, mount point /recordings, XFS

/dev/sdb (second internal disk) - 500.1GB SATA ~73 MB/s disk reads

sdb1, 4GB, swap space
sdb2, 56GB, mount point /livetv, XFS
sdb3, 400GB, mount point /recordings2, XFS

/dev/sdc (USB 2.0 - 500.1GB External drive) ~31 MB/s disk reads

sdc1, 400GB, mount point /media/disk, FAT32 ***
sdc2, 60GB, mount point /datashare, FAT32

(*** will hold videos, music, and pictures, and I assume I'll need to
fix these locations in Myth, and Samba)

Does this look like a better use of disk space for a dedicated FE/BE
MythTV system?

Is there any performance increase/decrease that would mean I should do
this differently?

Will the new sda1 partition be large enough, or is 60GB too big?

I see XFS, and EXT4, talked about a lot for large file types. Which of
these, or possibly another, would be the best choice for the recordings
directory?

I'm thinking FAT32 on the USB drive for portability, if I ever want to
mount it temporarily on another system. Shouldn't be a problem for
MythTV, should it?

Lastly, I have 2GB of RAM installed in this system, should I even bother
with the swap partition?

Any advice, guidance, or constructive criticism is welcome!

Thanks,

-JIm
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mikep at randomtraveller

Feb 8, 2010, 4:34 AM

Post #5 of 22 (1904 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Raymond Wagner wrote:
> On 2/7/2010 22:37, MythTV wrote:
>> Recording speed with 12 tuners requires 12 concurrent streams to be
>> written to the disk. The
>> bottleneck becomes the seek times and rotational latency.
>>
>
> The innermost track of a 1.5TB 5400RPM drive is going to push 50MB/s
> sequentially. With some safe estimates of 30 seeks/sec, and a 15ms seek
> latency, almost half the time would be spent seeking, dropping
> throughput to 25MB/s (ignoring onboard cache and NCQ). Those twelve
> recordings each at a respectable 16mbps will use 24MB/s. That puts it
> in the realm of possible, but probably not without problems. Of course
> if you own 12 tuners, you should have no problem getting multiple hard
> drives to record to.
>
>
> On 2/7/2010 23:00, Ben Kamen wrote:
>> Could always start striping drives in a raid 0 kinda deal. :D
>
> No. Striping causes the same exact problem. You still have all the
> disks seeking in unison, you just end up with significantly higher
> throughput. If you've got proper hardware RAID with write-back cache,
> it may be able to re-order the writes to be more sequential. The whole
> purpose of Storage Groups added in 0.21 was to allow you to add multiple
> independent drives to MythTV, and allow it to properly load balance
> between them.
>
Don't forget that along with the 12 different streams you're writing to those
disks, you're *also* writing seek information for 12 different recordings to the
database - on another disk, one assumes. Consider therefore the requirements
needed for database buffering, throughput, etc.

--

Mike Perkins

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

bkamen at benjammin

Feb 8, 2010, 8:27 AM

Post #6 of 22 (1900 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On 2/8/2010 1:37 AM, Raymond Wagner wrote:
> On 2/7/2010 22:37, MythTV wrote:
>> Recording speed with 12 tuners requires 12 concurrent streams to be
>> written to the disk. The
>> bottleneck becomes the seek times and rotational latency.
>
> On 2/7/2010 23:00, Ben Kamen wrote:
>> Could always start striping drives in a raid 0 kinda deal. :D
>
> No. Striping causes the same exact problem. You still have all the disks
> seeking in unison, you just end up with significantly higher throughput.
> If you've got proper hardware RAID with write-back cache, it may be able
> to re-order the writes to be more sequential. The whole purpose of
> Storage Groups added in 0.21 was to allow you to add multiple
> independent drives to MythTV, and allow it to properly load balance
> between them.

I wasn't saying in Software (although I didn't exactly except it) - it would most definitely be in hardware.
And that hardware would have to have a delayed-write caching mechanism.
I wonder how many modern MB with RAID capable SATA controllers support such a thing.

I'm using the Myth storage group option having added (2) 250GB drives making sure they were on different IDE busses.
It's fabulous.

-Ben
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

travis at tabbal

Feb 8, 2010, 9:01 AM

Post #7 of 22 (1905 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

>
>
> Will the new sda1 partition be large enough, or is 60GB too big?
>

What's it for? The OS/DB? If so, drop it and put the OS/DB on a different
drive. An old IDE drive will work fine, SSD if you really want performance.
I used an old 40GB IDE drive from an old TiVo, it worked great. With 12
tuners you might want something a little more speedy for the DB though.

Sharing the OS and DB with the recordings is a recipe for lots of seeks,
hosing your I/O speed. Put your swap on the OS drive as well. For the size
of the system you're talking about, the disks holding recordings should hold
ONLY recordings. I'd partition them into a single partition with XFS. Don't
share spindles, it's not worth it. Why the live TV partition? Just let Myth
balance the I/O by putting the whole-drives in the same storage group. If
you find your I/O system can't keep up, add more disks to the storage group.
For this reason, unless you need huge space, I'd buy 500G or smaller drives
and just use more of them. If you can't fit more disks in there, time to set
up a dedicated backend box.

I see XFS, and EXT4, talked about a lot for large file types. Which of
> these, or possibly another, would be the best choice for the recordings
> directory?
>

EXT4 does pretty well with large files, but I still like XFS for recordings.
Set the pre-allocation size up around 500M and you will keep fragmentation
low. Again, keeps seeking down.

> I'm thinking FAT32 on the USB drive for portability, if I ever want to
> mount it temporarily on another system. Shouldn't be a problem for MythTV,
> should it?
>

It will work, but be more fragile. Back up the data somewhere just in case.
Which you should be doing anyway.

> Lastly, I have 2GB of RAM installed in this system, should I even bother
> with the swap partition?

My backend has 2GB and rarely uses any swap. It's also running other
processes. So you might be able to get away without swap, I generally put it
in anyway though. If a process needs to use it, it generally doesn't just it
long.

jaw1959 at gmail

Feb 8, 2010, 10:19 AM

Post #8 of 22 (1900 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

My backend has 2GB and rarely uses any swap. It's also running other
processes. So you might be able to get away without swap, I generally put it
in anyway though. If a process needs to use it, it generally doesn't just it
long.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

I have a backend/frontend running Ubuntu 9.10, and the response of the
system would get quite sluggish after running a while. I found ghat setting
"swappiness = 0" made a major difference (google "swappiness" for
specifics). Now my backend never uses swap unless it absolutly needs to.
My machine has 4GB of ram, so your results may vary. My machine generally
uses 1.8gb of ram for programs, and the rest for cache (I only have 3.6gb of
my 4.0gb available; my onboard video uses the rest)

Good luck

beckett.jim at gmail

Feb 8, 2010, 12:44 PM

Post #9 of 22 (1881 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Travis Tabbal wrote:
>
>
> Will the new sda1 partition be large enough, or is 60GB too big?
>
>
>
> What's it for? The OS/DB? If so, drop it and put the OS/DB on a
> different drive. An old IDE drive will work fine, SSD if you really
> want performance. I used an old 40GB IDE drive from an old TiVo, it
> worked great. With 12 tuners you might want something a little more
> speedy for the DB though.
>
> Sharing the OS and DB with the recordings is a recipe for lots of
> seeks, hosing your I/O speed. Put your swap on the OS drive as well.
> For the size of the system you're talking about, the disks holding
> recordings should hold ONLY recordings. I'd partition them into a
> single partition with XFS. Don't share spindles, it's not worth it.
> Why the live TV partition? Just let Myth balance the I/O by putting
> the whole-drives in the same storage group. If you find your I/O
> system can't keep up, add more disks to the storage group. For this
> reason, unless you need huge space, I'd buy 500G or smaller drives and
> just use more of them. If you can't fit more disks in there, time to
> set up a dedicated backend box.
>
>
> I see XFS, and EXT4, talked about a lot for large file types.
> Which of these, or possibly another, would be the best choice for
> the recordings directory?
>
>
>
> EXT4 does pretty well with large files, but I still like XFS for
> recordings. Set the pre-allocation size up around 500M and you will
> keep fragmentation low. Again, keeps seeking down.
>
>
>
> I'm thinking FAT32 on the USB drive for portability, if I ever
> want to mount it temporarily on another system. Shouldn't be a
> problem for MythTV, should it?
>
>
>
> It will work, but be more fragile. Back up the data somewhere just in
> case. Which you should be doing anyway.
>
>
>
> Lastly, I have 2GB of RAM installed in this system, should I even
> bother with the swap partition?
>
>
>
> My backend has 2GB and rarely uses any swap. It's also running other
> processes. So you might be able to get away without swap, I generally
> put it in anyway though. If a process needs to use it, it generally
> doesn't just it long.
> ------------------------------------------------------------------------
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
Travis,

Great points! This is exactly the kind of advice I was seeking...

The answer to the question of why I have a LiveTV partion is, 'I don't
know, It seemed like a good idea at the time'... IIRC, I think I was
trying to grasp the 'storage groups' concept at the time-
http://www.mythtv.org/wiki/MythVideo_.22_Transition_Guide

I do have a 320GB IDE drive in a computer in storage that I can use for
the OS, DB, SWAP, and Multimedia, that is on the USB disk now. I don't
know why this didn't occur to me before... (I seem to be racking up the
"DUH" moments lately...)

Using this drive would allow me to use both of the SATA drives for
recordings only, and provides enough extra space that I could use my USB
drive as a portable again, which eliminates my need for shared data
directory.

Doing it this way, my new configuration would look like this:

/dev/hda (IDE internal disk) - 320GB IDE (298GB formatted)

hda1, 96GB, mount point /, Ext3
hda2, 200GB, mount point /multimedia, XFS (for videos, music, and
pictures)
hda4, 2GB, SWAP

/dev/sda (first SATA internal disk) - 500.1GB SATA (460GB formatted)

sda1, 460GB, mount point /recordings1, XFS

/dev/sdb (second internal disk) - 500.1GB SATA (460GB formatted)

sdb1, 400GB, mount point /recordings2, XFS

Which strikes me as a very simple way to get the most out of my existing
drives...

So, this is the route I think I want to take, unless there is a
compelling reason not to do it this way.

Thank you!

-Jim

beckett.jim at gmail

Feb 8, 2010, 12:47 PM

Post #10 of 22 (1883 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Josh White wrote:
>
> I have a backend/frontend running Ubuntu 9.10, and the response of the
> system would get quite sluggish after running a while. I found ghat
> setting "swappiness = 0" made a major difference (google "swappiness"
> for specifics). Now my backend never uses swap unless it absolutly
> needs to. My machine has 4GB of ram, so your results may vary. My
> machine generally uses 1.8gb of ram for programs, and the rest for
> cache (I only have 3.6gb of my 4.0gb available; my onboard video uses
> the rest)
>
> Good luck

Thanks for bringing swappiness up.

I would probably have never encountered that, and I know I would have
never tried to find it by name.

I haven't noticed the system using swap to speak of, but I expect I'll
play around with it anyway.

- Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

cpinkham at bc2va

Feb 8, 2010, 7:13 PM

Post #11 of 22 (1864 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

* On Mon Feb 08, 2010 at 12:44:07PM -0800, Jim Beckett wrote:
> /dev/sda (first SATA internal disk) - 500.1GB SATA (460GB formatted)
>
> sda1, 460GB, mount point /recordings1, XFS
>
> /dev/sdb (second internal disk) - 500.1GB SATA (460GB formatted)
>
> sdb1, 400GB, mount point /recordings2, XFS

For Storage Groups, you never want to set the Storage Group to point at
the root directory of the filesystem. You want to point the Storage Group
at a subdirectory on the filesystem. This way if the filesystem is not
mounted properly for some reason, MythTV will detect this since the Storage
Group directory doesn't exist and will not use that directory instead of
recording to the mount point itself potentially filling up your root
partition.

Something like this:

sda1 -> /recordings1
sdb1 -> /recordings2

and your Storage Group points at:

/recordings1/mythtv and
/recordings2/mythtv

or something like that.

The mount points on my fileserver are setup like this:

/nas/video1
/nas/video2
/nas/video3
/nas/video4
/nas/video5

with my Storage Group dirs of:

Default -> /nas/video1/mythtv/recordings
Default -> /nas/video2/mythtv/recordings
Default -> /nas/video3/mythtv/recordings
Default -> /nas/video4/mythtv/recordings
Default -> /nas/video5/mythtv/recordings
(I have the 'mythtv' extra dir in there for my production setup because I
also have a 'dev' equivalent on the same fileserver)

Videos -> /nas/video1/movies/
Videos -> /nas/video5/movies/
Fanart -> /nas/video1/images/fanart
Banners -> /nas/video1/images/banners
Screenshots -> /nas/video1/images/screenshots
Coverart -> /nas/video1/images/coverart

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

beckett.jim at gmail

Feb 8, 2010, 9:12 PM

Post #12 of 22 (1850 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Chris Pinkham wrote:
> For Storage Groups, you never want to set the Storage Group to point at
> the root directory of the filesystem. You want to point the Storage Group
> at a subdirectory on the filesystem. This way if the filesystem is not
> mounted properly for some reason, MythTV will detect this since the Storage
> Group directory doesn't exist and will not use that directory instead of
> recording to the mount point itself potentially filling up your root
> partition.
Chris,

Thanks for this explanation above.

I don't remember why I was prompted to do it this way originally, but I did.

I ended up using the super-duper creative directory name 'files':

/recordings/files/
/recordings2/files/
/videos/files
/music/files
...etc.

Is there a technical reason that the fanart, banners, screenshots, and
coverart directories are all two levels down? Or, is it just aesthetic
preference?

/>Videos -> /nas/video5/movies/
>Fanart -> /nas/video1/images/fanart
>Banners -> /nas/video1/images/banners
>Screenshots -> /nas/video1/images/screenshots
>Coverart -> /nas/video1/images/coverart/

Considering the new disk configuration:

/dev/hda

hda1, 96GB, mount point /, Ext3
hda2, 200GB, mount point /multimedia, XFS
hda3, 2GB, SWAP

/dev/sda

sda1, 460GB, mount point /recordings1, XFS

/dev/sdb

sdb1, 400GB, mount point /recordings2, XFS

...the storage groups (with matching directory structure on disk) should
end up like this:

Default -> /recordings/files/
Default -> /recordings2/files/
Music -> /multimedia/files/music/
Pictures -> /multimedia/files/pictures/
Videos -> /multimedia/files/videos/
Fanart -> /multimedia/files/images/fanart/
Banners -> /multimedia/files/images/banners/
Screenshots -> /multimedia/files/images/screenshots/
Coverart -> /multimedia/files/images/coverart/

Again, thanks to you, and everyone else who has responded, for all the
assistance!

- Jim

cpinkham at bc2va

Feb 8, 2010, 10:43 PM

Post #13 of 22 (1836 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

* On Mon Feb 08, 2010 at 09:12:16PM -0800, Jim Beckett wrote:
> Is there a technical reason that the fanart, banners, screenshots, and
> coverart directories are all two levels down? Or, is it just aesthetic
> preference?

No particular reason, just preference.

> Music -> /multimedia/files/music/

This one won't be around until 0.24 at least. :) As far as I know, MythMusic
has been modified to work with Storage Groups yet, but it is in the plan
I believe, including streaming from remote SG's just like MythVideo can now
do.

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

cpinkham at bc2va

Feb 8, 2010, 10:56 PM

Post #14 of 22 (1834 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

* On Tue Feb 09, 2010 at 01:43:52AM -0500, Chris Pinkham wrote:
> * On Mon Feb 08, 2010 at 09:12:16PM -0800, Jim Beckett wrote:
> > Is there a technical reason that the fanart, banners, screenshots, and
> > coverart directories are all two levels down? Or, is it just aesthetic
> > preference?
>
> No particular reason, just preference.
>
> > Music -> /multimedia/files/music/
>
> This one won't be around until 0.24 at least. :) As far as I know, MythMusic
> has been modified to work with Storage Groups yet, but it is in the plan
> I believe, including streaming from remote SG's just like MythVideo can now
> do.

Sorry for any confusion, I meant "MythMusic hasn't been modified".

--
Chris
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mythtv at ncc1701

Feb 8, 2010, 11:16 PM

Post #15 of 22 (1840 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

>
> So in order to clean up my storage issues, I think I would probably
> want
> to have it set up like this, and used balanced IO in MythTV for
> recordings:
>
> /dev/sda (first internal disk) - 500.1GB SATA ~73 MB/s disk reads
>
> sda1, 60GB, mount point /, Ext3
> sda2, 400GB, mount point /recordings, XFS
>
> /dev/sdb (second internal disk) - 500.1GB SATA ~73 MB/s disk reads
>
> sdb1, 4GB, swap space
> sdb2, 56GB, mount point /livetv, XFS
> sdb3, 400GB, mount point /recordings2, XFS
>
> /dev/sdc (USB 2.0 - 500.1GB External drive) ~31 MB/s disk reads
>
> sdc1, 400GB, mount point /media/disk, FAT32 ***
> sdc2, 60GB, mount point /datashare, FAT32
>

Opinions vary on this topic, but on a single disk, breaking it into
multiple
partitions increases seek time significantly.

For example, /dev/sdb2 = livetv and /dev/sdb3 = recordings..

If you watch live tv WHILE recording, the disk head will need to seek
from
one extreme to another.

Keep the minimum number of partitions, one per disk if at all possible.

Use multiple disks, or directories within a partition to separate
recordings.

BTW, Why do you separate LIVE TV from RECORDINGS? I treat them as the
same
thing, because fundamentally they are.

Recorded Live TV wont 'get in the way' of scheduled recordings because
Myth
will aggressively expire them.

That'll be 2c.

Dave

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

raymond at wagnerrp

Feb 8, 2010, 11:21 PM

Post #16 of 22 (1841 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On 2/9/2010 02:16, MythTV wrote:
> Opinions vary on this topic, but on a single disk, breaking it into
> multiple
> partitions increases seek time significantly.
>
> For example, /dev/sdb2 = livetv and /dev/sdb3 = recordings..
>
> If you watch live tv WHILE recording, the disk head will need to seek
> from
> one extreme to another.
>

That makes absolutely no sense. The disk doesn't care how your
partitions are set up. It's going to have to seek from one file to the
other regardless.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mythtv at ncc1701

Feb 8, 2010, 11:32 PM

Post #17 of 22 (1834 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

> > I have a backend/frontend running Ubuntu 9.10, and the response of
> the
> > system would get quite sluggish after running a while. I found ghat
> > setting "swappiness = 0" made a major difference (google
"swappiness"
> > for specifics). Now my backend never uses swap unless it absolutly
> > needs to. My machine has 4GB of ram, so your results may vary.
My
> > machine generally uses 1.8gb of ram for programs, and the rest for
> > cache (I only have 3.6gb of my 4.0gb available; my onboard video
uses
> > the rest)
> >
> > Good luck

Googling swappiness brought the following:

You can make some tweaks to your /etc/sysctl.conf file in order to free
up memory faster. As an example I have these for my MySQL database that
MythTV uses:

# Control the degree the kernel prefers to swap when trying to free
memory (default 60)
vm.swappiness=0

# Reclaim cache memory faster for directory and inode objects (default
100)
vm.vfs_cache_pressure=10000

# Let dirty data consume this percentage of memory (default 10)
vm.dirty_ratio = 1

# Let dirty active data consume this percentage of memory (default 5)
vm.dirty_background_ratio = 1

# Wake up pdflush to write to disc (default 500)
vm.dirty_writeback_centisecs = 250

# How old data can be before it should be considered dirty and written
to disk (5min) (default 3000)
vm.dirty_expire_centisecs = 3000

I've applied them to try. Sounds like they limit the number of dirty
pages, but increase the
amount of time they are allowed to stay in memory.

Any comments?

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

mythtv at ncc1701

Feb 8, 2010, 11:35 PM

Post #18 of 22 (1833 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

> -----Original Message-----
> From: mythtv-users-bounces [at] mythtv [mailto:mythtv-users-
> bounces [at] mythtv] On Behalf Of Raymond Wagner
> Sent: Tuesday, 9 February 2010 6:22 PM
> To: Discussion about mythtv
> Subject: Re: [mythtv-users] Planning Tuners and Disk I/O for a
> MythTVBackend System
>
> On 2/9/2010 02:16, MythTV wrote:
> > Opinions vary on this topic, but on a single disk, breaking it into
> > multiple
> > partitions increases seek time significantly.
> >
> > For example, /dev/sdb2 = livetv and /dev/sdb3 = recordings..
> >
> > If you watch live tv WHILE recording, the disk head will need to
seek
> > from
> > one extreme to another.
> >
>
> That makes absolutely no sense. The disk doesn't care how your
> partitions are set up. It's going to have to seek from one file to
the
> other regardless.

That makes absolutely no sense.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

raymond at wagnerrp

Feb 8, 2010, 11:44 PM

Post #19 of 22 (1840 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On 2/9/2010 02:35, MythTV wrote:
>> -----Original Message-----
>> From: mythtv-users-bounces [at] mythtv [mailto:mythtv-users-
>> bounces [at] mythtv] On Behalf Of Raymond Wagner
>> Sent: Tuesday, 9 February 2010 6:22 PM
>> To: Discussion about mythtv
>> Subject: Re: [mythtv-users] Planning Tuners and Disk I/O for a
>> MythTVBackend System
>>
>> On 2/9/2010 02:16, MythTV wrote:
>>
>>> Opinions vary on this topic, but on a single disk, breaking it into
>>> multiple
>>> partitions increases seek time significantly.
>>>
>>> For example, /dev/sdb2 = livetv and /dev/sdb3 = recordings..
>>>
>>> If you watch live tv WHILE recording, the disk head will need to
>>>
> seek
>
>>> from
>>> one extreme to another.
>>>
>>>
>> That makes absolutely no sense. The disk doesn't care how your
>> partitions are set up. It's going to have to seek from one file to
>>
> the
>
>> other regardless.
>>
> That makes absolutely no sense.
>

The disk is going to have to seek any time you do anything other than
sequential access. If you are reading one file while writing another,
the disk will keep seeking back and forth between those two files. It
makes no difference whether they're on the same or separate partitions.
Claiming partitioning will increase, decrease, or in any way affect how
much you seek is blatantly false.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

stevehodge at gmail

Feb 9, 2010, 12:16 AM

Post #20 of 22 (1835 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

On Tue, Feb 9, 2010 at 8:44 PM, Raymond Wagner <raymond [at] wagnerrp> wrote:

> The disk is going to have to seek any time you do anything other than
> sequential access. If you are reading one file while writing another, the
> disk will keep seeking back and forth between those two files. It makes no
> difference whether they're on the same or separate partitions. Claiming
> partitioning will increase, decrease, or in any way affect how much you seek
> is blatantly false.

Seek time does depend on how far the head has to move. If two files are in a
separate partitions then you are guaranteed to have to seek the distance
between the two partitions every time you switch from accessing one file to
the other. If they are in the same partition (and the single partition
covers the same area as the two separate partitions did) then the two files
may be closer or further than that, depending on how the filesystem is laid
out. It'd depend mainly how full the partition is and how many other files
were written between the two you're accessing. It is certainly false to say
that it makes no difference whether they're on the same or separate
partitions.

Cheers,
Steve

beckett.jim at gmail

Feb 9, 2010, 12:47 AM

Post #21 of 22 (1829 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

Chris Pinkham wrote:
> * On Tue Feb 09, 2010 at 01:43:52AM -0500, Chris Pinkham wrote:
>
>> * On Mon Feb 08, 2010 at 09:12:16PM -0800, Jim Beckett wrote:
>>
>>> Music -> /multimedia/files/music/
>>>
>> This one won't be around until 0.24 at least. :) As far as I know, MythMusic
>> has been modified to work with Storage Groups yet, but it is in the plan
>> I believe, including streaming from remote SG's just like MythVideo can now
>> do.
>>
>
> Sorry for any confusion, I meant "MythMusic hasn't been modified"
No confusion here. Thanks again for the heads up. I guess I'll be ready
for it when/if it gets here!

beckett.jim at gmail

Feb 9, 2010, 1:32 AM

Post #22 of 22 (1818 views)
Permalink

Re: Planning Tuners and Disk I/O for a MythTVBackend System [In reply to]

MythTV wrote:
> BTW, Why do you separate LIVE TV from RECORDINGS? I treat them as the
> same
> thing, because fundamentally they are.
>
> Recorded Live TV wont 'get in the way' of scheduled recordings because
> Myth
> will aggressively expire them.
>
> That'll be 2c.
>
> Dave
I honestly don't remember why I put a live tv partition in there.

I think it must have been one of those, "It seemed like a good idea at
the time", type of things.

I think I was trying to guarantee a certain amount of space for other
partitions.

I did this when I was trying to understand the concept of storage
groups, and took it way too far...

I do know that I made the disk partitioning scheme a lot more
complicated than it needed to be...

Travis replied earlier with a post about him using an old IDE drive for
the OS.

That reminded me that I have reasonably large IDE disk in storage that
will be better suited for the system than the USB disk.

So, I'm reclaiming the USB disk from this system, and using the IDE disk
instead.

The simplified configuration is going to look like this:

/dev/hda

hda1, 96GB, mount point /, Ext3 (OS, DB, mythweb, etc.)
hda2, 200GB, mount point /multimedia, XFS (for videos, coverart,
banners, fanart, music, pictures, etc.)
hda3, 2GB, SWAP

/dev/sda

sda1, 460GB, mount point /recordings1, XFS (just recordings)

/dev/sdb

sdb1, 400GB, mount point /recordings2, XFS (just recordings)

Thanks,

-Jim
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads